Daily Rotation Search and Hot Topics Progress

Shell City and/or Daily Rotation -- Got something to say?

Daily Rotation Search and Hot Topics Progress

Postby bob » 02/04/10, 9:53 pm

OK, this time I have it. I've concocted a script that removes duplicates from the database. Not implemented yet, but thoroughly tested. This one works -- solves the problem of flat out dupes, and dupes from the same story being posted multiple days. Also solves the problem of sites making slight changes to the title for the same story throughout the day... but that's a little bit iffier, depends on what they change and the nature of their links.

This should do wonders for the Hot Topics scripting too, since duplicates won't be counted as two hits anymore, but I need to work with that a little more before I know whether I can fully automate the process or not.

Should be fully implemented by Monday.
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby bob » 02/08/10, 4:12 pm

OK, think I have it. Tell me if you see any repetition. Some hincky points not visible to users... Thinking them through to clear them up.

Also, as predicted, Hot Topics is playing nicer -- see this:

http://www.dailyrotation.com/news/hottopics.php

Still needs work. I might go with a one word model instead of word pairs -- might be a lot of day-to-day repetition though. For example, microsoft, apple and google would place high in the scores daily... Once again, have to think it through.
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby bob » 02/18/10, 12:25 am

Hot Topics - not sure I can make it automatic at all. No matter how complicated and screwey a scheme I come up with for automatically discovering hot topice, several of the so-called topics are nonsense.

http://www.dailyrotation.com/news/hot.php

OK, still, maybe I'll go to hand picking them. I might not need to change them very often... I've set up a beta attempt on my test page, here, right under the top10 headlines. See what you think:

http://www.dailyrotation.com/index2.php

My idea is that the first row will be hand picked and change every day or so... Or at least new topics added and removed, kind of daily.... The second row is pretty permanent, since it contains topics of pretty much ongoing interest.

Or maybe I can just come up with a semi-permanent set of topics.

Any suggestions?
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby bob » 02/28/10, 9:36 pm

>>>>Or maybe I can just come up with a semi-permanent set of topics.


Our "All The Latest" feature's really getting a lot of use. Over the last couple hours alone, 60 hits. Any ideas for topics to add to it? Currently it has: Apple, Google, Microsoft, Linux, Windows, Security, Smartphones, Open Source, Games
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby Gerry » 03/01/10, 2:13 am

Ubuntu
Gerry
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 5727
Joined: 12/04/01, 12:00 am
Location: Perth, Western Australia

Postby bob » 03/01/10, 5:36 am

I can do that --

There must be loads of more general terms that would result in turning up lots of headlines lots of people are interested in...?
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby Gerry » 03/01/10, 5:44 am

currently the searches are returning nothing
Gerry
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 5727
Joined: 12/04/01, 12:00 am
Location: Perth, Western Australia

Postby bob » 03/01/10, 6:30 am

I'm working on it--date problem, since it's 3-1-10 here, Perl's having a hard time figuring day -1, -2, -3 etc.... "All the Latest" is supposed to go back 4 days, including today. Getting late here though, and I'm getting tired.... I'll set up something temporary and work on the problem tomorrow.
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby Gerry » 03/01/10, 10:45 am

No problems mate. Not an issue for me, just letting you know.

If you wanted to stop issues like that it's pretty simple to set up a server on your own machine and then test the code locally before uploading it to the live server. Again just FYI in case it's what you want.
Gerry
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 5727
Joined: 12/04/01, 12:00 am
Location: Perth, Western Australia

Postby bob » 03/01/10, 10:22 pm

OK, got it fixed -- my original method was too simplistic, just subtracting from the current date....

This seems to work:



Code: Select all
my $timeYesterday = time - 24 * 60 * 60;
my ($year, $month, $day) = (localtime($timeYesterday))[5,4,3];
my $yd = sprintf ("%02d-%02d-%04d", $month+1, $day, $year+1900);


my $timeYesterday1 = time - 48 * 60 * 60;
my ($year, $month, $day) = (localtime($timeYesterday1))[5,4,3];
my $yd1 = sprintf ("%02d-%02d-%04d", $month+1, $day, $year+1900);


my $timeYesterday2 = time - 72 * 60 * 60;
my ($year, $month, $day) = (localtime($timeYesterday2))[5,4,3];
my $yd2 = sprintf ("%02d-%02d-%04d", $month+1, $day, $year+1900);



probably can be compressed a bit....

Thanks for the advice, Gerry. Actually I've got a notebook here I'm not using. Maybe I'll set it up to match my server, just for a test box.
Last edited by bob on 03/02/10, 12:09 am, edited 1 time in total.
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby bob » 03/01/10, 10:32 pm

BTW, Script above is thanks to

Thanks to http://luckylarry.co.uk/2009/07/getting ... localtime/
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby bob » 04/04/10, 12:55 am

Our "All the Latest" feature has gotten more than 7000 hits since we created it. I'd say that's worthwhile.

Any ideas for more terms to add to it? Currently it has: Apple, Google, Microsoft, Linux, Windows, Security, Smartphones, Open Source, Games, and Ubuntu. I think I might add "Censorship." Any others?
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis

Postby Gerry » 04/04/10, 11:09 am

Oh nice idea! I'd click that one. :)
I answer rhetorical questions for my own enjoyment.
Gerry
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 5727
Joined: 12/04/01, 12:00 am
Location: Perth, Western Australia

Postby bob » 04/14/10, 1:14 am

Censorship's not working out. I need to rethink that one.... Taking it down. Click on it and you'll see why.... My search engine doesn't lend itself to boolian searches and the word "censor" doesn't come near to covering all possibilities.

http://www.dailyrotation.com/cgi-bin/DB ... rds=CENSOR
WYSIWTF
bob
BIG GIANT HEAD I Get Free Beer
BIG GIANT HEAD I Get Free Beer
 
Posts: 7565
Joined: 12/03/01, 12:00 am
Location: St. Louis


Return to Site Stuff

Who is online

Users browsing this forum: No registered users and 1 guest

cron