[RFCI-Discuss] A Great Day... I Think

[Derek J. Balling]

On Dec 5, 2005, at 2:58 PM, Alex van den Bogaerdt wrote:
> Create a linked list, no beginning no end.  Start at #1, process as
> many as will fit in the amount of time you consider proper, continue
> next night where you stopped.
>
> Insert new entries just before the next one to process (i.e. they
> go at the very end of the list).  Remove entries from the chain as
> appropriate.

My thinking was actually something similar:

Create a "last_checked" timestamp. Order by last_checked in ascending  
order, limit "X" (the "X" to be determined). When a site is inserted,  
"last_checked" would be set to the same as submitted/accepted times.  
Do "X" many, finish for the night, re-do that query the following  
night. As the number of listings grows, the number of nights it takes  
to finish grows, etc.

> Should there be a need to permanently keep a record in the database,
> remove it from the chain but not from the database.

Or, in that case, I set the last_checked timestamp to something far  
in the future.

> Variants are possible, such as:
> - use spare cpu cycles and network space (ala seti at home)
> - distribute processing (also ala seti at home; sorry, I can't  
> participate)

I can't/won't trust others. It's a control issue. ;-)

> - process as fast as you can from midnight to 5am, much slower the  
> rest
>   of the time

Well, similarly, I don't want to necessarily consume assloads of  
donated bandwidth ....

D


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2365 bytes
Desc: not available
Url : http://lists.megacity.org/pipermail/rfci-discuss/attachments/20051205/7ea1b3e8/smime.bin