[EMAIL PROTECTED] wrote:
Hi, Andrzej,
On Tue, Jul 06, 2004 at 03:02:46PM +0200, Andrzej Bialecki wrote:
Benefits are obvious: saves bandwidth and CPU for parsing, and also gives an important information about how quickly the resource is changing.
Yes, it's the time to have this feature. Furthermore, information collected this way is by itself valuable. To me, there may be a need to store such info in its own data file (database), rather than in ./segments/*/fetchlist or ./segments/*/fetcher. Doing so opens door to other possibilities, besides obvious flexibility. Some tools will have to be modified accordingly. That won't be very difficult.
I don't think it is required to introduce another segment part, if you're not interested in keeping historical values of the fetch interval. The current combination of getFetchInterval() and getNextFetchTime() is sufficient to store the current values required by the algorithm.
-- Best regards, Andrzej Bialecki
------------------------------------------------- Software Architect, System Integration Specialist CEN/ISSS EC Workshop, ECIMF project chair EU FP6 E-Commerce Expert/Evaluator ------------------------------------------------- FreeBSD developer (http://www.freebsd.org)
-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
Nutch-developers mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
