In my search sites there are several pages that change very often. New links at the pages are added every minute. If I won't recrawl those frequently updated pages every 5 minutes, some links will be lost.
Other sites and pages do not change so often. They can be rechecked every several days or a week. Doesn't matter that much. Is there any way of making Nutch to recrawl the frequently updated pages every 5 minutes, and go to others rarely? Thanks for help. Marcin Okraszewski ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
