Is there a way to programatically get at Nutch's robots.txt cache after a fetch? I have other non-nutch fetchers that operate on the same URLs and it'd be silly to waste the accesses.
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
