Well actually it's the same thing i noticed crawldb operations are slooow,
actually generating and then updating the database takes almost as long as
crawling. I'm not sure if there's something i can do about it. I tried
generating huge segments like 500 000 urls each but that's not very good if
the crawl fails for some reason.
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
