Now I am doing just an intranet crawl, do I just do daily crawls? andy -----Original Message----- From: Gal Nitzan [mailto:[EMAIL PROTECTED] Sent: Thursday, January 26, 2006 9:05 AM To: [email protected] Subject: Re: After the crawl
Hi Andy, There are a few commands for keeping your index up 2 date. after 30 days (default, can be changed in nutch-site.xml) generate - generate a fetch list to be crawled in a new segments fetch - will fetch the pages listed in the fetchlist updatedb - update the web db with the list of pages and links found in the fetch process invertlinks - update the link db index - update your indexes dedup - remove duplicates merge - merge your indexes On Thu, 2006-01-26 at 05:54 -0500, Andy Morris wrote: > After running the initial crawl what command do I need to run on a > weekly or daily basis to keep my indexes up to date...is it "fetch" > > Andy > ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
