If fetch failure happens I always create the fetcher.done file and slice the segment into a new one. Can I skip my usual procedure and just merge everything?
-----Original Message----- From: Andrzej Bialecki [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 10, 2005 2:07 PM To: [email protected] Subject: Re: Collapsing segments Wilkerson, Cory wrote: > All, > > Sorry for the quasi-repost. > > Is there anyway to manually collapse segments into a single searchable > segment? I had to kill a crawl after building nearly 100MB worth of > index data that I'd *love* to be able to search against - but it seems > like I need nutch to do some sort of cleanup before it's usable. Good news: partial segments are usable as is, in the worst case the processing speed (indexing/searching etc) is slower. Even better news: the SegmentMergeTool (available as "mergesegs" command from the nutch script) does exactly this - it merges several segments into one, and much more. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com ------------------------------------------------------- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
