Thanks for the follow up! On the flipside, the new fetcher process screams. I'm getting 20million a day easily without much cpu load (just gobs of bandwidth)
The piece i need to work on is the distributed webdb as its taking a loooooooooooooonnnnnnnnnng time to analyze the db (or do any db work for that matter!) -byron --- Doug Cutting <[EMAIL PROTECTED]> wrote: > Byron Miller wrote: > > Never mind, i see that the entire segment > directory > > has changed format.. > > Yes, unfortunately the architectural changes for the > upcoming 0.5 > release required incompatible changes to the file > format, and refetching > will be required when folks upgrade to this release. > > This should be an unusual event. I try hard to make > things > back-compatible, especially file formats. But in > this case that > couldn't be achieved without a lot of extra work. > > Sorry! > > Doug > > > ------------------------------------------------------- > This SF.Net email sponsored by Black Hat Briefings & > Training. > Attend Black Hat Briefings & Training, Las Vegas > July 24-29 - > digital self defense, top technical experts, no > vendor pitches, > unmatched networking opportunities. Visit > www.blackhat.com > _______________________________________________ > Nutch-developers mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/nutch-developers > ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
