I actually tried but couldn't get it to compile when I tried.
It's the tmpdir that fills up the sorted.0 and sorted.1 files get really big (>100 gigs each).
I'm running on Linux 2.6.9 on an opteron box:
Linux jeb 2.6.9-gentoo-r14 #1 SMP Tue Mar 8 12:05:13 PST 2005 x86_64 AMD Opteron(tm) Processor 242 AuthenticAMD GNU/Linux
Java version: java version "1.5.0_01" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_01-b08) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_01-b08, mixed mode)
For fetchlist, I just grabbed one of the larger ones: http://bourg.net/~gus/fetchlist.tar.gz
Thanks, Gus
On Thu, 17 Mar 2005, Stefan Groschupf wrote:
Can you please try this with the latest code in subversion.
I can not reproduce this problem, I used the 0.6 release many times but never note such an problem.
Please try with the latest code and in case the problem does still occurs, post the OS, Java version, and a compressed fetch list, so may people can reproduce the problem.
Am 16.03.2005 um 21:58 schrieb Gus Bourg:
No answer on this? :(
Gus
On Fri, 11 Mar 2005, Gus Bourg wrote:
New user, sorry if this has already been discussed. I'm doing whole web indexing on a dual opteron with 250 gigs of space. My segment directory is about 7.5 gigs and my db directory is about 2.1 gigs. I'm running version 0.6.
My problem is that when I go to run bin/nutch/analyze db 2 it runs out of space. Is it normal that it'll eat up 200+ gigs for a 7.5 gigs worth of segments?
--------------------------------------------------------------- company: http://www.media-style.com forum: http://www.text-mining.org blog: http://www.find23.net
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Nutch-general mailing list Nutch-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nutch-general
