Hi all,
First of all....I've read all the posts regarding this problem in the
mailing list!! :)
I'm try to index more than 200k documents. I'm reading those documents
through an nfs mount partition. Everything seems fine till we arrive at
40k-50k documents....then the fetcher fails with the error "Hung Threads"!!
These are the configurations that i've tried:
1) topN=20.000
fetcher.threads=10
ulimit -n=1024
MergeFactor=20
file.limit=1M
----> Hung Threads
2) topN=5000
fetcher.threads=10
ulimit -n=1024
MergeFactor=20
file.limit=1M
----> Hung Threads
3) topN=5000
fetcher.threads=5
ulimit -n=1024
MergeFactor=20
file.limit=1M
----> Too many open file
4) topN=5000
fetcher.threads=5
ulimit -n=4096
MergeFactor=10
file.limit=1M
----> Hung Threads
Can anyone please give me a clue as to what is going on?!?
Thanks,
Luca
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
