After many tries...the problem seems solved!
I've changed the hadoop-site.xml file adding these lines:
<property>
<name>mapred.speculative.execution</name>
<value>false</value>
</property>
I hope this will help someone else!!
Thanks
Luca Rondanini
Research and Development
[EMAIL PROTECTED]
Tel: +39 06 91 62 00 55
Fax: +39 06 233 200 102
http://www.translated.net
Luca Rondanini wrote:
> Hi all,
>
> First of all....I've read all the posts regarding this problem in the
> mailing list!! :)
>
> I'm try to index more than 200k documents. I'm reading those documents
> through an nfs mount partition. Everything seems fine till we arrive at
> 40k-50k documents....then the fetcher fails with the error "Hung Threads"!!
>
> These are the configurations that i've tried:
>
> 1) topN=20.000
> fetcher.threads=10
> ulimit -n=1024
> MergeFactor=20
> file.limit=1M
>
> ----> Hung Threads
>
> 2) topN=5000
> fetcher.threads=10
> ulimit -n=1024
> MergeFactor=20
> file.limit=1M
>
> ----> Hung Threads
>
>
> 3) topN=5000
> fetcher.threads=5
> ulimit -n=1024
> MergeFactor=20
> file.limit=1M
>
> ----> Too many open file
>
>
> 4) topN=5000
> fetcher.threads=5
> ulimit -n=4096
> MergeFactor=10
> file.limit=1M
>
> ----> Hung Threads
>
>
>
> Can anyone please give me a clue as to what is going on?!?
> Thanks,
> Luca
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
