Re: exception during fetch using hadoop

2006-02-24 Thread Mike Smith
Hi Doug, Unfortunately my core limit was 0 at that time and I am running another configuration with smaller number of threads since last night and it has not crashed yet. As soon as it fails I will send you more detailed log information. But this is usually happen: I'v been using three three mach

Re: exception during fetch using hadoop

2006-02-24 Thread Doug Cutting
It looks like the child JVM is silently exiting. The "error reading child output" just shows that the child's standard output has been closed, and the "child error" says the JVM exited with non-zero. Perhaps you can get a core dump by setting 'ulimit -c' to something big. JVM core dumps ca

exception during fetch using hadoop

2006-02-23 Thread Mike Smith
I have been getting this exception during fetching for almost a month. This exception stops the whole crawl. It happens on and off! Any Idea?? We are really stocked with this problem. I am using 3 data node and 1 name server. 060223 173809 task_m_b8ibww fetching http://www.heartcenter.com/94fall