Mike Smith wrote: > I am in the same state again, and same reduce jobs keep failing on > different > machines. I cannot get the dump using kill -3 pid, it does not make the > thread to quit. Also, I tried to place some log into FetcherOutputFormat, > but because of this bug: > *https://issues.apache.org/jira/browse/HADOOP-406*<https://issues.apache.org/jira/browse/HADOOP-406> > > > The logging is not possible in the childs threads. Do you have any > idea why > the reducers doesn't catch the QUIT signal from the cache. I am > running the > latest version on SVN, otherwise I could log some key,value and url > filtering information at the reduce stage.
SIGQUIT should not make the JVM quit, it should produce a thread dump on stderr. You need to manually pick up the process that corresponds to the child JVM of the task, e.g. with top(1) or ps(1), and then execute 'kill -SIGQUIT <pid>'. You can use Hadoop's log4j.properties to quickly enable a lot of log info, including stderr - put it in conf on every tasktracker and restart the cluster. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
