[ http://issues.apache.org/jira/browse/NUTCH-151?page=comments#action_12361242 ]
Paul Baclace commented on NUTCH-151: ------------------------------------ Analysis: CommandRunner uses CyclicBarrier is to synchronize the thread that does the exec (lets call it the main thread) with the io pipe threads. Before the barrier timeout occurs, the barrier causes the main thread to wait for all the io pipes to finish, which they do only when EOF occurs after the subprocess finishes. Each pipe sees the EOF and then uses the barrier synchronization. After all the pipes await the barrier, they and the main thread continue and everything is fine. If and only if the barrier timeout occurs, then the main thread uses Thread.interrupt() to tell each io pipe that it is time to finish up, even if EOF was not seen and bytes were still being pumped (this was the intention in the original code). After the barrier synchronization is finished and did not time out, if _waitForExit is true, the main thread gets the exit value (return code) from the subprocess by calling Process.exitValue() in a busy loop that throws an exceptions each time through the loop, which is makes the somewhat expensive busy loop (test, sleep, repeat) much, much more expensive (test, throw, catch, repeat, and no sleep!). For a quick running subprocess, no exception is thrown because it is finished; the exit value is retrieved and the caller can get it with getExitValue(). If a timeout occurred, no attempt is made to obtain the exit value (presumably because it has not exited). > CommandRunner can hang after the main thread exec is finished and has > inefficient busy loop > ------------------------------------------------------------------------------------------- > > Key: NUTCH-151 > URL: http://issues.apache.org/jira/browse/NUTCH-151 > Project: Nutch > Type: Bug > Components: indexer > Versions: 0.8-dev > Environment: all > Reporter: Paul Baclace > > I encountered a case where the JVM of a Tasktracker child did not exit after > the main thread returned; a thread dump showed only the threads named STDOUT > and STDERR from CommandRunner as non-daemon threads, and both were doing a > read(). > CommandRunner usually works correctly when the subprocess is expected to be > finished before the timeout or when no timeout is used. By _usually_, I mean > in the absence of external thread interrupts. The busy loop that waits for > the process to finish has a sleep that is skipped over by an exception; this > causes the waiting main thread to compete with the subprocess in a tight loop > and effectively reduces the available cpu by 50%. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
