MG>below
________________________________ From: Siegfried Goeschl <siegfried.goes...@it20one.com> Sent: Sunday, May 28, 2017 3:41 PM To: Commons Users List Cc: cgama...@gmail.com Subject: Re: Proper use of Executors Hi Chris, there are couple of things to consider * You are using a PumpStreamHander but the STDERR is not consumed. Each process has an internal buffer (size depends on the OS) and when the buffer is full any write to STDERR is blocked * That could happen if the process being executed actually writes some error messages :-) * Are you 100% sure that the processes will terminate? See ExecuteWatchdog * You might habe a look at ProcessDestroyer to cleanup during shutdown MG>correct ThreadMonitor needs a watchdog for timeout e.g. ExecuteWatchdog watchDog = new ExecuteWatchdog(60L * 60L * 1000L); MG>once timeout is achieved you will need ProcessDestroyer to kill the process ShutdownHookProcessDestroyer processDestroyer = new ShutdownHookProcessDestroyer(); MG>Executor needs to identify its watchdog and ProcessDestroyer executor.setStreamHandler(streamHandler); //PumpStreamHandler executor.setWorkingDirectory(new File(getExecutionDirectory())); //which directory to write to executor.setProcessDestroyer(processDestroyer); //what is processDestroyer executor.setWatchdog(watchDog); //identify watchdog // Async Executor doesn't block. DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler(); //Executor HashMap // Inherit the current process's environment variables and add the user-defined ones @SuppressWarnings("unchecked") Map<String, String> newEnvironment = EnvironmentUtils.getProcEnvironment(); newEnvironment.putAll(this.environment); //place to stuff the results DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler(); //use executor to execute commandLine, environment HashHap executor.execute(command, newEnvironment, resultHandler); //return results return resultHandler; http://www.programcreek.com/java-api-examples/index.php?api=org.apache.commons.exec.DefaultExecuteResultHandler Java Code Example org.apache.commons.exec ...<http://www.programcreek.com/java-api-examples/index.php?api=org.apache.commons.exec.DefaultExecuteResultHandler> www.programcreek.com This page provides Java code examples for org.apache.commons.exec.DefaultExecuteResultHandler. The examples are extracted from open source Java projects from GitHub. Thanks in advance, Siegfried Goeschl > On 27 May 2017, at 14:27, Chris Gamache <cgama...@gmail.com> wrote: > > Hi all, > > I'm using org.apache.commons:commons-exec:1.3 on Java 8. > > I'm having an issue where my Tomcat server is bleeding out hundreds of > threads and all of the memory in the form of Executors that I'm running but > don't seem to be closing down ... When the server finally grinds to a halt > I have to restart. When I do it looks like this at shutdown time: > > <snip> > > 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11] > org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The > web application [ROOT##000252] appears to have started a thread named [Exec > Default Executor] but has failed to stop it. This is very likely to create > a memory leak. Stack trace of thread: > java.lang.Object.wait(Native Method) > java.lang.Object.wait(Object.java:502) > java.lang.UNIXProcess.waitFor(UNIXProcess.java:396) > org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:364) > org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) > org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) > java.lang.Thread.run(Thread.java:745) > > 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11] > org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The > web application [ROOT##000252] appears to have started a thread named [Exec > Stream Pumper] but has failed to stop it. This is very likely to create a > memory leak. Stack trace of thread: > java.io.FileInputStream.readBytes(Native Method) > java.io.FileInputStream.read(FileInputStream.java:255) > java.io.BufferedInputStream.fill(BufferedInputStream.java:246) > java.io.BufferedInputStream.read1(BufferedInputStream.java:286) > java.io.BufferedInputStream.read(BufferedInputStream.java:345) > java.io.FilterInputStream.read(FilterInputStream.java:107) > org.apache.commons.exec.StreamPumper.run(StreamPumper.java:107) > java.lang.Thread.run(Thread.java:745) > > </snip> > > And my thread dump is a mile long. > > I am certainly willing to concede I'm Doing It Wrong(tm) ... Here's the > relevant code. It is called from a regular method in a regular class, > nothing fancy: > > CommandLine cmdLine = CommandLine.parse(command.toString()); > DefaultExecutor executor = new DefaultExecutor(); > PumpStreamHandler esh = new PumpStreamHandler(os,null,is); > executor.setStreamHandler(esh); > executor.execute(cmdLine); > > `is` and `os` are passed in on the constructor. Their opens and closes are > managed well and cleaned up on the outside of this class... > Are there further steps I'm missing to ensure the threads I'm creating are > getting shut down properly and the resources they are using are being > returned? > > Any help is much appreciated.