[JIRA] (JENKINS-54513) Builds occupying executor but not running

[russell.gal...@gmail.com (JIRA)]

Title: Message Title

Russell Gallop edited a comment on JENKINS-54513   Re: Builds occupying executor but not running Just seen what seems to be this same issue again. On this occasion it was all one job running parallel on a couple of Linux machines. In this case the job was reporting: {code:java} [TASK1] Still waiting to schedule task [TASK2] Still waiting to schedule task [TASK3] Still waiting to schedule task [TASK4] Still waiting to schedule task {code} There were only 3 executors available for this and according to the Build Executor Status they have all been assigned to "<folder> » <job> #476 (part)". In this case there were blocked threads in PrintStream.java:805 but not the ones I saw before in PrintStream.java:355 (threads on master): {code:java} Executor #0 for <agent1> : executing PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=<folder>/<job>#476,label=<label>,context=CpsStepContext[56:node]:Owner[<folder>/<job>/476:<folder>/<job> #476],cookie=null,auth=null} "Executor #0 for <agent1> : executing PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=<folder>/<job>#476,label=<label>,context=CpsStepContext[56:node]:Owner[<folder>/<job>/476:<folder>/<job> #476],cookie=null,auth=null}" Id=5916973 Group=main BLOCKED on java.io.PrintStream@208dc6f1 owned by "jenkins.util.Timer [#9]" Id=80 at java.io.PrintStream.println(PrintStream.java:805) -  blocked on java.io.PrintStream@208dc6f1 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask$PlaceholderExecutable.run(ExecutorStepExecution.java:694) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Executor #0 for <agent2> : executing PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=<folder>/<job>#476,label=<label>,context=CpsStepContext[62:node]:Owner[<folder>/<job>/476:<folder>/<job> #476],cookie=null,auth=null} "Executor #0 for <agent2> : executing PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=<folder>/<job>#476,label=<label>,context=CpsStepContext[62:node]:Owner[<folder>/<job>/476:<folder>/<job> #476],cookie=null,auth=null}" Id=5917131 Group=main BLOCKED on java.io.PrintStream@107ec0b6 owned by "jenkins.util.Timer [#10]" Id=81 at java.io.PrintStream.println(PrintStream.java:805) -  blocked on java.io.PrintStream@107ec0b6 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask$PlaceholderExecutable.run(ExecutorStepExecution.java:694) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Executor #0 for <agent3> : executing PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=<folder>/<job>#476,label=<label>,context=CpsStepContext[59:node]:Owner[<folder>/<job>/476:<folder>/<job> #476],cookie=null,auth=null} "Executor #0 for <agent3> : executing PlaceholderExecutable:ExecutorStepExecution.PlaceholderTask{runId=<folder>/<job>#476,label=<label>,context=CpsStepContext[59:node]:Owner[<folder>/<job>/476:<folder>/<job> #476],cookie=null,auth=null}" Id=5917098 Group=main BLOCKED on java.io.PrintStream@7e577653 owned by "jenkins.util.Timer [#2]" Id=65 at java.io.PrintStream.println(PrintStream.java:805) -  blocked on java.io.PrintStream@7e577653 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask$PlaceholderExecutable.run(ExecutorStepExecution.java:694) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) {code} As before there are no processes being run by the Jenkins agent. The Timer threads look like: {code:java} jenkins.util.Timer [#10]jenkins.util.Timer [#10]"jenkins.util.Timer [#10]" Id=81 Group=main RUNNABLE (in native) at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:326) at java.io.PrintStream.write(PrintStream.java:480) -  locked java.io.PrintStream@107ec0b6 at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) -  locked java.io.OutputStreamWriter@1f820291 at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.newLine(PrintStream.java:546) -  locked java.io.PrintStream@107ec0b6 at java.io.PrintStream.println(PrintStream.java:807) -  locked java.io.PrintStream@107ec0b6 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$1.run(ExecutorStepExecution.java:117) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@64a11e8a jenkins.util.Timer [#2]"jenkins.util.Timer [#2]" Id=65 Group=main RUNNABLE (in native) at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:326) at java.io.PrintStream.write(PrintStream.java:480) -  locked java.io.PrintStream@7e577653 at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) -  locked java.io.OutputStreamWriter@6a5bc51 at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.newLine(PrintStream.java:546) -  locked java.io.PrintStream@7e577653 at java.io.PrintStream.println(PrintStream.java:807) -  locked java.io.PrintStream@7e577653 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$1.run(ExecutorStepExecution.java:117) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@32819625 jenkins.util.Timer [#9]"jenkins.util.Timer [#9]" Id=80 Group=main RUNNABLE (in native) at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:326) at java.io.PrintStream.write(PrintStream.java:480) -  locked java.io.PrintStream@208dc6f1 at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) -  locked java.io.OutputStreamWriter@9b4ca4c at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.newLine(PrintStream.java:546) -  locked java.io.PrintStream@208dc6f1 at java.io.PrintStream.println(PrintStream.java:807) -  locked java.io.PrintStream@208dc6f1 at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$1.run(ExecutorStepExecution.java:117) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@718a3c9d {code} This appears to be trying to write something to disk. Can someone tell what and where this would be trying to write? Add Comment   This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.