А guess at the cause. There is a jenkins-weekly-restart job on the instance, described as "Restart Jenkins when idle to pick up system library updates. Runs on Saturdays." Its last run was #51 on 22 August at 12:53, so the controller JVM has been up since then.
That fits what the trace complains about. The spawn helper is a small binary that ships with the JDK, and Jenkins own hint singles out in-place JDK updates. If the JDK on that machine was updated some time after 22 August, the running JVM would keep looking for the helper it started with, and every fork would fail the way it does now. I can't check that from outside, so this is a guess rather than a diagnosis. If it holds, the fix is the restart already scheduled for Saturday, just sooner. Either way the EC2 agents are unaffected - what is waiting on it is auto-critic and the pre-commit dryruns. ср, 26 авг. 2026 г. в 10:40, Aleksandr Efimov <[email protected]>: > Hi all, > > Something happened to the Jenkins controller between 03:39 and 06:44 UTC > today: every job that runs on the built-in node now dies on its first shell > step. The good news is that they fail fast - duration 0s) > > Building on the built-in node in workspace /var/lib/jenkins/workspace/... > java.io.IOException: Failed to exec spawn helper > Caused: java.io.IOException: Cannot run program "/bin/bash": Failed to > exec spawn helper > > Example: https://jenkins.impala.io/job/gerrit-auto-critic/24918/console > > So far this hits gerrit-auto-critic #24918, gerrit-auto-critic-test #6011 > and gerrit-verify-dryrun-external #3786, #3787,#3788 - all of them on the > built-in node. That node was fine a few hours earlier: gerrit-auto-critic > #24917 passed at 03:28 UTC in 87s, gerrit-auto-critic-test #6010 at 03:39 > UTC. Jobs on EC2 agents are unaffected - ubuntu-20.04-build-only, > rat-check-ub2004 and python-check-ub2004 all passed around 07:10 UTC. > > Two practical effects. Pre-commit dryruns can't be started at all, and I > don't see a way around that, since gerrit-verify-dryrun sits on the > built-in node as well. The other one is quieter and easier to miss: > auto-critic has stopped commenting, so a patch set with no auto-critic > remarks currently looks exactly like a patch set it had nothing to say > about. > > The trace ends with Jenkins own list of possible causes and fixes, > restarting the controller JVM being the first of them. If someone with > admin access has a moment, could you take a look? > > Thanks! >
