Hi all, Something happened to the Jenkins controller between 03:39 and 06:44 UTC today: every job that runs on the built-in node now dies on its first shell step. The good news is that they fail fast - duration 0s)
Building on the built-in node in workspace /var/lib/jenkins/workspace/... java.io.IOException: Failed to exec spawn helper Caused: java.io.IOException: Cannot run program "/bin/bash": Failed to exec spawn helper Example: https://jenkins.impala.io/job/gerrit-auto-critic/24918/console So far this hits gerrit-auto-critic #24918, gerrit-auto-critic-test #6011 and gerrit-verify-dryrun-external #3786, #3787,#3788 - all of them on the built-in node. That node was fine a few hours earlier: gerrit-auto-critic #24917 passed at 03:28 UTC in 87s, gerrit-auto-critic-test #6010 at 03:39 UTC. Jobs on EC2 agents are unaffected - ubuntu-20.04-build-only, rat-check-ub2004 and python-check-ub2004 all passed around 07:10 UTC. Two practical effects. Pre-commit dryruns can't be started at all, and I don't see a way around that, since gerrit-verify-dryrun sits on the built-in node as well. The other one is quieter and easier to miss: auto-critic has stopped commenting, so a patch set with no auto-critic remarks currently looks exactly like a patch set it had nothing to say about. The trace ends with Jenkins own list of possible causes and fixes, restarting the controller JVM being the first of them. If someone with admin access has a moment, could you take a look? Thanks!
