Re: Upgrade 1.598 -> 1.609.1 breaks jobs on 1 slave, revert doesn't fix

[Ross Oliver]

Root cause turned out to be an outdated commons-io jar I had installed for 
another project in the Jenkins user's ~/Library/Java/Extensions directory. 
 When I updated to the latest version, both Jenkins and the other project 
were happy.  The problem manifested at the time of the upgrade attempt 
because that was the first time the Jenkins slave on that host had been 
restarted since the installation of the outdated jar.

On Wednesday, June 10, 2015 at 3:16:14 PM UTC-7, Ross Oliver wrote:
>
> Greetings,
>
> I am running a Jenkins master and several slaves all on Mac OS 10.10. 
>  Yesterday I attempted an upgrade from 1.598 to 1.609.1, and immediately 
> all jobs on one slave started failing, unable to check out from git.  The 
> other slaves were unaffected an continued to operate normally.  After some 
> basic troubleshooting, I decided I needed to get these jobs working again, 
> so I reverted to 1.598.  I also had to manually restore the config.xml from 
> the previous day, as the storage of node data in 1.609.1 was incompatible 
> with 1.598.  The revert seems successful, but the jobs on the 1 slave were 
> still failing with the same errors.  The jobs would run fine if moved to 
> another slave.  Still other than the slave.jar file, I could not find 
> anything on the slave that the upgrade/revert might have changed.  For 
> troubleshooting, I created a very simple freestyle job that executed a 
> single shell command "echo this is here"  That job failed with the 
> following log message (slave host name removed per company confidentiality 
> policy):
>
> Started by user ross_oliver 
> [EnvInject] - Loading node environment variables.
> Building remotely on  (java7 master2) in workspace 
> 
> FATAL: command execution failedjava.io.IOException 
> : Remote 
> call on  failed
>   at hudson.remoting.Channel.call(Channel.java:760) 
> 
>   at hudson.Launcher$RemoteLauncher.launch(Launcher.java:916) 
> 
>   at hudson.Launcher$ProcStarter.start(Launcher.java:381) 
> 
>   at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) 
> 
>   at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) 
> 
>   at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 
> 
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
>  
> 
>   at hudson.model.Build$BuildExecution.build(Build.java:199) 
> 
>   at hudson.model.Build$BuildExecution.doRun(Build.java:160) 
> 
>   at 
> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) 
> 
>   at hudson.model.Run.execute(Run.java:1718) 
> 
>   at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 
> 
>   at hudson.model.ResourceController.execute(ResourceController.java:89) 
> 
>   at hudson.model.Executor.run(Executor.java:240) 
> 
> Caused by: java.lang.NoSuchFieldError 
> : 
> NULL_OUTPUT_STREAM
>   at hudson.Launcher$ProcStarter.(Launcher.java:152) 
> 
>   at hudson.Launcher.launch(Launcher.java:408) 
> 
>   at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1129) 
> 

Upgrade 1.598 -> 1.609.1 breaks jobs on 1 slave, revert doesn't fix

[Ross Oliver]

Greetings,

I am running a Jenkins master and several slaves all on Mac OS 10.10. 
 Yesterday I attempted an upgrade from 1.598 to 1.609.1, and immediately 
all jobs on one slave started failing, unable to check out from git.  The 
other slaves were unaffected an continued to operate normally.  After some 
basic troubleshooting, I decided I needed to get these jobs working again, 
so I reverted to 1.598.  I also had to manually restore the config.xml from 
the previous day, as the storage of node data in 1.609.1 was incompatible 
with 1.598.  The revert seems successful, but the jobs on the 1 slave were 
still failing with the same errors.  The jobs would run fine if moved to 
another slave.  Still other than the slave.jar file, I could not find 
anything on the slave that the upgrade/revert might have changed.  For 
troubleshooting, I created a very simple freestyle job that executed a 
single shell command "echo this is here"  That job failed with the 
following log message (slave host name removed per company confidentiality 
policy):

Started by user ross_oliver 
[EnvInject] - Loading node environment variables.
Building remotely on  (java7 master2) in workspace 
FATAL: command execution failedjava.io.IOException 
: Remote 
call on  failed
at hudson.remoting.Channel.call(Channel.java:760) 

at hudson.Launcher$RemoteLauncher.launch(Launcher.java:916) 

at hudson.Launcher$ProcStarter.start(Launcher.java:381) 

at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) 

at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) 

at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 

at 
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
 

at hudson.model.Build$BuildExecution.build(Build.java:199) 

at hudson.model.Build$BuildExecution.doRun(Build.java:160) 

at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) 

at hudson.model.Run.execute(Run.java:1718) 

at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 

at hudson.model.ResourceController.execute(ResourceController.java:89) 

at hudson.model.Executor.run(Executor.java:240) 

Caused by: java.lang.NoSuchFieldError 
: 
NULL_OUTPUT_STREAM
at hudson.Launcher$ProcStarter.(Launcher.java:152) 

at hudson.Launcher.launch(Launcher.java:408) 

at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1129) 

at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101) 

at hudson.remoting.UserRequest.perform(UserRequest.java:121) 

at hudson.remoting.UserRequest.perform(UserRequest.java:49) 

at hudson.remoting.Request$2.run(Request.java:324)