[JIRA] (JENKINS-61851) Repeated build failures after agent disconnection
Title: Message Title
Mikaël Barbero commented on JENKINS-61851
Re: Repeated build failures after agent disconnection
Connection is back to stable without any change... In the meantime, I've found some tweaks that may be worth trying for anyone facing similar issues. It's basically tweaking the proxy connection, in particular: proxy_buffering on; {{ proxy_buffer_size 32k;}} Also, increasing the proxy_send_timeout and proxy_read_timeout may help. Again, this is just things to try in case it happens again. I did not have to do any of these changes for the connection to be back to stable. Closing as this is most probably not a Jenkins issue.
Add Comment
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.205714.1586438248000.19268.1588152240253%40Atlassian.JIRA.
[JIRA] (JENKINS-61851) Repeated build failures after agent disconnection
Title: Message Title Mikaël Barbero closed an issue as Not A Defect closing Jenkins / JENKINS-61851 Repeated build failures after agent disconnection Change By: Mikaël Barbero Status: Open Closed Resolution: Not A Defect Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit
[JIRA] (JENKINS-61851) Repeated build failures after agent disconnection
Title: Message Title
Mikaël Barbero updated an issue
Jenkins / JENKINS-61851
Repeated build failures after agent disconnection
Change By:
Mikaël Barbero
We've been facing a couple of build failure because of agent disconnection for no apparent reasons. Failures are following the same pattern:The build logs say (time of error):{ { noformat} 9:45:50 FATAL: command execution failedjava.nio.channels.ClosedChannelException at jenkins.agents.WebSocketAgents$Session.closed(WebSocketAgents.java:141) at jenkins.websocket.WebSocketSession.onWebSocketSomething(WebSocketSession.java:91)...Caused: java.io.IOException: Backing channel 'our-windows10-agent' is disconnected. at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216) {noformat } } On the agent, the logs say:{ { noformat} Apr 09, 2020 9:45:50 AM hudson.Launcher$RemoteLaunchCallable$1 joinINFO: Failed to synchronize IO streams on the channel hudson.remoting.Channel@11010e5e:our-windows10-agenthudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@11010e5e:our-windows10-agent": Remote call on our-windows10-agent failed. The channel is closing down or has closed downat hudson.remoting.Channel.call(Channel.java:991)at hudson.remoting.Channel.syncIO(Channel.java:1730)...Caused by: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@11010e5e:our-windows10-agent": channel is already closedat hudson.remoting.Engine$1AgentEndpoint.onClose(Engine.java:590)at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusEndpointWrapper.onClose(TyrusEndpointWrapper.java:1251)Apr 09, 2020 9:45:50 AM hudson.remoting.UserRequest performWARNING: LinkageError while performing UserRequest:UserRPCRequest:hudson.Launcher$RemoteProcess.join[](4)java.lang.LinkageError: Failed to load hudson.util.ProcessTreeat hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:391)...Caused by: java.lang.NoClassDefFoundError: hudson/util/ProcessTreeRemoting$IProcessTreeat java.lang.ClassLoader.defineClass1(Native Method)...Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTreeRemoting$IProcessTreeat java.net.URLClassLoader.findClass(URLClassLoader.java:382) {noformat } } On the master, the agent logs say (file slave.log.1, mtime = Apr 9 09:45):{ { noformat} Inbound agent connected from x.x.x.xRemoting version: 4.3This is a Windows agentonOnline: class org.jenkinsci.modules.slave_installer.impl.ComputerListenerImpl reported an exception: java.net.MalformedURLException: no protocol: jnlpJars/slave.jarAgent successfully connected and onlineERROR: Connection terminatedjava.nio.channels.ClosedChannelException at jenkins.agents.WebSocketAgents$Session.closed(WebSocketAgents.java:141) at jenkins.websocket.WebSocketSession.onWebSocketSomething(WebSocketSession.java:91) at jenkins.websocket.WebSockets$$Lambda$282/940934C0.invoke(Unknown Source) at com.sun.proxy.$Proxy74.onWebSocketClose(Unknown Source) at
[JIRA] (JENKINS-61851) Repeated build failures after agent disconnection
Title: Message Title
Mikaël Barbero created an issue
Jenkins / JENKINS-61851
Repeated build failures after agent disconnection
Issue Type:
Bug
Assignee:
Jeff Thompson
Components:
remoting
Created:
2020-04-09 13:17
Environment:
master = Jenkins 2.229, in docker image (linux) proxy passed by nginx, openjdk 1.8.0_242 agent = remoting 4.3 (with -webSocket connection), windows10, openjdk 1.9.0_202
Priority:
Major
Reporter:
Mikaël Barbero
We've been facing a couple of build failure because of agent disconnection for no apparent reasons. Failures are following the same pattern: The build logs say (time of error): {{ 9:45:50 FATAL: command execution failed java.nio.channels.ClosedChannelException at jenkins.agents.WebSocketAgents$Session.closed(WebSocketAgents.java:141) at jenkins.websocket.WebSocketSession.onWebSocketSomething(WebSocketSession.java:91) ... Caused: java.io.IOException: Backing channel 'our-windows10-agent' is disconnected. at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)}} On the agent, the logs say: {{ Apr 09, 2020 9:45:50 AM hudson.Launcher$RemoteLaunchCallable$1 join INFO: Failed to synchronize IO streams on the channel hudson.remoting.Channel@11010e5e:our-windows10-agent hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@11010e5e:our-windows10-agent": Remote call on our-windows10-agent failed. The channel is closing down or has closed down at hudson.remoting.Channel.call(Channel.java:991) at hudson.remoting.Channel.syncIO(Channel.java:1730) ... Caused by: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@11010e5e:our-windows10-agent": channel is already closed at hudson.remoting.Engine$1AgentEndpoint.onClose(Engine.java:590) at
[JIRA] (JENKINS-56307) Create a NodeProvisioner.Strategy for Kubernetes plugin to reduce provisioning time
Title: Message Title Mikaël Barbero commented on JENKINS-56307 Re: Create a NodeProvisioner.Strategy for Kubernetes plugin to reduce provisioning time runze xia awesome! Let me know if/how I can help! Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.197887.1551263076000.948.1568885220308%40Atlassian.JIRA.
[JIRA] (JENKINS-56307) Create a NodeProvisioner.Strategy for Kubernetes plugin to reduce provisioning time
Title: Message Title Mikaël Barbero commented on JENKINS-56307 Re: Create a NodeProvisioner.Strategy for Kubernetes plugin to reduce provisioning time It looks like it would be a very good fit. Thanks for sharing. Are you starting the backport of the EC2 no-delay strategy to the Kubernetes plugin? Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.197887.1551263076000.930.1568885040666%40Atlassian.JIRA.
[JIRA] (JENKINS-56307) Create a NodeProvisioner.Strategy for Kubernetes plugin to reduce provisioning time
Title: Message Title Mikaël Barbero created an issue Jenkins / JENKINS-56307 Create a NodeProvisioner.Strategy for Kubernetes plugin to reduce provisioning time Issue Type: New Feature Assignee: Carlos Sanchez Components: kubernetes-plugin Created: 2019-02-27 10:24 Labels: performance kuberenetes-plugin Priority: Major Reporter: Mikaël Barbero Currently, one have to fiddle with default provisionner flags (https://github.com/jenkinsci/kubernetes-plugin#over-provisioning-flags) to improve responsiveness of agent provisioning (at the risk of overprovisioning). Even with recommended values, some useless delays are introduced. I guess that in the context of a Kubernetes cloud, we could provide a simpler, yet more efficient, NodeProvisioner.Strategy implementation that would speed up the provisioning time. WDYT? Add Comment
[JIRA] (JENKINS-50879) The secretVolume is not mounted to "jnlp" container in the multicontainer pod
Title: Message Title Mikaël Barbero commented on JENKINS-50879 Re: The secretVolume is not mounted to "jnlp" container in the multicontainer pod Javier was on the right track, I sent a PR with a test + a fix https://github.com/jenkinsci/kubernetes-plugin/pull/371 Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50879) The secretVolume is not mounted to "jnlp" container in the multicontainer pod
Title: Message Title
Mikaël Barbero edited a comment on JENKINS-50879
Re: The secretVolume is not mounted to "jnlp" container in the multicontainer pod
I'm not familiar enough with the code to add a test, but this example reproduces the issue: { { code:groovy} podTemplate(label: 'mypod', }}{{ containers: [ }} {{ containerTemplate(name: 'busybox', image: 'busybox', ttyEnabled: true, command: '/bin/cat'), }} {{ ], }} {{ volumes: [ }} {{ emptyDirVolume(mountPath: '/etc/mount1', memory: true), }} {{ ]) { }} {{ }} {{ node ('mypod') { }} {{ stage('Run') { }} {{ container('busybox') { }} {{ sh """ }} {{ ls /etc/mount1 }} {{ """ }} {{ } }} {{ container('jnlp') { }} {{ sh """ }} {{ ls /etc/mount1 }} {{ """ } } {{ } }} {{ } }} {{ } }} { { code } }} The build fails on the second ls in the jnlp container as the folder does not exist:{{+ ls /etc/mount1}}{{ls: /etc/mount1: No such file or directory}}
Add Comment
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-50879) The secretVolume is not mounted to "jnlp" container in the multicontainer pod
Title: Message Title
Mikaël Barbero commented on JENKINS-50879
Re: The secretVolume is not mounted to "jnlp" container in the multicontainer pod
I'm not familiar enough with the code to add a test, but this example reproduces the issue: {{podTemplate(label: 'mypod', }} containers: [ containerTemplate(name: 'busybox', image: 'busybox', ttyEnabled: true, command: '/bin/cat'), ], volumes: [ emptyDirVolume(mountPath: '/etc/mount1', memory: true), ]) { {{ }} node ('mypod') { stage('Run') { container('busybox') { sh """ ls /etc/mount1 """ {{ }}} container('jnlp') { sh """ ls /etc/mount1 """ {{ }}} {{ }}} {{ }}} } The build fails on the second ls in the jnlp container as the folder does not exist: + ls /etc/mount1 ls: /etc/mount1: No such file or directory
Add Comment
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
