[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael Greco edited a comment on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Maybe try the LTS? ... uggg I try to be a "start-up" kind of guy ... it sounds like there's maybe some integration tests that need to be part of the project ... If you got access to spinning up another VM maybe launch the LTS version and try it out. I know I keep the jenkins data in a docker volume so moving around between these versions to try stuff out on different docker hosts for exactly these situations is helpful. I'm running 2.140 but maybe the plugin works better with the LTS (ok I'm reaching out side the box). Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael Greco edited a comment on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Maybe try the LTS? ... uggg I try to be a "start-up" kind of guy ... it sounds like there's maybe some integration tests that need to be part of the project ... If you got access to spinning up another VM maybe launch the LTS version and try it out. I know I keep the jenkins data in a docker volume so moving around between these versions to try stuff out on different docker hosts for exactly these situations is helpful. I'm running 2.140 but maybe the plugin works better with the LTS ? (ok I'm reaching out side the box ) cause if the plugin has a bug then . ..) Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael Greco edited a comment on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Maybe try the LTS? ... uggg I try to be a "start-up" kind of guy ... it sounds like there's maybe some integration tests that need to be part of the project ... If you got access to spinning up another VM maybe launch the LTS version and try it out. I know I keep the jenkins data in a docker volume so moving around between these versions to try stuff out on different docker hosts for exactly these situations is helpful. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael Greco commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Maybe try the LTS? ... uggg I try to be a "start-up" kind of guy ... it sounds like there's maybe some integration tests that need to be part of the project ... Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" If left unfixed for much longer, our organization is going to be forced to use another technology for CICD since this is causing widespread pain and confidence lost in this technology among our hundreds of developers who are using Jenkins at our company. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael Greco edited a comment on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" I've been noticing this for MONTHS. And in case people don't realize the master branch of the docker-plugin wasn't building today 9/17/18 :[https://ci.jenkins.io/job/Plugins/job/docker-plugin/] Anyways this weekend I loaded docker-plugin build 1.1.5 and today on every build I was getting "The channel is closing down or has closed down" as my jobs would still appear to be running even though obviously the container was gone. I would up downgrading to an older build I have : |1.1.5-SNAPSHOT (private-554bbf8a-win2012-6d34b0$)| in which the problem seems to happen less. I went so far as to rebuild some of my "build containers" as they are created "FROM jenkinsci/slave" and I noticed that has had an update sometime in August. Again It made no difference using the "released 1.1.5" version of docker-plugin (every thing wound up in the state of "The channel is closing down or has closed down") and that's when I noticed the master branch isn't building either ... so I just went back to my earlier build. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael Greco commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" I've been noticing this for MONTHS. And in case people don't realize the master branch of the docker-plugin wasn't building today 9/17/18 : https://ci.jenkins.io/job/Plugins/job/docker-plugin/ Anyways this weekend I loaded docker-plugin build 1.1.5 and today on every build I was getting "The channel is closing down or has closed down" as my jobs would still appear to be running even though obviously the container was gone. I would up downgrading to an older build I have : 1.1.5-SNAPSHOT (private-554bbf8a-win2012-6d34b0$) in which the problem seems to happen less. I went so far as to rebuild some of my "build containers" as they are created "FROM jenkinsci/slave" and I noticed that has had an update sometime in August. Again It made no difference using the "released 1.1.5" version of docker-plugin (every thing wound up in the state of "The channel is closing down or has closed down") and that's when I noticed the master branch isn't building either ... so I just went back to my earlier build. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Michael McCallum commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" I concur this is a pretty serious issue, I've tried a number or workarounds like timeouts to restart the job but once it hangs its stuck. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title
Jon B commented on JENKINS-49707
Re: Pipeline hangs: "The channel is closing down or has closed down"
Repro code:
def jobs = [:]
jobs["Do Work"] = getWork()
parallel jobs
println "Parallel run completed."
def getWork() {
return {
node('general') {
sh """|#!/bin/bash
|set -ex
|echo "going to sleep..."
|sleep 300
|echo "yay I made it to the end."
|""".stripMargin()
}
}
}
To repro, run this pipeline and once the control flow hits the sleep, terminate the executor's host and it will hang with something like this:
[Do Work] Cannot contact ip-172-31-237-68.us-west-2.compute.internal: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on ip-172-31-237-68.us-west-2.compute.internal failed. The channel is closing down or has closed down
It fails with this error every time I try it.
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title
Jon B edited a comment on JENKINS-49707
Re: Pipeline hangs: "The channel is closing down or has closed down"
Repro code:{code:java}def jobs = [:]jobs["Do Work"] = getWork()parallel jobsprintln "Parallel run completed."def getWork() { return {node('general') { sh """|#!/bin/bash|set -ex|echo "going to sleep..."|sleep 300|echo "yay I made it to the end."|""".stripMargin()} }} {code}To repro, run this pipeline and once the control flow hits the sleep, terminate the executor's host and it will hang with something like this:{code:java}[Do Work] Cannot contact ip-172-31-237-68.us-west-2.compute.internal: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on ip-172-31-237-68.us-west-2.compute.internal failed. The channel is closing down or has closed down {code}It fails hangs with this error every time I try it.
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B edited a comment on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" I don't mean to be dramatic but this is literally the biggest problem in all of Jenkins as far as I can tell. If we lose an ec2 host while an executor is doing parallel work, we *badly need* for the parallel item to restart on a another healthy node executor . When it just plain hangs, we can't do that and the user experience of hanging is not acceptable.I would recommend elevating the urgency here to the highest level to get this triaged. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" I don't mean to be dramatic but this is literally the biggest problem in all of Jenkins as far as I can tell. If we lose an ec2 host while an executor is doing parallel work, we badly need for the parallel item to restart on a healthy node. When it just plain hangs, we can't do that and the user experience of hanging is not acceptable. I would recommend elevating the urgency here to the highest level to get this triaged. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" The repro case here is pretty simple: 1) Create a parallel job (even a job that just does a sleep) 2) Terminate the executor's host while its running It hangs with this error every time. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Federico Naum commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Has someone with experiencing this issue had a look at this new plugin https://plugins.jenkins.io/remoting-kafka Oleg Nenashev I can see you are very involved with it. Looks promising, is lacking some documentation, but I'll play with to see if I can get it working, and report back to see If that solve my connection issues. Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Alex Slaughter commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" We have also been greatly effected by this issue. A resolution would be very nice Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Alex Slaughter updated an issue Jenkins / JENKINS-49707 Pipeline hangs: "The channel is closing down or has closed down" Change By: Alex Slaughter Priority: Minor Major Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Shital Savekar commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" I would like to increase the Priority of this issue to "Major" since this issue is affecting a lot of users. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Oleg Nenashev updated an issue Jenkins / JENKINS-49707 Pipeline hangs: "The channel is closing down or has closed down" Change By: Oleg Nenashev Component/s: _unsorted Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Oleg Nenashev commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Please provide the following info: Support bundle for the timeframe of the outage: https://wiki.jenkins.io/display/JENKINS/Support+Core+Plugin Agent threaddump dump for the timeframe of the outage Agent filesystem log for the time of the outage You can find some pointers here: https://speakerdeck.com/onenashev/day-of-jenkins-2017-dealing-with-agent-connectivity-issues?slide=51 Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" i just changed the JIRA "component" field for this to "remoting". Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B updated an issue Jenkins / JENKINS-49707 Pipeline hangs: "The channel is closing down or has closed down" Change By: Jon B Component/s: remoting Component/s: workflow-durable-task-step-plugin Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B commented on JENKINS-49707 Re: Pipeline hangs: "The channel is closing down or has closed down" Oleg Nenashev Should this be redesignated a remoting bug? I'm not sure how to unblock my pipelines that are hanging from this issue. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-49707) Pipeline hangs: "The channel is closing down or has closed down"
Title: Message Title Jon B updated an issue Jenkins / JENKINS-49707 Pipeline hangs: "The channel is closing down or has closed down" Change By: Jon B Summary: Pipeline stuck hangs : "The channel is closing down or has closed down" Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
