[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick updated JENKINS-62181 Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Jesse Glick Status: In Progress Review Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23226.1588863960268%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Jesse Glick Labels: deadlock lts-candidate regression Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23223.1588863660166%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Jesse Glick Component/s: core Component/s: remoting Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23214.1588863600894%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick started work on JENKINS-62181 Change By: Jesse Glick Status: Open In Progress Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23221.1588863601389%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick assigned an issue to Jesse Glick Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Jesse Glick Assignee: Jeff Thompson Jesse Glick Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23218.1588863601005%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Jesse Glick Labels: deadlock regression Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23209.1588863600713%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Jesse Glick commented on JENKINS-62181 Re: Remoting deadlock when launching remote agent The “new” method was just an override of an existing synchronized method, though the deadlock does smell like a regression in the new code before the super call. You have a custom logger on hudson.remoting.Channel (or hudson.remoting) at FINE or below I think? As a workaround, try removing that logger. Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23205.1588863240399%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Daniel Beck commented on JENKINS-62181 Re: Remoting deadlock when launching remote agent FYI Jesse Glick since your change is mentioned. Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.206113.1588790553000.23111.1588840740184%40Atlassian.JIRA.
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Bart Heupers updated an issue Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Bart Heupers After launching an agent it keeps hanging with the following text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent This happens since a couple of weeks jstack reports a deadlock in attachment : Found one Java-level deadlock: = "pool-1-thread-4 for channel id=17236": waiting to lock monitor 0x7fbd8c015bc8 (object 0x00076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1), which is held by "Channel reader thread: channel" "Channel reader thread: channel": waiting to lock monitor 0x7fbd9401aed8 (object 0x00076ded1cc8, a hudson.remoting.RemoteClassLoader), which is held by "pool-1-thread-4 for channel id=17236" On april 3 2020 in commit [https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d] a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java Add Comment
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Bart Heupers updated an issue Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Change By: Bart Heupers After launching an agent it keeps hangin hanging with the follwing following text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent jstack reports a deadlock in attachment : Found one Java-level deadlock:="pool-1-thread-4 for channel id=17236": waiting to lock monitor 0x7fbd8c015bc8 (object 0x00076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1), which is held by "Channel reader thread: channel""Channel reader thread: channel": waiting to lock monitor 0x7fbd9401aed8 (object 0x00076ded1cc8, a hudson.remoting.RemoteClassLoader), which is held by "pool-1-thread-4 for channel id=17236" On april 3 2020 in commit [https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d] a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java Add Comment
[JIRA] (JENKINS-62181) Remoting deadlock when launching remote agent
Title: Message Title Bart Heupers created an issue Jenkins / JENKINS-62181 Remoting deadlock when launching remote agent Issue Type: Bug Assignee: Jeff Thompson Attachments: jenkins-remoting-deadlock.log Components: remoting Created: 2020-05-06 18:42 Environment: Jenkins 2.231 Remoting version 4.3 Manifest-Version: 1.0 Application-Name: Jenkins Remoting Agent Permissions: all-permissions Codebase: * Trusted-Library: true Build-Jdk-Spec: 1.8 Version: 4.3 Created-By: Maven Jar Plugin 3.2.0 Main-Class: hudson.remoting.Launcher Labels: deadlock Priority: Critical Reporter: Bart Heupers After launching an agent it keeps hangin with the follwing text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent jstack reports a deadlock in attachment : Found one Java-level deadlock: = "pool-1-thread-4 for channel id=17236": waiting to lock monitor
