[JIRA] (JENKINS-5413) SCM polling getting hung
Title: Message Title Meng Xin Zhu commented on JENKINS-5413 Re: SCM polling getting hung Still happening on Jenkins LTS 2.19.4. My job is periodically polling from git repo(every 5 minutes). However the scm polling might hang infinitely without timeout. The subsequent manually job building also is blocked by scm polling. It's definitely a critical issue to impact the usability of jenkins. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-5413) SCM polling getting hung
Derek Seibert commented on JENKINS-5413 SCM polling getting hung Our team encounters this issue almost daily using the Dimensions SCM plugin. We run a single instance Jenkins server which polls a Stream ever 30 minutes. I wanted to comment just to explain how we first noticed the issue was occurring in case anybody searching this issue starts in the same place. We select the Dimensions Polling Log for our job and see the following at maybe 9AM or 10AM. The polling has hung at this point and we need to restart our application server. Started on Mar 25, 2013 8:30:35 AM We expect to see something like. Started on Mar 25, 2013 8:30:35 AM Done. Took 19 sec No changes This is why this issue is so troubling. There is no notification trigger when "Started on..." has just been sitting there hung for a while, and no further polling can be done by that job without a restart of the application server. Our lead is hesitant to use a script to kill SCM threads because we don't know/haven't fully investigated the source of the issue. I figured this would be the best place to discuss it. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] (JENKINS-5413) SCM polling getting hung
Derek Seibert edited a comment on JENKINS-5413 SCM polling getting hung Our team encounters this issue almost daily using the Dimensions SCM plugin. We run a single instance Jenkins server which polls a Stream ever 30 minutes. I wanted to comment just to explain how we first noticed the issue was occurring in case anybody searching this issue starts in the same place. We select the Dimensions Polling Log for our job and see the following at maybe 9AM or 10AM. The polling has hung at this point and we need to restart our application server. Started on Mar 25, 2013 8:30:35 AM We expect to see something like. Started on Mar 25, 2013 8:30:35 AM Done. Took 19 sec No changes This is why this issue is so troubling. There is no notification trigger when "Started on..." has just been sitting there hung for a while, and no further polling can be done by that job without a restart of the application server. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] (JENKINS-5413) SCM polling getting hung
lacostej commented on JENKINS-5413 SCM polling getting hung Derek, Not sure if Dimensions plugin is using a native call or not hunder the hood. Could you make a threaddump and/or a list of processes ? J This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] (JENKINS-5413) SCM polling getting hung
Linards L commented on JENKINS-5413 SCM polling getting hung Have not noticed that for long time now. But .. in the v1.48x version series I had similar problems, but the dissapeared. Remember we also did some scheduled rebooting temperings ... now using v1.494. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] (JENKINS-5413) SCM polling getting hung
Linards L edited a comment on JENKINS-5413 SCM polling getting hung Have not noticed that for long time now. But .. in the v1.48x version series I had similar problems, but then they dissapeared. Remember we also did some scheduled rebooting temperings ... now using v1.494. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] (JENKINS-5413) SCM polling getting hung
Raja Aluri
commented on JENKINS-5413
SCM polling getting hung
For people who are on windows and want to setup a scheduled task. Here is a oneliner in powershell.
PS C:\Users\jenkins
PS C:\Users\jenkins
PS C:\Users\jenkins
PS C:\Users\jenkins tasklist /FI "IMAGENAME eq ssh.exe" /FI "Status eq Unknown" /NH | %{ $_.Split(' *',[StringSplitOptions]"RemoveEmptyEntries")[1]} |ForEach-Object {taskkill /F /PID $_}
PS C:\Users\jenkins
PS C:\Users\jenkins
PS C:\Users\jenkins
PS C:\Users\jenkins
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] (JENKINS-5413) SCM polling getting hung
lacostej updated JENKINS-5413 SCM polling getting hung I encountered a very similar issue, yet I have a slightly different setup: 1 master 1 slave yet the polling was stuck on the master only SCM polling hanging (warning displayed in Jenkins configure screen). Oldest hanging thread is more than 2 days old. it seems it all started with a Unix process that in some way never returned: ps -aef| grep jenkins 300 12707 1 0 26Nov12 ?? 1336:33.09 /usr/bin/java -Xmx1024M -XX:MaxPermSize=128M -jar /Applications/Jenkins/jenkins.war 300 98690 12707 0 Sat03PM ?? 0:00.00 git fetch -t https://github.com/jenkinsci/testflight-plugin.git +refs/heads/*:refs/remotes/origin/* 300 98692 98690 0 Sat03PM ?? 4:39.72 git-remote-https https://github.com/jenkinsci/testflight-plugin.git https://github.com/jenkinsci/testflight-plugin.git 0 3371 3360 0 8:20PM ttys0000:00.02 su jenkins 300 4017 3372 0 8:52PM ttys0000:00.00 grep jenkins 0 10920 10896 0 19Nov12 ttys0010:00.03 login -pfl jenkins /bin/bash -c exec -la bash /bin/bash Running Jenkins 1.479 I killed the processes and associated threads, and it started being better. Polling doesn't enfore timeouts ? Change By: lacostej (14/Jan/13 8:15 PM) Attachment: DUMP1.txt This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-5413) SCM polling getting hung
Mandeep Rai
commented on JENKINS-5413
SCM polling getting hung
I modified the script a little bit:
Jenkins.instance.getTrigger("SCMTrigger").getRunners().each()
{
item -
println(item.getTarget().name)
println(item.getDuration())
println(item.getStartTime())
long millis = Calendar.instance.time.time - item.getStartTime()
if(millis (1000 * 60 * 3)) // 1000 millis in a second * 60 seconds in a minute * 3 minutes
{
Thread.getAllStackTraces().keySet().each()
{
tItem -
if (tItem.getName().contains("SCM polling") tItem.getName().contains(item.getTarget().name))
{
println "Interrupting thread " + tItem.getName();
tItem.interrupt()
}
}
}
}
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-5413) SCM polling getting hung
Mandeep Rai
edited a comment on JENKINS-5413
SCM polling getting hung
I modified the script a little bit:
Jenkins.instance.getTrigger("SCMTrigger").getRunners().each()
{
item -
println(item.getTarget().name)
println(item.getDuration())
println(item.getStartTime())
long millis = Calendar.instance.time.time - item.getStartTime()
if(millis (1000 * 60 * 3)) // 1000 millis in a second * 60 seconds in a minute * 3 minutes
{
Thread.getAllStackTraces().keySet().each()
{
tItem -
if (tItem.getName().contains("SCM polling") tItem.getName().contains(item.getTarget().name))
{
println "Interrupting thread " + tItem.getName();
tItem.interrupt()
}
}
}
}
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-5413) SCM polling getting hung
Alex Lorenz updated JENKINS-5413 SCM polling getting hung Change By: Alex Lorenz (27/Jun/12 11:33 AM) Summary: SCMpolling onslaves gettinghung Priority: Major Critical This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-5413) SCM polling getting hung
Alex Lorenz commented on JENKINS-5413 SCM polling getting hung This does not only happen on slaves, but also on single machine Jenkins systems. With us here at TomTom, it happens regularly and makes us lose valuable builds. Escalate - Critical This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-5413) SCM polling getting hung
Christian Wolfgang commented on JENKINS-5413 SCM polling getting hung Joe: Yes, the ClearCase UCM plugin. We experienced the slaves to hang when having uncaught runtime exceptions and thus the master's polling thread will never be joined. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
