[JIRA] [other] (JENKINS-23572) Too many open files exception
Damon G updated JENKINS-23572 Too many open files exception Change By: Damon G (26/Jun/14 11:46 AM) Description: Wecanrunforabout2daysandthenthesystembasicallybecomesunresponsiveandneedstoberestarted.Everythingwasgoingfineformanymonthsatversion1.5.46butoncewewentto1.5.66or1.5.68webegintoseetoomanyopenfilesexceptionsuntiljenkinsneededarestart.Attachedarethestacktraces.IdidntchooseaspecificcomponentbecauseIwasntsurewhichcomponenttologitagainst.Ialsodontseeanyexistingopenissuesthatmatchthis.Hopeopeningthiswasvalid.Iveattachedalistofallfilesopened(usinglsof)byjenkinsaswellasthejenkinslogcontainingthestacktraces.IonlyincludedacoupleofthestacktracesbecauseIhaveafeelingtheoutputoflsofmaybemoreilluminating.Happytoprovideanythingmoreyouneed.ThisisoneofmyfirstdefectssoIhopeImgivingenoughdetailwithoutitalreadybeingaknownissue. Update- Wedidanexperiementwherewerepeatedlyranthejenkinscli.Wedidthisbecauseourjenkinsjobsutilizeitheavilyandweweresuspiciousitmightberelated.Foreveryjenkinsclicommandexecutedgeneratedanotheropenfilereferencethatdoesntappeartogetclosed. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [other] (JENKINS-23572) Too many open files exception
Damon G updated JENKINS-23572 Too many open files exception Change By: Damon G (26/Jun/14 11:49 AM) Description: Wecanrunforabout2daysandthenthesystembasicallybecomesunresponsiveandneedstoberestarted.Everythingwasgoingfineformanymonthsatversion1.5.46butoncewewentto1.5.66or1.5.68webegintoseetoomanyopenfilesexceptionsuntiljenkinsneededarestart.Attachedarethestacktraces.IdidntchooseaspecificcomponentbecauseIwasntsurewhichcomponenttologitagainst.Ialsodontseeanyexistingopenissuesthatmatchthis.Hopeopeningthiswasvalid.Iveattachedalistofallfilesopened(usinglsof)byjenkinsaswellasthejenkinslogcontainingthestacktraces.IonlyincludedacoupleofthestacktracesbecauseIhaveafeelingtheoutputoflsofmaybemoreilluminating.Happytoprovideanythingmoreyouneed.ThisisoneofmyfirstdefectssoIhopeImgivingenoughdetailwithoutitalreadybeingaknownissue.Update-Wedidanexperiementwherewerepeatedlyranthejenkinscli.Wedidthisbecauseourjenkinsjobsutilizeitheavilyandweweresuspiciousitmightberelated.Foreveryjenkinsclicommandexecutedgeneratedanotheropenfilereferencethatdoesntappeartogetclosed. Itcorrespondstoentriesinlsoflikethis: java1230jenkins3772usock0,70t053062cantidentifyprotocol Itdoesntlooklikethisfilesareevergettingclosed. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [other] (JENKINS-23572) Too many open files exception
Damon G updated JENKINS-23572 Too many open files exception Change By: Damon G (26/Jun/14 11:51 AM) Description: Wecanrunforabout2daysandthenthesystembasicallybecomesunresponsiveandneedstoberestarted.Everythingwasgoingfineformanymonthsatversion1.5.46butoncewewentto1.5.66or1.5.68webegintoseetoomanyopenfilesexceptionsuntiljenkinsneededarestart.Attachedarethestacktraces.IdidntchooseaspecificcomponentbecauseIwasntsurewhichcomponenttologitagainst.Ialsodontseeanyexistingopenissuesthatmatchthis.Hopeopeningthiswasvalid.Iveattachedalistofallfilesopened(usinglsof)byjenkinsaswellasthejenkinslogcontainingthestacktraces.IonlyincludedacoupleofthestacktracesbecauseIhaveafeelingtheoutputoflsofmaybemoreilluminating.Happytoprovideanythingmoreyouneed.ThisisoneofmyfirstdefectssoIhopeImgivingenoughdetailwithoutitalreadybeingaknownissue.Update-Wedidanexperiementwherewerepeatedlyranthejenkinscli.Wedidthisbecauseourjenkinsjobsutilizeitheavilyandweweresuspiciousitmightberelated.Foreveryjenkinsclicommandexecutedgeneratedanotheropenfilereferencethatdoesntappeartogetclosed.Itcorrespondstoentriesinlsoflikethis:java1230jenkins3772usock0,70t053062cantidentifyprotocolItdoesntlooklikethisfilesareevergettingclosed. Ifyoulookinthatlsoffileyoullseeroughly11,000linesjustliketheoneabove. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [other] (JENKINS-23572) Too many open files exception
Damon G created JENKINS-23572 Too many open files exception Issue Type: Bug Affects Versions: current Assignee: Unassigned Attachments: jenkins_stack_trace.log, lsof.txt Components: other Created: 25/Jun/14 2:12 PM Description: We can run for about 2 days and then the system basically becomes unresponsive and needs to be restarted. Everything was going fine for many months at version 1.5.46 but once we went to 1.5.66 or 1.5.68 we begin to see too many open files exceptions until jenkins needed a restart. Attached are the stack traces. I didn't choose a specific component because I wasn't sure which component to log it against. I also don't see any existing open issues that match this. Hope opening this was valid. I've attached a list of all files opened (using lsof) by jenkins as well as the jenkins log containing the stack traces. I only included a couple of the stack traces because I have a feeling the output of lsof may be more illuminating. Happy to provide anything more you need. This is one of my first defects so I hope I'm giving enough detail without it already being a known issue. Environment: Ubuntu 12.04 50 + ssh ubuntu 12.04 slaves. Project: Jenkins Priority: Major Reporter: Damon G This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
