[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg edited a comment on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 @Oleg Nenashev, Did you try it with 1200 active jobs? anyway this is what solved my problem. I guess you have issue with 100+ slave machines connected to Jenkins with high load Jenkins. @Kohsuke Kawaguchi, because I have 64GB RAM, I can't do it, I can't save 64GB RAM on my HDD and anyway my problem is already solved so it's save to close it. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg closed JENKINS-20620 as Incomplete Memory Leak on Jenkins LTS 1.509.4/1.532.1 Change By: Ronen Peleg (12/Mar/15 6:41 PM) Status: Resolved Closed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 @Oleg Nenashev, this is what solved my problem. I guess you have issue with 100+ slave machines connected to Jenkins. @Kohsuke Kawaguchi, because I have 64GB RAM, I can't do it, I can't save 64GB RAM on my HDD and anyway my problem is already solved so it's save to close it. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Kohsuke Kawaguchi resolved JENKINS-20620 as Incomplete Memory Leak on Jenkins LTS 1.509.4/1.532.1 We need more information to be able to solve problems like this. Please see https://wiki.jenkins-ci.org/display/JENKINS/I%27m+getting+OutOfMemoryError for how to get the details we need to be able to work on problems like this. I'm not doubting that you are seeing the problem, and for that I am sorry. Please get us the details we need so that we can fix the problem. If you cannot post a heap dump, please get at least the histogram summary. Change By: Kohsuke Kawaguchi (09/Jun/14 9:30 PM) Status: Open Resolved Resolution: Incomplete This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Update: The solution was to delete some slave machines from the Jenkins nodes. It turns out that Jenkins can't handle more than 100 slave machines. Currently we have (after cleanup) 70 slave machines and no memory leak! BTW: The memory leak issue occurs only on Jenkins Master running on Linux O/S with more than 100 slave machines, Actually on Windows O/S this issue doesn't exist! This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Thanks Oleg. I can't dump it because it's to big (40GB) and for some reason the "jmap" unable to connect to the Jenkins process. Anyway I downgraded the version to 1.537... Oleg, If you have a system with 600 jobs with high activities + 40 slave machines online and you don't have this issue it's very strange because I test these versions on several different machines and each time the problem reproduced (after 12-24 hours). This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Oleg Nenashev commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Probably, we have different job contents/plugins (e.g. we don't use Maven plugin; xUnit components are not popular due to integration with external systems). Could you provide a list of your plugins? Since we know about stable/unstable versions, we can try to find an interoperability issue if it exists. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Oleg, I sent you email with our Jenkins Plugins list. Thank You This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Update: I checked the release version: 1.539 that on JoJ, this version is memory leaks free but this version suffers from other bugs such as problem when using Build multi-configuration project (Matrix job) + Builds of a concurrently executable job. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Update: I checked also the 1.54X versions (1.541, 1.542, 1.543, 1.544) and the memory leak issue is also on all release 1.54X versions! This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Oleg Nenashev commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Could you collect a heap dump / any other memory statistics of your system? We use 1.509.4 with a quite similar configuration, but we have not experienced such memory leaks (the uptime is close to 3 months). There are many other issues, but seems that the memory is OK... This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
vjuranek commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 It seems that JoJ 1 runs without any problems, so this memory leak maybe appears only on some specific configurations (or somehow fixed in 1.539?). Did you only upgrade Jenkins core or did you also upgrade any plugins (so we can exclude memory leak in the plugins)? By system with 50 jobs you mean 50 jobs present or 50 jobs run in parallel most of the time? Thanks Btw: there's no Jenkins support team, it's community project based on volunteers, if you want a support team, you should check Cloudbees or some other provider which offers support for Jenkins. 1 https://ci.jenkins-ci.org/ This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg commented on JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Thank You for your help First with all the respect to JoJ they doesn't have the system load that we are facing here and has you can see it's snapshots version. Anyway, I didn't try the release version 1.539... BTW: In order to solve memory leaks you must use load software such as Load Runner. I tried the new LTS versions after update plugins and without update plugins, still the same issue same bug and as I wrote this bug doesn't happens on previous version at all. Of course it community, open source and all of that... but don't forget there is also Enterprise edition that including "Jenkins support team" and I just wonder why they not solving this super critical issue but I guess this all another story... This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg updated JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Change By: Ronen Peleg (16/Dec/13 9:14 AM) Environment: LinuxServerwith24CPUsand64GBRAMJenkinsversionLTS1.509.4/1.532.1onJettyMemoryallocatedforJenkins/Jettyprocess:42GB Load Environment :Jenkinsworkingwith600jobswithhighactivities +40slavemachines(LinuxandWindows) This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg updated JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Change By: Ronen Peleg (04/Dec/13 6:36 AM) Labels: 1.509.4 1.509.4LTS1.532.11.532.1LTS JenkinsLTS Description: AfterJenkinsupgradedtoLTS1.509.4fromLTS1.509.3Inoticethatovertime(24hours)Jenkinsbecomesveryslowly.ItturnsoutthatJenkins(underJettyservice)slowlyeatstheservermemory.Itstakesabout24hourstotakeallthememoryallocatedtoJenkins(42GB).Seethesnapshotswithexamples...TEST#1onLTS1.509.4:1.MachinewithJettyupafterrestart2.JenkinsUsed-AfteroneHour:22GB3.JenkinsUsed-After12Hours:27GB4.JenkinsUsed-After20Hours:35GB-Memoryleaksbetween10:00-10:20asyoucanseeafterGCitsstillthinkJavain-useandfailtocleanuptheallmemoryasitshouldbe.5.JenkinsUsed-After23Hours:39GB-VeryslowresponseandHeapisalmost100%TEST#2onLTS1.509.4:ItriedtodomanualGC,Doesnthelp!(seeattachfile:Monitor_Memory_Over_Time_Manual_GC)TEST#3onLTS1.509.3:UnfortunatelyIdowngradetoLTS1.509.3because ofthe memoryleak , formeitsablockerissue! On BTW:In versionLTS1.509.3Jenkinsworksstablewithoutanymemoryleaks...(seeattachfile:Good_GC_A1.509.3) butwithonebigunsolvedproblem,Icantrenamejobs(deadlock)!TEST#4withLTS1 . 532.1: Sameissue!Jenkinsstuckwith100%memoryusageafteronly12hours! Thanks ThankYou ,Ronen. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg updated JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Change By: Ronen Peleg (04/Dec/13 5:27 AM) Summary: MemoryLeakonJenkinsLTS1.509.4 /1.532.1 Environment: LinuxServerwith24CPUsand64GBRAMJenkinsversionLTS1.509.4 /1.532.1 onJettyMemoryallocatedforJenkins/Jettyprocess:42GBLoad:Jenkinsworkingwith600jobswithhighactivities This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-20620) Memory Leak on Jenkins LTS 1.509.4/1.532.1
Ronen Peleg updated JENKINS-20620 Memory Leak on Jenkins LTS 1.509.4/1.532.1 Change By: Ronen Peleg (04/Dec/13 8:27 PM) Attachment: Good_GC_B1.509.3.jpg Description: AfterJenkinsupgradedtoLTS1.509.4fromLTS1.509.3Inoticethatovertime(24hours)Jenkinsbecomesveryslowly.ItturnsoutthatJenkins(underJettyservice)slowlyeatstheservermemory.Itstakesabout24hourstotakeallthememoryallocatedtoJenkins(42GB).Seethesnapshotswithexamples...TEST#1onLTS1.509.4:1.MachinewithJettyupafterrestart2.JenkinsUsed-AfteroneHour:22GB3.JenkinsUsed-After12Hours:27GB4.JenkinsUsed-After20Hours:35GB-Memoryleaksbetween10:00-10:20asyoucanseeafterGCitsstillthinkJavain-useandfailtocleanuptheallmemoryasitshouldbe.5.JenkinsUsed-After23Hours:39GB-VeryslowresponseandHeapisalmost100%TEST#2onLTS1.509.4:ItriedtodomanualGC,Doesnthelp!(seeattachfile:Monitor_Memory_Over_Time_Manual_GC)TEST#3onLTS1.509.3:UnfortunatelyIdowngradetoLTS1.509.3becauseofthememoryleak,formeitsablockerissue! BTW:In Pleasenotethaton versionLTS1.509.3Jenkinsworksstable evenonhighenvironment withoutanymemory leaks leak ...( see See attach file files :Good_GC_A1.509.3 andGood_GC_B1.509.3 )but withonebig unfortunatelythereisaBIG unsolvedproblem /buginthisversion ,Icantrenamejobs( deadlock) Deadlock ! whichsolvedonthenextversionLTS1.509.4/1.532.1thatIcantusebecauseofthememoryleak). TEST#4withLTS1.532.1:Sameissue!Jenkinsstuckwith100%memoryusageafteronly12hours!ThankYou,Ronen. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
