https://issues.jenkins-ci.org/browse/JENKINS-34919
On Wednesday, May 18, 2016 at 10:45:44 AM UTC-4, Ugo Bellavance wrote: > > We only have 1 Jenkins server so I'll create a JIRA, > > Thanks a lot for your help :)! > > On Wednesday, May 18, 2016 at 10:40:21 AM UTC-4, Stephen Connolly wrote: >> >> If you have no slaves and only run builds on the master then we can >> definitively state that JENKINS-34573 is not the root cause. If you have >> slaves (even if idle or unused) then we cannot. >> >> If you have a definitive answer to the above question then you should >> probably create a JIRA >> >> On 18 May 2016 at 14:29, Ugo Bellavance <ug...@lubik.ca> wrote: >> >>> Hi Stephen, >>> >>> We don't use slaves. We only have one master. >>> >>> Should I create an issue in JIRA for my specific issue? >>> >>> (is there a reason why you got the conservation out of the group?) >>> >>> Thanks, >>> >>> On Wed, May 18, 2016 at 8:56 AM, Stephen Connolly < >>> stephen.al...@gmail.com> wrote: >>> >>>> Hard to say for certain as that could still be references maintained by >>>> the unexported objects that are pending unexport. If you can get your >>>> instance into that state and then have it sit idle for a couple of >>>> hours... >>>> if it is JENKINS-34573 then you should see the retained memory reduce as >>>> one reference is cleared every minute. Alternatively disconnect and >>>> reconnect all your slaves and see if that frees the memory up - >>>> JENKINS-34573 is only an issue for long running busy slaves >>>> >>>> On 17 May 2016 at 19:01, Ugo Bellavance <ug...@lubik.ca> wrote: >>>> >>>>> Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool >>>>> and here's what I found: >>>>> >>>>> One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" >>>>> loaded by "<system class loader>" occupies 6,801,517,856 (89.16%) bytes. >>>>> The instance is referenced by >>>>> org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded >>>>> by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The >>>>> memory >>>>> is accumulated in one instance of >>>>> "java.util.concurrent.RunnableScheduledFuture[]" loaded by "<system class >>>>> loader>". >>>>> >>>>> Does it match JENKINS-34213? >>>>> >>>>> On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: >>>>>> >>>>>> Jenkins 2.5 has the fix IIUC >>>>>> >>>>>> On 17 May 2016 at 14:28, Stephen Connolly <stephen.al...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> It is fixed, but in the latest version of remoting... which is not >>>>>>> something you can upgrade without either building a custom build of >>>>>>> Jenkins >>>>>>> or upgrading Jenkins. >>>>>>> >>>>>>> You could crack open your jenkins.war and replace the remoting.jar >>>>>>> with the fixed version and seal it back up again and see if that fixes >>>>>>> your >>>>>>> issue... probably the quickest way to confirm if my theory as to the >>>>>>> source >>>>>>> of your leak is correct, but you would need to know what you are doing >>>>>>> >>>>>>> On 17 May 2016 at 14:14, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>>> >>>>>>>> Thanks, but I can see that this issue is fixed but I don't know how >>>>>>>> to update my Jenkins install so that I have the fix. I'm using 1.656. >>>>>>>> >>>>>>>> Ugo >>>>>>>> >>>>>>>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I will repeat: >>>>>>>>> >>>>>>>>> > I am suspecting JENKINS-34213 may be your issue. >>>>>>>>> >>>>>>>>> On 17 May 2016 at 13:12, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> If it helps, you might avoid the crash by installing the >>>>>>>>>>>> monitoring plugin, and triggering garbage collection once the >>>>>>>>>>>> memory is >>>>>>>>>>>> approaching the maximum allocated heap size. This is a workaround >>>>>>>>>>>> until >>>>>>>>>>>> someone is able to diagnose the root cause. I have an open issue : >>>>>>>>>>>> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought >>>>>>>>>>>> I'd run the suggestion. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> Have anyone monitored the JVM with VisualVM? I have found that >>>>>>>>>>> it looks like a specific memory pool may be filling up: Old Gen. >>>>>>>>>>> I'm >>>>>>>>>>> trying now with: >>>>>>>>>>> >>>>>>>>>>> -Djava.awt.headless=true -Dcom.sun.management.jmxremote >>>>>>>>>>> -Xms10752m -Xmx10752m -XX:NewRatio=10 >>>>>>>>>>> >>>>>>>>>>> We'll see in a few days. In the last few days, the problem have >>>>>>>>>>> occured each morning at around 4:45. I'm not sure if the same jobs >>>>>>>>>>> run on >>>>>>>>>>> the week-ends as well. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> From what I can see, it will always fill the memory. Probably a >>>>>>>>>> leak, but I don't know how to be 100% sure about it. When I try to >>>>>>>>>> load a >>>>>>>>>> heap dump in VisualVM, but it shows nothing except "Not supported >>>>>>>>>> for this >>>>>>>>>> JVM. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Jenkins Users" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to jenkinsci-use...@googlegroups.com. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Jenkins Users" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to jenkinsci-use...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc-4444-b46c-9d8fe4efc1a2%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc-4444-b46c-9d8fe4efc1a2%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Jenkins Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to jenkinsci-use...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/jenkinsci-users/a7867041-c143-4d08-a7ee-22e4b78a4418%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/jenkinsci-users/a7867041-c143-4d08-a7ee-22e4b78a4418%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Jenkins Users" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/jenkinsci-users/qLwBFyQ84Z4/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> jenkinsci-use...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMyz55b4ppFhQ3STgH5fK7cwmanns8VR_yFmmDLZ5BVKeA%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMyz55b4ppFhQ3STgH5fK7cwmanns8VR_yFmmDLZ5BVKeA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Ugo Bellavance (ug...@lubik.ca) >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to jenkinsci-use...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/CACSSk%2BUNBm15yaMbhm85tcd%3D50aN2E8eKbf0BMvv0TEdoWS3Og%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/jenkinsci-users/CACSSk%2BUNBm15yaMbhm85tcd%3D50aN2E8eKbf0BMvv0TEdoWS3Og%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/3c5e05e1-196a-4def-8447-10d1135a5c04%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
