Re: Jenkins using a lot more resources after upgrade
https://issues.jenkins-ci.org/browse/JENKINS-34919 On Wednesday, May 18, 2016 at 10:45:44 AM UTC-4, Ugo Bellavance wrote: > > We only have 1 Jenkins server so I'll create a JIRA, > > Thanks a lot for your help :)! > > On Wednesday, May 18, 2016 at 10:40:21 AM UTC-4, Stephen Connolly wrote: >> >> If you have no slaves and only run builds on the master then we can >> definitively state that JENKINS-34573 is not the root cause. If you have >> slaves (even if idle or unused) then we cannot. >> >> If you have a definitive answer to the above question then you should >> probably create a JIRA >> >> On 18 May 2016 at 14:29, Ugo Bellavance <ug...@lubik.ca> wrote: >> >>> Hi Stephen, >>> >>> We don't use slaves. We only have one master. >>> >>> Should I create an issue in JIRA for my specific issue? >>> >>> (is there a reason why you got the conservation out of the group?) >>> >>> Thanks, >>> >>> On Wed, May 18, 2016 at 8:56 AM, Stephen Connolly < >>> stephen.al...@gmail.com> wrote: >>> >>>> Hard to say for certain as that could still be references maintained by >>>> the unexported objects that are pending unexport. If you can get your >>>> instance into that state and then have it sit idle for a couple of >>>> hours... >>>> if it is JENKINS-34573 then you should see the retained memory reduce as >>>> one reference is cleared every minute. Alternatively disconnect and >>>> reconnect all your slaves and see if that frees the memory up - >>>> JENKINS-34573 is only an issue for long running busy slaves >>>> >>>> On 17 May 2016 at 19:01, Ugo Bellavance <ug...@lubik.ca> wrote: >>>> >>>>> Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool >>>>> and here's what I found: >>>>> >>>>> One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" >>>>> loaded by "" occupies 6,801,517,856 (89.16%) bytes. >>>>> The instance is referenced by >>>>> org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded >>>>> by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The >>>>> memory >>>>> is accumulated in one instance of >>>>> "java.util.concurrent.RunnableScheduledFuture[]" loaded by ">>>> loader>". >>>>> >>>>> Does it match JENKINS-34213? >>>>> >>>>> On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: >>>>>> >>>>>> Jenkins 2.5 has the fix IIUC >>>>>> >>>>>> On 17 May 2016 at 14:28, Stephen Connolly <stephen.al...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> It is fixed, but in the latest version of remoting... which is not >>>>>>> something you can upgrade without either building a custom build of >>>>>>> Jenkins >>>>>>> or upgrading Jenkins. >>>>>>> >>>>>>> You could crack open your jenkins.war and replace the remoting.jar >>>>>>> with the fixed version and seal it back up again and see if that fixes >>>>>>> your >>>>>>> issue... probably the quickest way to confirm if my theory as to the >>>>>>> source >>>>>>> of your leak is correct, but you would need to know what you are doing >>>>>>> >>>>>>> On 17 May 2016 at 14:14, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>>> >>>>>>>> Thanks, but I can see that this issue is fixed but I don't know how >>>>>>>> to update my Jenkins install so that I have the fix. I'm using 1.656. >>>>>>>> >>>>>>>> Ugo >>>>>>>> >>>>>>>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I will repeat: >>>>>>>>> >>>>>>>>> > I am suspecting JENKINS-34213 may be your issue. >>>>>>>>> >>>>>>>>> On 17 May 2016 at 13:12, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>>>>> >&g
Re: Jenkins using a lot more resources after upgrade
We only have 1 Jenkins server so I'll create a JIRA, Thanks a lot for your help :)! On Wednesday, May 18, 2016 at 10:40:21 AM UTC-4, Stephen Connolly wrote: > > If you have no slaves and only run builds on the master then we can > definitively state that JENKINS-34573 is not the root cause. If you have > slaves (even if idle or unused) then we cannot. > > If you have a definitive answer to the above question then you should > probably create a JIRA > > On 18 May 2016 at 14:29, Ugo Bellavance <ug...@lubik.ca > > wrote: > >> Hi Stephen, >> >> We don't use slaves. We only have one master. >> >> Should I create an issue in JIRA for my specific issue? >> >> (is there a reason why you got the conservation out of the group?) >> >> Thanks, >> >> On Wed, May 18, 2016 at 8:56 AM, Stephen Connolly < >> stephen.al...@gmail.com > wrote: >> >>> Hard to say for certain as that could still be references maintained by >>> the unexported objects that are pending unexport. If you can get your >>> instance into that state and then have it sit idle for a couple of hours... >>> if it is JENKINS-34573 then you should see the retained memory reduce as >>> one reference is cleared every minute. Alternatively disconnect and >>> reconnect all your slaves and see if that frees the memory up - >>> JENKINS-34573 is only an issue for long running busy slaves >>> >>> On 17 May 2016 at 19:01, Ugo Bellavance <ug...@lubik.ca > >>> wrote: >>> >>>> Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool and >>>> here's what I found: >>>> >>>> One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" >>>> loaded by "" occupies 6,801,517,856 (89.16%) bytes. >>>> The instance is referenced by >>>> org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded >>>> by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The >>>> memory >>>> is accumulated in one instance of >>>> "java.util.concurrent.RunnableScheduledFuture[]" loaded by ">>> loader>". >>>> >>>> Does it match JENKINS-34213? >>>> >>>> On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: >>>>> >>>>> Jenkins 2.5 has the fix IIUC >>>>> >>>>> On 17 May 2016 at 14:28, Stephen Connolly <stephen.al...@gmail.com> >>>>> wrote: >>>>> >>>>>> It is fixed, but in the latest version of remoting... which is not >>>>>> something you can upgrade without either building a custom build of >>>>>> Jenkins >>>>>> or upgrading Jenkins. >>>>>> >>>>>> You could crack open your jenkins.war and replace the remoting.jar >>>>>> with the fixed version and seal it back up again and see if that fixes >>>>>> your >>>>>> issue... probably the quickest way to confirm if my theory as to the >>>>>> source >>>>>> of your leak is correct, but you would need to know what you are doing >>>>>> >>>>>> On 17 May 2016 at 14:14, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>> >>>>>>> Thanks, but I can see that this issue is fixed but I don't know how >>>>>>> to update my Jenkins install so that I have the fix. I'm using 1.656. >>>>>>> >>>>>>> Ugo >>>>>>> >>>>>>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly wrote: >>>>>>>> >>>>>>>> I will repeat: >>>>>>>> >>>>>>>> > I am suspecting JENKINS-34213 may be your issue. >>>>>>>> >>>>>>>> On 17 May 2016 at 13:12, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>
Re: Jenkins using a lot more resources after upgrade
Hi Stephen, We don't use slaves. We only have one master. Should I create an issue in JIRA for my specific issue? (is there a reason why you got the conservation out of the group?) Thanks, On Wed, May 18, 2016 at 8:56 AM, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > Hard to say for certain as that could still be references maintained by > the unexported objects that are pending unexport. If you can get your > instance into that state and then have it sit idle for a couple of hours... > if it is JENKINS-34573 then you should see the retained memory reduce as > one reference is cleared every minute. Alternatively disconnect and > reconnect all your slaves and see if that frees the memory up - > JENKINS-34573 is only an issue for long running busy slaves > > On 17 May 2016 at 19:01, Ugo Bellavance <u...@lubik.ca> wrote: > >> Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool and >> here's what I found: >> >> One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" loaded >> by "" occupies 6,801,517,856 (89.16%) bytes. The >> instance is referenced by >> org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded >> by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The memory >> is accumulated in one instance of >> "java.util.concurrent.RunnableScheduledFuture[]" loaded by "> loader>". >> >> Does it match JENKINS-34213? >> >> On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: >>> >>> Jenkins 2.5 has the fix IIUC >>> >>> On 17 May 2016 at 14:28, Stephen Connolly <stephen.al...@gmail.com> >>> wrote: >>> >>>> It is fixed, but in the latest version of remoting... which is not >>>> something you can upgrade without either building a custom build of Jenkins >>>> or upgrading Jenkins. >>>> >>>> You could crack open your jenkins.war and replace the remoting.jar with >>>> the fixed version and seal it back up again and see if that fixes your >>>> issue... probably the quickest way to confirm if my theory as to the source >>>> of your leak is correct, but you would need to know what you are doing >>>> >>>> On 17 May 2016 at 14:14, Ugo Bellavance <ug...@lubik.ca> wrote: >>>> >>>>> Thanks, but I can see that this issue is fixed but I don't know how to >>>>> update my Jenkins install so that I have the fix. I'm using 1.656. >>>>> >>>>> Ugo >>>>> >>>>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly wrote: >>>>>> >>>>>> I will repeat: >>>>>> >>>>>> > I am suspecting JENKINS-34213 may be your issue. >>>>>> >>>>>> On 17 May 2016 at 13:12, Ugo Bellavance <ug...@lubik.ca> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> If it helps, you might avoid the crash by installing the >>>>>>>>> monitoring plugin, and triggering garbage collection once the memory >>>>>>>>> is >>>>>>>>> approaching the maximum allocated heap size. This is a workaround >>>>>>>>> until >>>>>>>>> someone is able to diagnose the root cause. I have an open issue : >>>>>>>>> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought >>>>>>>>> I'd run the suggestion. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> Have anyone monitored the JVM with VisualVM? I have found that it >>>>>>>> looks like a specific memory pool may be filling up: Old Gen. I'm >>>>>>>> trying >>>>>>>> now with: >>>>>>>> >>>>>>>> -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xms10752m >>>>>>>> -Xmx10752m -XX:NewRatio=10 >>>>>>>> >>>>>>>> We'll see in
Re: Releases
On Tuesday, May 17, 2016 at 3:18:47 PM UTC-4, Daniel Beck wrote: > > > > On 17.05.2016, at 13:57, Ugo Bellavance <ug...@lubik.ca > > wrote: > > > > Ok, but how can I apply a fix then? I've been told that my problem may > be caused by JENKINS-34213, this currently has a status of "Fixed". How > can I update my code to have the fix? > > > > It's in Jenkins 2.4. The fix is not yet available on the LTS release line. > Ok, that is great, but when it will be available on the LTS line, will I be able to update my install from 1.656 to the version that will contain the fix (1.651.3 I guess)? Is there a way to be sure that JENKINS-34213 is the problem that I hit? I do have a heap dump and I've done some analysis on it, but it's a bit too complex for me at this point. Should I put the dump somewhere and ask for it to be analyzed in JENKINS-34213? Thanks, -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/24acc447-0cb3-431c-a90a-cc70b0b9d72d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Jenkins using a lot more resources after upgrade
Ok, I did analyse the heap dump with Eclipse's Memory Analyzer Tool and here's what I found: One instance of "java.util.concurrent.ScheduledThreadPoolExecutor" loaded by "" occupies 6,801,517,856 (89.16%) bytes. The instance is referenced by org.tmatesoft.svn.core.wc.DefaultSVNRepositoryPool @ 0x561b36858 , loaded by "hudson.ClassicPluginStrategy$AntClassLoader2 @ 0x56057c428". The memory is accumulated in one instance of "java.util.concurrent.RunnableScheduledFuture[]" loaded by "". Does it match JENKINS-34213? On Tuesday, May 17, 2016 at 9:29:16 AM UTC-4, Stephen Connolly wrote: > > Jenkins 2.5 has the fix IIUC > > On 17 May 2016 at 14:28, Stephen Connolly <stephen.al...@gmail.com > > wrote: > >> It is fixed, but in the latest version of remoting... which is not >> something you can upgrade without either building a custom build of Jenkins >> or upgrading Jenkins. >> >> You could crack open your jenkins.war and replace the remoting.jar with >> the fixed version and seal it back up again and see if that fixes your >> issue... probably the quickest way to confirm if my theory as to the source >> of your leak is correct, but you would need to know what you are doing >> >> On 17 May 2016 at 14:14, Ugo Bellavance <ug...@lubik.ca > >> wrote: >> >>> Thanks, but I can see that this issue is fixed but I don't know how to >>> update my Jenkins install so that I have the fix. I'm using 1.656. >>> >>> Ugo >>> >>> On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly wrote: >>>> >>>> I will repeat: >>>> >>>> > I am suspecting JENKINS-34213 may be your issue. >>>> >>>> On 17 May 2016 at 13:12, Ugo Bellavance <ug...@lubik.ca> wrote: >>>> >>>>> >>>>> >>>>> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary wrote: >>>>>>> >>>>>>> Hi, >>>>>>> If it helps, you might avoid the crash by installing the monitoring >>>>>>> plugin, and triggering garbage collection once the memory is >>>>>>> approaching >>>>>>> the maximum allocated heap size. This is a workaround until someone is >>>>>>> able >>>>>>> to diagnose the root cause. I have an open issue : >>>>>>> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought I'd >>>>>>> run the suggestion. >>>>>>> >>>>>>>> >>>>>>>> >>>>>> Have anyone monitored the JVM with VisualVM? I have found that it >>>>>> looks like a specific memory pool may be filling up: Old Gen. I'm >>>>>> trying >>>>>> now with: >>>>>> >>>>>> -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xms10752m >>>>>> -Xmx10752m -XX:NewRatio=10 >>>>>> >>>>>> We'll see in a few days. In the last few days, the problem have >>>>>> occured each morning at around 4:45. I'm not sure if the same jobs run >>>>>> on >>>>>> the week-ends as well. >>>>>> >>>>> >>>>> From what I can see, it will always fill the memory. Probably a leak, >>>>> but I don't know how to be 100% sure about it. When I try to load a heap >>>>> dump in VisualVM, but it shows nothing except "Not supported for this >>>>> JVM. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Jenkins Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to jenkinsci-use...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com?utm_medium=email_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to jenkinsci-use...@googlegroups.com . >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc--b46c-9d8fe4efc1a2%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc--b46c-9d8fe4efc1a2%40googlegroups.com?utm_medium=email_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/a7867041-c143-4d08-a7ee-22e4b78a4418%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Jenkins using a lot more resources after upgrade
Thanks, but I can see that this issue is fixed but I don't know how to update my Jenkins install so that I have the fix. I'm using 1.656. Ugo On Tuesday, May 17, 2016 at 8:56:34 AM UTC-4, Stephen Connolly wrote: > > I will repeat: > > > I am suspecting JENKINS-34213 may be your issue. > > On 17 May 2016 at 13:12, Ugo Bellavance <ug...@lubik.ca > > wrote: > >> >> >> On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: >>> >>> >>> >>> On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary wrote: >>>> >>>> Hi, >>>> If it helps, you might avoid the crash by installing the monitoring >>>> plugin, and triggering garbage collection once the memory is approaching >>>> the maximum allocated heap size. This is a workaround until someone is >>>> able >>>> to diagnose the root cause. I have an open issue : >>>> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought I'd run >>>> the suggestion. >>>> >>>>> >>>>> >>> Have anyone monitored the JVM with VisualVM? I have found that it looks >>> like a specific memory pool may be filling up: Old Gen. I'm trying now >>> with: >>> >>> -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xms10752m >>> -Xmx10752m -XX:NewRatio=10 >>> >>> We'll see in a few days. In the last few days, the problem have occured >>> each morning at around 4:45. I'm not sure if the same jobs run on the >>> week-ends as well. >>> >> >> From what I can see, it will always fill the memory. Probably a leak, >> but I don't know how to be 100% sure about it. When I try to load a heap >> dump in VisualVM, but it shows nothing except "Not supported for this JVM. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to jenkinsci-use...@googlegroups.com . >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com >> >> <https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com?utm_medium=email_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/145bf27e-4bcc--b46c-9d8fe4efc1a2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Jenkins using a lot more resources after upgrade
On Friday, May 13, 2016 at 8:50:56 AM UTC-4, Ugo Bellavance wrote: > > > > On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary wrote: >> >> Hi, >> If it helps, you might avoid the crash by installing the monitoring >> plugin, and triggering garbage collection once the memory is approaching >> the maximum allocated heap size. This is a workaround until someone is able >> to diagnose the root cause. I have an open issue : >> https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought I'd run >> the suggestion. >> >>> >>> > Have anyone monitored the JVM with VisualVM? I have found that it looks > like a specific memory pool may be filling up: Old Gen. I'm trying now > with: > > -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xms10752m > -Xmx10752m -XX:NewRatio=10 > > We'll see in a few days. In the last few days, the problem have occured > each morning at around 4:45. I'm not sure if the same jobs run on the > week-ends as well. > >From what I can see, it will always fill the memory. Probably a leak, but I don't know how to be 100% sure about it. When I try to load a heap dump in VisualVM, but it shows nothing except "Not supported for this JVM. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1b43cd1d-531b-4638-8e53-15d5f5670f91%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Releases
On Monday, May 16, 2016 at 4:18:09 PM UTC-4, Daniel Beck wrote: > > > > On 16.05.2016, at 17:15, Ugo Bellavance <ug...@lubik.ca > > wrote: > > > > However, there is nothing for 1.657 or 1.658 in either the current or > LTS release changelog. All I can find about these releases is on github: > [maven-release-plugin] copy for tag jenkins-1.658. Is that an artificial > bump in the version by some automated component? If only someone could > tell me if I could downgrade to 1.651 from 1.656 that would help. > > > > These were test releases since we performed some changes to the release > process right before 2.0 came out. They have no notable changes (one change > was implemented towards 1.657 and then reverted towards 1.658). > > See the diff between 1.656 and 1.658, the only real (but still very minor) > change is highlighted: > > https://github.com/jenkinsci/jenkins/compare/jenkins-1.656...jenkins-1.658#diff-e489e4f9a21586cacd9342ff6eeded61R31 > > > Just ignore them, like the changelog does ;-) > > Ok, but how can I apply a fix then? I've been told that my problem may be caused by JENKINS-34213, this currently has a status of "Fixed". How can I update my code to have the fix? Thanks, Ugo -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b604643a-7daa-4ebd-a79b-2b39ff6fc9fa%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Releases
Hi, I'm sorry, I didn't really follow the recent news about the changes in Jenkins' releases but I have done some reading and I still can't find an answer. Without knowing that there was an LTS version at 1.651, we upgraded our Jenkins system to what yum gave us: 1.656. Now we're experiencing problems (https://groups.google.com/forum/#!searchin/jenkinsci-users/resources/jenkinsci-users/qLwBFyQ84Z4/GWP4Ve_jOAAJ) and we don't know what are the options. From what I can see, one can: - Stick to 1.651 and get only fixes (1.651.2, 1.652.3, etc.) - Keep using the current release and get version 2.x - Be in the fog like us and wonder why there are still releases in 1.x in the current branch after 1.651 I saw this morning that 1.657 and 1.658 were out, so I downloaded the rpms and ran an rpm -qp --changelog on them to find out that this is not really used. Well, it redirects the user to the online changelog (http://jenkins-ci.org/changelog). However, there is nothing for 1.657 or 1.658 in either the current or LTS release changelog. All I can find about these releases is on github: [maven-release-plugin] copy for tag jenkins-1.658. Is that an artificial bump in the version by some automated component? If only someone could tell me if I could downgrade to 1.651 from 1.656 that would help. Thanks, -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6af9c371-d7c7-4d4f-ba72-69835d1a5813%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Jenkins using a lot more resources after upgrade
On Thursday, May 12, 2016 at 2:01:41 PM UTC-4, Raymond Accary wrote: > > Hi, > If it helps, you might avoid the crash by installing the monitoring > plugin, and triggering garbage collection once the memory is approaching > the maximum allocated heap size. This is a workaround until someone is able > to diagnose the root cause. I have an open issue : > https://issues.jenkins-ci.org/browse/JENKINS-34573 but thought I'd run > the suggestion. > >> >> Have anyone monitored the JVM with VisualVM? I have found that it looks like a specific memory pool may be filling up: Old Gen. I'm trying now with: -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xms10752m -Xmx10752m -XX:NewRatio=10 We'll see in a few days. In the last few days, the problem have occured each morning at around 4:45. I'm not sure if the same jobs run on the week-ends as well. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/19e79b21-0ade-4cd7-a681-ddbadbfee73b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Jenkins using a lot more resources after upgrade
On Thursday, May 12, 2016 at 4:31:37 AM UTC-4, Stephen Connolly wrote: > > I am suspecting JENKINS-34213 may be your issue. > I have new information. We installed the monitoring plugin and I started using VisualVM. I can see that, while I use -Xmx10752m, it only uses a little less than 8 GB. Is there a problem with my configuration? I also got a heap dump. JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Dcom.sun.management.jmxremote -Xmx10752m" (in /etc/sysconfig/jenkins on RHEL6). [image: zoom] -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/93a907bb-814a-42c0-b62f-c8e8f80c1598%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Jenkins using a lot more resources after upgrade
On Wednesday, May 11, 2016 at 12:32:59 PM UTC-4, John Mellor wrote: > > Yeah, I’m seeing that too, and I am also running Jenkins 1.656 version. > I’m only running about 400 jobs in Jenkins with 11 slaves, so it’s > definitely not the busiest build environment out there. I’ve currently > bumped the heap up to 5GB, and still do not have anywhere near enough to > run the backup plugin for example. I’m thinking of doubling it to see if > that helps, but your experience seems to say that this is also not going to > be enough. > I could try increasing again, but it doesn't make much sense. > > > This problem is also scaring me about whether unreasonable resource > utilization has been corrected in a near-future upgrade to Jenkins 2.2… > It's partly comforting, but also scary to see that we're not alone with this problem. Yes, it would be interesting to know if it has been fixed in version 2, and it would also be nice to know when this problem was introduced and if we can downgrade to a know working version. Should we open a bug in jenkins's JIRA? Have you seen https://issues.jenkins-ci.org/browse/JENKINS-34573? Have you generated a heap dump? I'm not too familiar with that (yet). Ugo -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d8605af0-9c53-434c-9d10-6b82fd762cf7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Jenkins using a lot more resources after upgrade
Hi, After some testing we upgraded Jenkins in production, from 1.617-1.1 to 1.656-1.1 on April 20. However, since the upgrade we're running into problems that we never saw before and it's now kind of out of control: - We had to raise the nofile parameter in limits.conf for the jenkins user because we were getting "Too many open files" errors. We raised it to 8192 soft, 10240 hard on April 28th. By looking into /proc/pid/ld/ I found out that most of these files where inet sockets. Got these errors again on May 8th and 9th. I didn't change anything yet for that issue - We had to raise the java heap memory. It was at -Xmx4096m, we raised it to -Xmx7168m, then to -Xmx10752m and adjusted the vRAM allocation for this VM accordingly, but it doesn't solve the problem, we're still getting "java.lang.OutOfMemoryError: Java heap space" errors. When that starts happening, we have to restart jenkins because the web interface is not responding anymore (which makes it difficult to troubleshoot because we can't really tell what is running at this moment. Note that we do run Selenium testing through IC, but nothing has changed drastically after the upgrade. RHEL updates were applied a few days before (Apr 6th) and that include a minor update to Firefox (38.0 => 38.7). We're using openjdk java JRE. Was updated to 1.7.0.99 (from 1.7.0.79) on April 6th, 1.7.0.101 on May 2nd. Has anyone experienced something similar? What are my best options? Our workaround now is to restart jenkins manually when needed. - Would it be possible to rollback to 1.617 without breaking anything? - Try the LTS version (that would mean a downgrade - would it break stuff?) - Jump to version 2 (2.2-1.1 available) - Inspect the JVM's memory - Any other idea is welcome Thanks in advance, Ugo -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4e6f40ca-cbf7-41b4-874e-18df1c04f9ce%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
