[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube resolved JENKINS-21933 as Cannot Reproduce PermGen leaks The client's script was causing issues. Change By: Heather Fraser-Dube (12/Jun/14 12:17 AM) Status: Open Resolved Resolution: CannotReproduce This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Christian, I don't understand why you would have 8 Jenkins instances. What other plugins do you have installed? Do you have other things (monitoring?) going on during that timeframe? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje updated JENKINS-21933 PermGen leaks Here's the # of classes ratcheting up over time. I don't have any jobs that just run early in the morning, so I'm not sure why it ratchets up at that point. Change By: Christian Höltje (17/Apr/14 9:46 PM) Attachment: ClassCount.png This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Eugene Konkov commented on JENKINS-21933 PermGen leaks It is sad. Workaround with -XX:+CMSClassUnloadingEnabled did't help us. We got Out of memory exception after less than two days of uptime. Is there a chance to get it fixed? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje commented on JENKINS-21933 PermGen leaks No. Unfortunately. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje commented on JENKINS-21933 PermGen leaks Heather: We're using winstone. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Eugene Konkov commented on JENKINS-21933 PermGen leaks We also experiencing this issue. Here is our environment: Sun Java java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode) Debian 6 Squeeze Jenkins 1.557. Will downgrade to 1.553 be workaround for this issue? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Jim you also need to enable garbage collection on classes loaded. See the comment from Abhishek above. If you are using more than 1 toolkit you may also need to increase the permgen area size. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Jim Goodspeed commented on JENKINS-21933 PermGen leaks We also seem to be experiencing a similar issue. Here is our environment: OpenJDK RHEL 6 Jenkins 1.553 (Winstone) -XX:MaxPermSize=256m -Xms1024m -Xmx1024m This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Christian, I examined the 2nd dump you sent. It does look like class loader issues. It looks like there are 8 toolkit class loaders being referenced. I dug further and see 8 copies of the class that manages the class loader. We expect 1, this is a class that is never instantiated, so I have the feeling that something with multiple different class loaders resulted in us being referenced. I am also seeing 8 of the jenkins.model.Jenkins class. How are you starting Jenkins, are you running it within winstone? tomcat? WAS? something else? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Zachary Young commented on JENKINS-21933 PermGen leaks Does RTC stand for "Rational Team Concert"? Using just the default/core SCM plugin, but just got this error: javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/tmp/jetty-0.0.0.0-8090-jenkins.war--any-/webapp/WEB-INF/lib/jenkins-core-1.550.jar!/lib/layout/layout.jelly:75:72: st:include java.lang.OutOfMemoryError: PermGen space at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:117) at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:717) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:795) Is this like the error you got? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje commented on JENKINS-21933 PermGen leaks So I was wrong about using the IBM JRE It eventually filled up the heap and started thrashing. I didn't get OOM errors, but performance went into the toilet. I had 2g of heap and it filled it up. I'll email the PHD to Heather directly. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks I have raised 305293: PermGen leaks with HJPlugin to continue tracking this. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Thanks Abhishek for the workaround. We do special class loading in order to access the toolkit(s). But the class loader should be cached and re-used. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Is it a single build you are building over and over when it runs out of permgen space? Does the build run any ant tasks? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Thought about this more. We shouldn't be using Antrunner afik. Is this happening with a single job with a single toolkit and a single build user defined? If the answer is No... How many toolkits do you have defined? How many any other jobs on your Jenkins instance? Do they use different toolkits? Different build users ids? Is polling enabled? If so at what frequency and for how many jobs? What version of the toolkit are you using? Which version of Jenkins? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Heather Fraser-Dube commented on JENKINS-21933 PermGen leaks Are you using a build definition or a build workspace? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Abhishek Mukherjee commented on JENKINS-21933 PermGen leaks My company was also seeing this issue while trying to run on a Mac Mini using the home-brewed installation to begin, but upgraded to 1.552. I seem to have temporarily fixed this issue by adding some of the flags from JENKINS-13617, specifically: -Xmx768m -XX:MaxPermSize=256m -XX:+CMSClassUnloadingEnabled Unfortunately I don't know if it was increasing the perm size or the class unloading that did it. Just in case you guys need which plugins we use, since that seems like the most customizable part (this is my first Jenkins bug comment), we have these installed: Name Version Enabled Pinned analysis-core 1.55 true false android-emulator 2.10 true false android-lint 2.1 true false ant 1.2 true false checkstyle 3.39 true false chucknorris 0.5 true false credentials 1.10 true true cvs 2.11 true true dashboard-view 2.9.2 true false email-ext 2.37.2 true false external-monitor-job 1.2 true true findbugs 4.51 true false git 2.0.3 true false git-client 1.6.3 true false gitlab-hook 1.0.0 true false gitlab-merge-request-jenkins 1.1.2 true false gradle 1.23 true false gravatar 1.1 true false htmlpublisher 1.3 true false javadoc 1.1 true false ldap 1.8 true true mailer 1.8 true true matrix-auth 1.1 true false maven-plugin 2.1 true true multiple-scms 0.3 true false pam-auth 1.1 true true port-allocator 1.8 true false ruby-runtime 0.12 true false scm-api 0.2 true false ssh-credentials 1.6.1 true true ssh-slaves 1.6 true true subversion 2.2 true true token-macro 1.10 true false translation 1.11 true true windows-slaves 1.0 true false xunit 1.82 true false This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje updated JENKINS-21933 PermGen leaks Change By: Christian Höltje (26/Feb/14 3:44 AM) Description: ItappearsthatifyourunwithaJREthatusesPermGen(e.g.OpenJDK1.6)thattheJenkinsmasterwilleventuallyrunoutofPermGenspace.ItseemslikeitiscreatingagabillionAntclasses(approx.number)...Itlookslike O(n) acontantnumber perbuild.Youcanrecreatethisbystandingupasimplejenkins.warandaddingajobthatusesRTCSCMandthenbuildingit.EachbuildeatsupmorePermGenspace.Isuspectthisisrelatedto:https://bugs.eclipse.org/bugs/show_bug.cgi?id=130545IamworkingarounditbyusingIBMs1.6JRE,whichdoesnthavePermGen.However,itnoweatsalotofHeap...ImhopingthateverythingwillbeGCdcorrectly.Eitherway,thisshouldstillbefixed. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje commented on JENKINS-21933 PermGen leaks It was the `-XX:+CMSClassUnloadingEnabled` that did it. I didn't mention it because it is scary flag on the Oracle/OpenJDK. It is one of those things that may work or may blow things up. The IBM JRE doesn't have PermGen. Instead all the Class objects are stored on the Heap and as such, can be garbage collected like any other objects. The IBM JRE is meant to work this way. The ClassUnloading flag, on the other hand, is a sort of hack and isn't generally trusted for production use. Ciao! This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje commented on JENKINS-21933 PermGen leaks Heather: It isn't a single job. It's about 90 jobs, of which about 10-20 are actively changed through out the day. I'm guessing, but I'd say easily 100+ builds for RTC jobs a day. I only have one toolkit defined in the System Config. I only use one RTC user and system for all Jobs. Almost all the Jobs use RTC. Polling is enabled and it is polling once a minute. That's pretty aggressive and that's why I filed a bug (or CC'd a bug) about push notification from RTC. We're using RTC 4.0.1 across the board, including the toolkit. Jenkins is version 1.532.1 (the previous LTS) We are using build workspaces, not build definitions. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [teamconcert] (JENKINS-21933) PermGen leaks
Christian Höltje created JENKINS-21933 PermGen leaks Issue Type: Bug Assignee: Unassigned Components: teamconcert Created: 24/Feb/14 3:27 PM Description: It appears that if you run with a JRE that uses PermGen (e.g. OpenJDK 1.6) that the Jenkins master will eventually run out of PermGen space. It seems like it is creating a gabillion Ant classes (approx. number)... It looks like O per build. You can recreate this by standing up a simple jenkins.war and adding a job that uses RTC SCM and then building it. Each build eats up more PermGen space. I suspect this is related to: https://bugs.eclipse.org/bugs/show_bug.cgi?id=130545 I am working around it by using IBM's 1.6 JRE, which doesn't have PermGen. However, it now eats a lot of Heap... I'm hoping that everything will be GC'd correctly. Either way, this should still be fixed. Project: Jenkins Labels: permgen Priority: Major Reporter: Christian Höltje This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
