Hey everyone, Sorry for resurrecting that thread, but are there any solutions for this problem? It's basically killing our instance often.
best, Andrew On Wed, Aug 1, 2012 at 11:47 AM, Andrew Melo <andrew.m...@gmail.com> wrote: > Oh, wow, I didn't notice, but jenkins has autopopulated a user for > everyone that ever committed on the project. There's like 30 people, > ~4000 commits, so I could see why that would take a while :) > > On Wed, Aug 1, 2012 at 11:42 AM, Andrew Melo <andrew.m...@gmail.com> wrote: >> On Wed, Aug 1, 2012 at 11:37 AM, Slide <slide.o....@gmail.com> wrote: >>> Can you gist your global config.xml and something from one of your >>> jobs as well? Please remember to sanitize it. >> >> We actually keep it stored in SCM. https://github.com/dmwm/jenkins/ >> >> And the following is the gist for the job we run each commit (didn't >> make it in for some reason...) >> >> https://gist.github.com/3228599 >> >>> >>> On Wed, Aug 1, 2012 at 9:34 AM, Andrew Melo <andrew.m...@gmail.com> wrote: >>>> On Wed, Aug 1, 2012 at 11:26 AM, Slide <slide.o....@gmail.com> wrote: >>>>> No, because its only looking for the email address because it wants to >>>>> send an email to that user. >>>> >>>> I don't know who's getting emailed. I don't remember setting it up for >>>> anything, and we actually wrote some scripts that turn jenkins >>>> success/failures into Github issues, so having jenkins also send >>>> emails would be redundant. >>>> >>>> I don't supposed there's a global flag to disable email? (I don't see >>>> one at "manage jenkins") >>>> >>>> -Andrew >>>>> >>>>> On Wed, Aug 1, 2012 at 9:24 AM, Andrew Melo <andrew.m...@gmail.com> wrote: >>>>>> On Wed, Aug 1, 2012 at 11:22 AM, Slide <slide.o....@gmail.com> wrote: >>>>>>> This is a huge issue with the email-ext plugin as well when it does >>>>>>> email address resolution. Quite a number of people have complained >>>>>>> about how long it takes. I have yet to come up with a good solution. >>>>>>> The perforce plugin has a similar issue. >>>>>> >>>>>> If I just stick a dummy address in every user's profile, will that work? >>>>>> >>>>>>> >>>>>>> slide >>>>>>> >>>>>>> On Wed, Aug 1, 2012 at 9:17 AM, Vojtech Juranek <vjura...@redhat.com> >>>>>>> wrote: >>>>>>>> Looks like you it does search for user's email: >>>>>>>> hudson.scm.SubversionMailAddressResolverImpl.findMailAddressFor >>>>>>>> and spends time parsing changelogs: >>>>>>>> hudson.scm.SubversionChangeLogParser.parse >>>>>>>> >>>>>>>> I guess you have quite large instance, otherwise this operation would >>>>>>>> be quite >>>>>>>> fast. >>>>>>>> If you have some job, which has set up option to send an email to devs >>>>>>>> who >>>>>>>> broke the build, if the user hasn't specified an email, Jenkins tries >>>>>>>> to find it >>>>>>>> e.g. in git or SVN changelogs and search all projects and builds so if >>>>>>>> you >>>>>>>> have large instance with several dozen thousands of builds if can take >>>>>>>> pretty >>>>>>>> long time. >>>>>>>> >>>>>>>> You can fix it by setting up correct email for the user. >>>>>>>> If you have installed git plugin, make sure you have 1.1.16 (I hope it >>>>>>>> was >>>>>>>> fixed in this version) or higher. Git plugin made this search even if >>>>>>>> the user >>>>>>>> has set up email correctly >>>>>>>> >>>>>>>> On Wednesday 01 August 2012 10:56:43 Andrew Melo wrote: >>>>>>>>> On Wed, Aug 1, 2012 at 10:36 AM, Vojtech Juranek <vjura...@redhat.com> >>>>>>>> wrote: >>>>>>>>> > On Wednesday 01 August 2012 10:07:15 Andrew Melo wrote: >>>>>>>>> >> On Wed, Aug 1, 2012 at 9:48 AM, Vojtech Juranek >>>>>>>>> >> <vjura...@redhat.com> >>>>>>>> wrote: >>>>>>>>> >> > quick way how to look what the thread consuming CPU is doing is >>>>>>>>> >> > to do >>>>>>>>> >> > thread dump (e.g. using jstack $PID) and use top with threads on >>>>>>>>> >> > (H >>>>>>>>> >> > option) and then look up, see e.g. >>>>>>>>> >> > http://code.nomad-labs.com/2010/11/18/identifying-which-java-thread-is- >>>>>>>>> >> > consuming-most-cpu/ >>>>>>>>> >> >>>>>>>>> >> I see. I apparently don't have jstack on this machine :/. Does it >>>>>>>>> >> only >>>>>>>>> >> come with the JDK, or can I find it somewhere on the JRE? Once I >>>>>>>>> >> find >>>>>>>>> >> the offending thread, should it be pretty obvious what it does? >>>>>>>>> > >>>>>>>>> > jstack is part of JDK >>>>>>>>> > >>>>>>>>> > you can see the stack trace via Jenkins UI navigating to >>>>>>>>> > $JENKINS_URL/threadDump but not sure if your (or any) Jenkins >>>>>>>>> > version >>>>>>>>> > provides thread IDs. >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > Once you identify the offending thread, it should be obvious what >>>>>>>>> > it does >>>>>>>>> > (but it may not be obvious why it does what it does:-) >>>>>>>>> >>>>>>>>> Okay, I installed the jdk, and I looked some more. >>>>>>>>> >>>>>>>>> Using top, I see one jenkins thread taking the lionsshare of the time: >>>>>>>>> >>>>>>>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >>>>>>>>> 24580 jenkins 25 0 3246m 743m 18m R 88.6 24.7 790:53.39 java >>>>>>>>> 24591 jenkins 15 0 3246m 743m 18m S 0.0 24.7 40:14.51 java >>>>>>>>> 25163 jenkins 15 0 3246m 743m 18m S 0.0 24.7 28:21.42 java >>>>>>>>> 24601 jenkins 15 0 3246m 743m 18m S 0.0 24.7 27:24.95 java >>>>>>>>> 24581 jenkins 15 0 3246m 743m 18m S 0.0 24.7 26:39.58 java >>>>>>>>> 24589 jenkins 18 0 3246m 743m 18m S 0.2 24.7 24:41.60 java >>>>>>>>> 24604 jenkins 15 0 3246m 743m 18m S 0.0 24.7 23:47.46 java >>>>>>>>> 24603 jenkins 15 0 3246m 743m 18m S 0.6 24.7 17:05.23 java >>>>>>>>> 24484 jenkins 15 0 3246m 743m 18m S 0.4 24.7 14:45.39 java >>>>>>>>> 24612 jenkins 18 0 3246m 743m 18m S 0.0 24.7 11:50.45 java >>>>>>>>> 24610 jenkins 15 0 3246m 743m 18m S 0.0 24.7 10:34.41 java >>>>>>>>> 24564 jenkins 15 0 3246m 743m 18m S 0.0 24.7 8:56.60 java >>>>>>>>> 24602 jenkins 15 0 3246m 743m 18m S 0.0 24.7 8:30.98 java >>>>>>>>> 24565 jenkins 16 0 3246m 743m 18m S 11.5 24.7 8:22.85 java >>>>>>>>> 24609 jenkins 15 0 3246m 743m 18m S 0.0 24.7 8:12.30 java >>>>>>>>> 24582 jenkins 15 0 3246m 743m 18m S 0.6 24.7 3:48.67 java >>>>>>>>> 24590 jenkins 15 0 3246m 743m 18m S 0.0 24.7 3:24.27 java >>>>>>>>> 24579 jenkins 15 0 3246m 743m 18m S 0.0 24.7 3:22.16 java >>>>>>>>> 24486 jenkins 15 0 3246m 743m 18m S 0.0 24.7 2:33.77 java >>>>>>>>> 24973 jenkins 15 0 3246m 743m 18m S 0.0 24.7 2:18.32 java >>>>>>>>> 24983 jenkins 15 0 3246m 743m 18m S 0.0 24.7 2:07.91 java >>>>>>>>> 24838 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:38.35 java >>>>>>>>> 24845 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:32.56 java >>>>>>>>> 25037 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:16.63 java >>>>>>>>> 25038 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:07.00 java >>>>>>>>> 24491 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:05.38 java >>>>>>>>> 24611 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:02.82 java >>>>>>>>> 24488 jenkins 15 0 3246m 743m 18m S 0.0 24.7 1:00.30 java >>>>>>>>> >>>>>>>>> >>>>>>>>> Then if I run jstack, I get the following backtrace: >>>>>>>>> >>>>>>>>> https://gist.github.com/3228105 >>>>>>>>> >>>>>>>>> Does that look useful at all? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Andrew >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Website: http://earl-of-code.com >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Andrew Melo >>>>> >>>>> >>>>> >>>>> -- >>>>> Website: http://earl-of-code.com >>>> >>>> >>>> >>>> -- >>>> -- >>>> Andrew Melo >>> >>> >>> >>> -- >>> Website: http://earl-of-code.com >> >> >> >> -- >> -- >> Andrew Melo > > > > -- > -- > Andrew Melo -- -- Andrew Melo -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
