Issue Type: Bug Bug
Affects Versions: current
Assignee: abayer
Components: throttle-concurrents
Created: 17/Dec/13 2:34 PM
Description:

Jenkins had stopped responding to browser requests for Jenkins pages and I think it may be caused by the recent upgrade to Throttle Concurrent Builds 1.8.1

Requests were getting blocked waiting on 0x00000004181e4520

"Handling GET /jenkins/ : RequestHandlerThread[#171]" daemon prio=10 tid=0x00000000168ee800 nid=0x193b waiting for monitor entry [0x000000004335b000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at hudson.model.Queue.getItems(Queue.java:687)
	- waiting to lock <0x00000004181e4520> (a hudson.model.Queue)
	at hudson.model.Queue$CachedItemList.get(Queue.java:216)
	at hudson.model.Queue.getApproximateItemsQuickly(Queue.java:717)
	at hudson.model.View.getApproximateQueueItemsQuickly(View.java:483)
	at sun.reflect.GeneratedMethodAccessor355.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

This seems to be waiting on Throttle Concurrent Builds code.
Further dumps after 10 minutes, 20 minutes and 30 minutes showed this same stack trace.

"Thread-126" daemon prio=10 tid=0x00002aaae0529800 nid=0x1785 runnable [0x0000000046590000]
   java.lang.Thread.State: RUNNABLE
	at java.util.WeakHashMap$HashIterator.hasNext(WeakHashMap.java:875)
	at java.util.AbstractCollection.toArray(AbstractCollection.java:139)
	at java.util.ArrayList.<init>(ArrayList.java:164)
	at hudson.plugins.throttleconcurrents.ThrottleJobProperty.getCategoryProjects(ThrottleJobProperty.java:141)
	- locked <0x000000041a79b778> (a java.util.HashMap)
	at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:118)
	at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:90)
	at hudson.model.Queue.isBuildBlocked(Queue.java:937)
	at hudson.model.Queue.maintain(Queue.java:1006)
	- locked <0x00000004181e4520> (a hudson.model.Queue)
	at hudson.model.Queue$1.call(Queue.java:303)
	at hudson.model.Queue$1.call(Queue.java:300)
	at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:69)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:104)
	at java.lang.Thread.run(Thread.java:724)

   Locked ownable synchronizers:
	- None

CPU usage was at ~100% for this thread for the 30 minutes that I was watching it before I restarted Jenkins.
(6021 = 0x1785)

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                   
 6021 rcbuild_  35  10 18.0g 6.2g  32m R 99.7 19.7  30:45.97 java

I have rolled back to Throttle Concurrent Builds 1.8 for now.

Still learning how to investigate thread dumps but please let me know if there is anything I can do to help.

Environment: Jenkins 1.544
Project: Jenkins
Priority: Critical Critical
Reporter: Geoff Cummings
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Reply via email to