Re: commit fail

2012-04-30 Thread Erick Erickson 
In the 3.6 world, LukeRequestHandler does some...er...really expensive
things when you click into the admin/schema browser. This is _much_
better in trunk BTW.

So, as Yonik says, LukeRequestHandler probably accounts for
one of the threads.

Does this occur when nobody is playing around with the admin
handler?

Erick

On Sat, Apr 28, 2012 at 10:03 AM, Yonik Seeley
yo...@lucidimagination.com wrote:
 On Sat, Apr 28, 2012 at 7:02 AM, mav.p...@holidaylettings.co.uk
 mav.p...@holidaylettings.co.uk wrote:
 Hi,

 This is what the thread dump looks like.

 Any ideas?

 Looks like the thread taking up CPU is in LukeRequestHandler

 1062730578@qtp-1535043768-5' Id=16, RUNNABLE on lock=, total cpu
 time=16156160.ms user time=16153110.msat
 org.apache.solr.handler.admin.LukeRequestHandler.getIndexedFieldsInfo(LukeR
 equestHandler.java:320)

 That probably accounts for the 1 CPU doing things... but it's not
 clear at all why commits are failing.

 Perhaps the commit is succeeding, but the client is just not waiting
 long enough for it to complete?

 -Yonik
 lucenerevolution.com - Lucene/Solr Open Source Search Conference.
 Boston May 7-10

Re: commit fail

2012-04-28 Thread mav.p...@holidaylettings.co.uk 
Hi,

This is what the thread dump looks like.

Any ideas?

Mav

Java HotSpot(TM) 64-Bit Server VM20.1-b02Thread Count: current=19,
peak=20, daemon=6'DestroyJavaVM' Id=26, RUNNABLE on lock=, total cpu
time=198450.ms user time=196890.ms'Timer-2' Id=25, TIMED_WAITING
on lock=java.util.TaskQueue@33799a1e, total cpu time=0.ms user
time=0.msat java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:509)
at java.util.TimerThread.run(Timer.java:462)
'pool-3-thread-1' Id=24, WAITING on
lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@
747541f8, total cpu time=0.ms user time=0.msat
sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
(AbstractQueuedSynchronizer.java:1987)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947
) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
907) 
at java.lang.Thread.run(Thread.java:662)
'pool-1-thread-1' Id=23, WAITING on
lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@
3e3e3c83, total cpu time=480.ms user time=460.msat
sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
(AbstractQueuedSynchronizer.java:1987)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947
) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
907) 
at java.lang.Thread.run(Thread.java:662)
'Timer-1' Id=21, TIMED_WAITING on lock=java.util.TaskQueue@67f6dc61, total
cpu time=180.ms user time=120.msat java.lang.Object.wait(Native
Method) 
at java.util.TimerThread.mainLoop(Timer.java:509)
at java.util.TimerThread.run(Timer.java:462)
'2021372560@qtp-1535043768-9 - Acceptor0 SocketConnector@0.0.0.0:8983'
Id=20, RUNNABLE on lock=, total cpu time=60.ms user time=60.msat
java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at org.mortbay.jetty.bio.SocketConnector.accept(SocketConnector.java:99)
at 
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708
) 
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:58
2) 
'1384828782@qtp-1535043768-8' Id=19, TIMED_WAITING on
lock=org.mortbay.thread.QueuedThreadPool$PoolThread@528acf6e, total cpu
time=274160.ms user time=273060.msat java.lang.Object.wait(Native
Method) 
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:62
6) 
'1715374531@qtp-1535043768-7' Id=18, RUNNABLE on lock=, total cpu
time=15725890.ms user time=15723380.msat
sun.management.ThreadImpl.getThreadInfo1(Native Method)
at sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:154)
at 
org.apache.jsp.admin.threaddump_jsp._jspService(org.apache.jsp.admin.thread
dump_jsp:264) 
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at 
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:
389) 
at 
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
 
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:327)
at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java
:275) 
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandle
r.java:1212) 
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
 
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at

Re: commit fail

2012-04-28 Thread Yonik Seeley 
On Sat, Apr 28, 2012 at 7:02 AM, mav.p...@holidaylettings.co.uk
mav.p...@holidaylettings.co.uk wrote:
 Hi,

 This is what the thread dump looks like.

 Any ideas?

Looks like the thread taking up CPU is in LukeRequestHandler

 1062730578@qtp-1535043768-5' Id=16, RUNNABLE on lock=, total cpu
 time=16156160.ms user time=16153110.msat
 org.apache.solr.handler.admin.LukeRequestHandler.getIndexedFieldsInfo(LukeR
 equestHandler.java:320)

That probably accounts for the 1 CPU doing things... but it's not
clear at all why commits are failing.

Perhaps the commit is succeeding, but the client is just not waiting
long enough for it to complete?

-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10

Re: commit fail

2012-04-27 Thread Yonik Seeley 
On Fri, Apr 27, 2012 at 8:23 PM, mav.p...@holidaylettings.co.uk
mav.p...@holidaylettings.co.uk wrote:
 Hi again,

 This is the only log entry I can find, regarding the failed commits…

 Still timing out as far as the client is concerned and there is actually 
 nothing happening on the server in terms of load (staging environment).

 1 CPU core seems busy constantly with solr but unsure what is happening.

You can get a thread dump to see what the various threads are doing
(use the solr admin, or kill -3).  Sounds like it could just be either
merging in progress or a commit in progress.

-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10