[
https://issues.apache.org/jira/browse/SOLR-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580112#comment-13580112
]
Raintung Li commented on SOLR-4408:
-----------------------------------
If SolrIndexSearcher can be used only finish execute the listeners one by one,
all listener can't block the thread. Otherwise can define the other thread to
monitor the execute complete then notify wait for getSearch method that easy to
troubleshooting. If the listeners don't need execute order sequence necessary,
the multiple threads concurrent handle the listeners to improve the start
efficiency.
> Server hanging on startup
> -------------------------
>
> Key: SOLR-4408
> URL: https://issues.apache.org/jira/browse/SOLR-4408
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.1
> Environment: OpenJDK 64-Bit Server VM (23.2-b09 mixed mode)
> Tomcat 7.0
> Eclipse Juno + WTP
> Reporter: Francois-Xavier Bonnet
> Assignee: Erick Erickson
> Fix For: 4.2, 5.0
>
> Attachments: patch-4408.txt
>
>
> While starting, the server hangs indefinitely. Everything works fine when I
> first start the server with no index created yet but if I fill the index then
> stop and start the server, it hangs. Could it be a lock that is never
> released?
> Here is what I get in a full thread dump:
> 2013-02-06 16:28:52
> Full thread dump OpenJDK 64-Bit Server VM (23.2-b09 mixed mode):
> "searcherExecutor-4-thread-1" prio=10 tid=0x00007fbdfc16a800 nid=0x42c6 in
> Object.wait() [0x00007fbe0ab10000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c34c1c48> (a java.lang.Object)
> at java.lang.Object.wait(Object.java:503)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1492)
> - locked <0x00000000c34c1c48> (a java.lang.Object)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1312)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1247)
> at
> org.apache.solr.request.SolrQueryRequestBase.getSearcher(SolrQueryRequestBase.java:94)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:213)
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:112)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:203)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:180)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816)
> at
> org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64)
> at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1594)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> "coreLoadExecutor-3-thread-1" prio=10 tid=0x00007fbe04194000 nid=0x42c5 in
> Object.wait() [0x00007fbe0ac11000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c34c1c48> (a java.lang.Object)
> at java.lang.Object.wait(Object.java:503)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1492)
> - locked <0x00000000c34c1c48> (a java.lang.Object)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1312)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1247)
> at
> org.apache.solr.handler.ReplicationHandler.getIndexVersion(ReplicationHandler.java:495)
> at
> org.apache.solr.handler.ReplicationHandler.getStatistics(ReplicationHandler.java:518)
> at
> org.apache.solr.core.JmxMonitoredMap$SolrDynamicMBean.getMBeanInfo(JmxMonitoredMap.java:232)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:333)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:319)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:512)
> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
> at
> org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:636)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:809)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:607)
> at
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1003)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1033)
> at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
> at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:624)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> "Thread-3" daemon prio=10 tid=0x00007fbe40742800 nid=0x42c2 waiting on
> condition [0x00007fbe0ad42000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ef8e6a08> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ExecutorCompletionService.take(ExecutorCompletionService.java:193)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:662)
> at
> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:340)
> at
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:98)
> at
> org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:273)
> at
> org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:254)
> at
> org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:372)
> at
> org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:98)
> at
> org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4461)
> - locked <0x00000000c2e916f8> (a java.util.HashMap)
> at
> org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5133)
> at
> org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5128)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> "GC Daemon" daemon prio=10 tid=0x00007fbe405af800 nid=0x42bf in Object.wait()
> [0x00007fbe383c6000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c2e46c30> (a sun.misc.GC$LatencyLock)
> at sun.misc.GC$Daemon.run(GC.java:117)
> - locked <0x00000000c2e46c30> (a sun.misc.GC$LatencyLock)
> "Service Thread" daemon prio=10 tid=0x00007fbe4010c000 nid=0x42bb runnable
> [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x00007fbe4010a000 nid=0x42ba waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x00007fbe40107000 nid=0x42b9 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x00007fbe40104800 nid=0x42b8 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x00007fbe400ad800 nid=0x42b7 in Object.wait()
> [0x00007fbe444d0000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c2e46ea8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x00000000c2e46ea8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
> "Reference Handler" daemon prio=10 tid=0x00007fbe400ab800 nid=0x42b6 in
> Object.wait() [0x00007fbe445d1000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c2e46c10> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x00000000c2e46c10> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x00007fbe40009800 nid=0x42b1 waiting on condition
> [0x00007fbe49487000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000c34bdc68> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.tomcat.util.threads.DedicatedThreadExecutor.execute(DedicatedThreadExecutor.java:56)
> at
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5127)
> - locked <0x00000000c2e90d70> (a
> org.apache.catalina.core.StandardContext)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> - locked <0x00000000c2e90d70> (a
> org.apache.catalina.core.StandardContext)
> at
> org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1035)
> - locked <0x00000000c2e90a08> (a org.apache.catalina.core.StandardHost)
> at
> org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:774)
> - locked <0x00000000c2e90a08> (a org.apache.catalina.core.StandardHost)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> - locked <0x00000000c2e90a08> (a org.apache.catalina.core.StandardHost)
> at
> org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1035)
> - locked <0x00000000c2e90850> (a
> org.apache.catalina.core.StandardEngine)
> at
> org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:291)
> - locked <0x00000000c2e90850> (a
> org.apache.catalina.core.StandardEngine)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> - locked <0x00000000c2e90850> (a
> org.apache.catalina.core.StandardEngine)
> at
> org.apache.catalina.core.StandardService.startInternal(StandardService.java:443)
> - locked <0x00000000c2e90850> (a
> org.apache.catalina.core.StandardEngine)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> - locked <0x00000000c2e8f640> (a
> org.apache.catalina.core.StandardService)
> at
> org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:724)
> - locked <0x00000000c2e8f628> (a [Lorg.apache.catalina.Service;)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> - locked <0x00000000c2e471c0> (a
> org.apache.catalina.core.StandardServer)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:620)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:304)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)
> "VM Thread" prio=10 tid=0x00007fbe400a3000 nid=0x42b5 runnable
> "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007fbe40017000 nid=0x42b3
> runnable
> "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007fbe40019000 nid=0x42b4
> runnable
> "VM Periodic Task Thread" prio=10 tid=0x00007fbe40116800 nid=0x42bc waiting
> on condition
> JNI global references: 189
> Heap
> PSYoungGen total 65280K, used 48438K [0x00000000eba00000,
> 0x00000000f3980000, 0x0000000100000000)
> eden space 62720K, 73% used
> [0x00000000eba00000,0x00000000ee6d19b8,0x00000000ef740000)
> from space 2560K, 99% used
> [0x00000000ef740000,0x00000000ef9bc010,0x00000000ef9c0000)
> to space 2560K, 0% used
> [0x00000000f3700000,0x00000000f3700000,0x00000000f3980000)
> ParOldGen total 41728K, used 9077K [0x00000000c2e00000,
> 0x00000000c56c0000, 0x00000000eba00000)
> object space 41728K, 21% used
> [0x00000000c2e00000,0x00000000c36dd478,0x00000000c56c0000)
> PSPermGen total 25280K, used 25218K [0x00000000b8800000,
> 0x00000000ba0b0000, 0x00000000c2e00000)
> object space 25280K, 99% used
> [0x00000000b8800000,0x00000000ba0a0b50,0x00000000ba0b0000)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
