[
https://issues.apache.org/jira/browse/LUCENE-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630270#action_12630270
]
Otis Gospodnetic commented on LUCENE-1381:
------------------------------------------
David, why not bring this up on java-user list first?
Are your 4 IndexWriters writing to the same index?
> Hanging while indexing/digesting on multiple threads
> ----------------------------------------------------
>
> Key: LUCENE-1381
> URL: https://issues.apache.org/jira/browse/LUCENE-1381
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 2.3.2
> Environment: Java HotSpot(TM) 64-Bit Server VM (1.5.0_16-b02 mixed
> mode) on 2.6.9-78.0.1.ELsmp #1 SMP x86_64 x86_64 x86_64 GNU/Linux
> Reporter: David Fertig
>
> With several older lucene projects already running and "stable", I have
> recently written a multi-threading indexer using to the 2.3.2 release.
> My volume is in the millions of documents indexed daily and I have been
> stress testing for a while now. My current setup has 3 JVMs, each running 6
> threads indexing different documents, with 1 IndexWriter per JVM. For
> stability testing, the indexer shutsdown and exits every 5-10 minutes, with a
> new JVM is started again for a clean restart. At this rate, I have noticed an
> rare, but eventually consistent internal hang/deadlock in all indexer threads
> while parsing documents. My 'manager' thread is alive and regularly polling
> the indexer threads and displaying their state variables, but the indexer
> threads themselves appear not to be making progress while using up nearly
> 100% of available CPU. Memory usage is relativly low and stable at 481m out
> of 2048m available.
> Most stack traces, and STAY in this state even after repeated inspections:
> (pressing CTRL-\ in active JVM window)
> ----------
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.5.0_16-b02 mixed mode):
> "Thread-6" prio=1 tid=0x0000002b25750920 nid=0x34f6 runnable
> [0x0000000041465000..0x0000000041465db0]
> at java.util.WeakHashMap.eq(WeakHashMap.java:254)
> at java.util.WeakHashMap.get(WeakHashMap.java:345)
> at
> org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:530)
> at
> org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:209)
> at
> org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:625)
> at org.apache.commons.digester.Rule.end(Rule.java:230)
> at org.apache.commons.digester.Digester.endElement(Digester.java:1130)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1241)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1685)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> at org.apache.commons.digester.Digester.parse(Digester.java:1685)
> ...
> "Thread-5" prio=1 tid=0x0000002b25754eb0 nid=0x34f5 runnable
> [0x0000000041364000..0x0000000041364d30]
> at java.lang.String.equals(String.java:858)
> at
> org.apache.commons.beanutils.MethodUtils$MethodDescriptor.equals(MethodUtils.java:833)
> at java.util.WeakHashMap.eq(WeakHashMap.java:254)
> at java.util.WeakHashMap.get(WeakHashMap.java:345)
> at
> org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:530)
> at
> org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:209)
> at
> org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:625)
> at org.apache.commons.digester.Rule.end(Rule.java:230)
> at org.apache.commons.digester.Digester.endElement(Digester.java:1130)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1241)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1685)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> at org.apache.commons.digester.Digester.parse(Digester.java:1685)
> ...
> "Thread-4" prio=1 tid=0x0000002b25754860 nid=0x34f4 runnable
> [0x0000000041263000..0x0000000041263cb0]
> at java.lang.String.equals(String.java:858)
> at
> org.apache.commons.beanutils.MethodUtils$MethodDescriptor.equals(MethodUtils.java:833)
> at java.util.WeakHashMap.eq(WeakHashMap.java:254)
> at java.util.WeakHashMap.get(WeakHashMap.java:345)
> at
> org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:530)
> at
> org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:209)
> at
> org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:625)
> at org.apache.commons.digester.Rule.end(Rule.java:230)
> at org.apache.commons.digester.Digester.endElement(Digester.java:1130)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1241)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1685)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> at org.apache.commons.digester.Digester.parse(Digester.java:1685)
> ...
> "Thread-3" prio=1 tid=0x0000002b2509d360 nid=0x34f3 runnable
> [0x0000000041162000..0x0000000041162c30]
> at java.lang.String.equals(String.java:858)
> at
> org.apache.commons.beanutils.MethodUtils$MethodDescriptor.equals(MethodUtils.java:833)
> at java.util.WeakHashMap.eq(WeakHashMap.java:254)
> at java.util.WeakHashMap.get(WeakHashMap.java:345)
> at
> org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:530)
> at
> org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:209)
> at
> org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:625)
> at org.apache.commons.digester.Rule.end(Rule.java:230)
> at org.apache.commons.digester.Digester.endElement(Digester.java:1130)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1241)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1685)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> at org.apache.commons.digester.Digester.parse(Digester.java:1685)
> ...
> "Thread-2" prio=1 tid=0x0000002b25083e00 nid=0x34f2 runnable
> [0x0000000041061000..0x0000000041061bb0]
> at java.util.WeakHashMap.expungeStaleEntries(WeakHashMap.java:289)
> at java.util.WeakHashMap.getTable(WeakHashMap.java:297)
> at java.util.WeakHashMap.get(WeakHashMap.java:341)
> at
> org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:530)
> at
> org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:209)
> at
> org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:625)
> at org.apache.commons.digester.Rule.end(Rule.java:230)
> at org.apache.commons.digester.Digester.endElement(Digester.java:1130)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:221)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:872)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1693)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> at org.apache.commons.digester.Digester.parse(Digester.java:1685)
> ...
> "Thread-1" prio=1 tid=0x0000002b2509a4a0 nid=0x34f1 runnable
> [0x0000000040f60000..0x0000000040f60b30]
> at java.util.WeakHashMap.expungeStaleEntries(WeakHashMap.java:289)
> at java.util.WeakHashMap.getTable(WeakHashMap.java:297)
> at java.util.WeakHashMap.get(WeakHashMap.java:341)
> at
> org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:530)
> at
> org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:209)
> at
> org.apache.commons.digester.CallMethodRule.end(CallMethodRule.java:625)
> at org.apache.commons.digester.Rule.end(Rule.java:230)
> at org.apache.commons.digester.Digester.endElement(Digester.java:1130)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:633)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:221)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:872)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(XMLDocumentFragmentScannerImpl.java:1693)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> at org.apache.commons.digester.Digester.parse(Digester.java:1685)
> ...
> "Low Memory Detector" daemon prio=1 tid=0x0000002b25002430 nid=0x34ef
> runnable [0x0000000000000000..0x0000000000000000]
> "CompilerThread1" daemon prio=1 tid=0x0000002b250009d0 nid=0x34ee waiting on
> condition [0x0000000000000000..0x0000000040c5c6d0]
> "CompilerThread0" daemon prio=1 tid=0x0000002b220c2ce0 nid=0x34ed waiting on
> condition [0x0000000000000000..0x0000000040b5b650]
> "AdapterThread" daemon prio=1 tid=0x0000002b220c1750 nid=0x34ec waiting on
> condition [0x0000000000000000..0x0000000000000000]
> "Signal Dispatcher" daemon prio=1 tid=0x0000002b220c02d0 nid=0x34eb runnable
> [0x0000000000000000..0x0000000000000000]
> "Finalizer" daemon prio=1 tid=0x0000002b220ae1e0 nid=0x34ea in Object.wait()
> [0x0000000040859000..0x0000000040859bb0]
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000002b0a8d7408> (a
> java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
> - locked <0x0000002b0a8d7408> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:136)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
> "Reference Handler" daemon prio=1 tid=0x0000002b220ab980 nid=0x34e9 in
> Object.wait() [0x0000000040758000..0x0000000040758b30]
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000002b0a7da3b0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:474)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0x0000002b0a7da3b0> (a java.lang.ref.Reference$Lock)
> "main" prio=1 tid=0x0000000040115ea0 nid=0x34e3 waiting on condition
> [0x0000007fbfffc000..0x0000007fbfffd280]
> at java.lang.Thread.sleep(Native Method)
> ...
> "VM Thread" prio=1 tid=0x0000002b220a7480 nid=0x34e8 runnable
> "GC task thread#0 (ParallelGC)" prio=1 tid=0x0000000040133fe0 nid=0x34e4
> runnable
> "GC task thread#1 (ParallelGC)" prio=1 tid=0x0000000040135630 nid=0x34e5
> runnable
> "GC task thread#2 (ParallelGC)" prio=1 tid=0x0000000040136480 nid=0x34e6
> runnable
> "GC task thread#3 (ParallelGC)" prio=1 tid=0x0000000040137300 nid=0x34e7
> runnable
> "VM Periodic Task Thread" prio=1 tid=0x0000002b25004e10 nid=0x34f0 waiting on
> condition
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
