Well, in the same processes I am using a jdbc connection to get all the relative paths to the documents I want to index, then I parse the documents to plain text using tones of open source libraries like POI, PFDBox etc.(which might account for java2d) then I add them to the index and commit every 2000 documents.
I write a db row for each row I index so I can resume where I left off after a crash or exception. My app will happily index for hours then after it hangs, the resumed indexing run will only last a few additional minutes! The thread dumps look the same. Cheers. ryantxu wrote: > > do you have anything custom going on? > > The fact that the lock is in java2d seems suspicious... > > > On Sep 23, 2009, at 7:01 PM, pof wrote: > >> >> I had the same problem again yesterday except the process halted >> after about >> 20mins this time. >> >> >> pof wrote: >>> >>> Hello, I was running a batch index the other day using the Solrj >>> EmbeddedSolrServer when the process abruptly froze in it's tracks >>> after >>> running for about 4-5 hours and indexing ~400K documents. There >>> were no >>> document locks so it would seem likely that there was some kind of >>> thread >>> deadlock. I was hoping someone might be able to tell me some >>> information >>> about the following thread dump taken at the time: >>> >>> Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode): >>> >>> "DestroyJavaVM" prio=10 tid=0x9322a800 nid=0xcef waiting on condition >>> [0x00000000..0x0018a044] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Java2D Disposer" daemon prio=10 tid=0x0a28cc00 nid=0xf1c in >>> Object.wait() >>> [0x0311d000..0x0311def4] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x97a96840> (a java.lang.ref.ReferenceQueue >>> $Lock) >>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >>> 133) >>> - locked <0x97a96840> (a java.lang.ref.ReferenceQueue$Lock) >>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >>> 149) >>> at sun.java2d.Disposer.run(Disposer.java:143) >>> at java.lang.Thread.run(Thread.java:636) >>> >>> "pool-1-thread-1" prio=10 tid=0x93a26c00 nid=0xcf7 waiting on >>> condition >>> [0x08a6a000..0x08a6b074] >>> java.lang.Thread.State: WAITING (parking) >>> at sun.misc.Unsafe.park(Native Method) >>> - parking to wait for <0x967acfd0> (a >>> java.util.concurrent.locks.AbstractQueuedSynchronizer >>> $ConditionObject) >>> at >>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) >>> at >>> java.util.concurrent.locks.AbstractQueuedSynchronizer >>> $ConditionObject.await(AbstractQueuedSynchronizer.java:1978) >>> at >>> java >>> .util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java: >>> 386) >>> at >>> java >>> .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java: >>> 1043) >>> at >>> java >>> .util >>> .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >>> 1103) >>> at >>> java.util.concurrent.ThreadPoolExecutor >>> $Worker.run(ThreadPoolExecutor.java:603) >>> at java.lang.Thread.run(Thread.java:636) >>> >>> "Low Memory Detector" daemon prio=10 tid=0x93a00c00 nid=0xcf5 >>> runnable >>> [0x00000000..0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "CompilerThread0" daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on >>> condition [0x00000000..0x096a7af4] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Signal Dispatcher" daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting >>> on >>> condition [0x00000000..0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Finalizer" daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait() >>> [0x005ca000..0x005caef4] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x966e6d40> (a java.lang.ref.ReferenceQueue >>> $Lock) >>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >>> 133) >>> - locked <0x966e6d40> (a java.lang.ref.ReferenceQueue$Lock) >>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >>> 149) >>> at java.lang.ref.Finalizer >>> $FinalizerThread.run(Finalizer.java:177) >>> >>> "Reference Handler" daemon prio=10 tid=0x09fd2c00 nid=0xcf1 in >>> Object.wait() [0x00579000..0x00579d74] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x966e6dc8> (a java.lang.ref.Reference$Lock) >>> at java.lang.Object.wait(Object.java:502) >>> at >>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) >>> - locked <0x966e6dc8> (a java.lang.ref.Reference$Lock) >>> >>> "VM Thread" prio=10 tid=0x09fcf800 nid=0xcf0 runnable >>> >>> "VM Periodic Task Thread" prio=10 tid=0x93a02400 nid=0xcf6 waiting on >>> condition >>> >>> JNI global references: 1072 >>> >>> Heap >>> def new generation total 36288K, used 23695K [0x93f10000, >>> 0x96670000, >>> 0x96670000) >>> eden space 32256K, 73% used [0x93f10000, 0x95633f60, 0x95e90000) >>> from space 4032K, 0% used [0x95e90000, 0x95e90000, 0x96280000) >>> to space 4032K, 0% used [0x96280000, 0x96280000, 0x96670000) >>> tenured generation total 483968K, used 72129K [0x96670000, >>> 0xb3f10000, >>> 0xb3f10000) >>> the space 483968K, 14% used [0x96670000, 0x9ace04b8, 0x9ace0600, >>> 0xb3f10000) >>> compacting perm gen total 23040K, used 22983K [0xb3f10000, >>> 0xb5590000, >>> 0xb7f10000) >>> the space 23040K, 99% used [0xb3f10000, 0xb5581ff8, 0xb5582000, >>> 0xb5590000) >>> No shared spaces configured. >>> >>> Cheers. Brett. >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Solrj-possible-deadlock-tp25530146p25531321.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > > > -- View this message in context: http://www.nabble.com/Solrj-possible-deadlock-tp25530146p25604919.html Sent from the Solr - User mailing list archive at Nabble.com.
