[
https://issues.apache.org/jira/browse/SOLR-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533820#comment-13533820
]
Dawid Weiss commented on SOLR-4205:
-----------------------------------
I pushed yet another minor release (it's damn hard to test against third party
libraries with ivy...) -- 2.0.7. It just contains an explicit flush but I don't
think this will make a difference.
Uwe, could you upgrade the ivy files and enable the build again? we'll see if
this helps or changes the behavior anyhow. I'm busy for the rest of the day but
I'll take a look at the results in the evening.
> Clover runs on ASF Jenkins idle dead without a test or any thread running in
> main() loop waiting for file descriptor
> --------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-4205
> URL: https://issues.apache.org/jira/browse/SOLR-4205
> Project: Solr
> Issue Type: Bug
> Components: Tests
> Environment: FreeBSD Jenkins
> Reporter: Uwe Schindler
> Assignee: Dawid Weiss
>
> I had to kill ASF Jenkins Clover builds two times after several 10 hours of
> inactivity in a random Solr test. I requested a stack trace before killing
> the only running JVM (clover runs with one JVM only, because clover does not
> like multiple processes writing the same clover metrics file).
> In both cases (4.x and trunk) the stack trace was looking identical after
> sending kill -3...
> https://builds.apache.org/job/Lucene-Solr-Clover-trunk/76/consoleFull
> (yesterday):
> {noformat}
> [junit4:junit4] HEARTBEAT J0 PID(81...@lucene.zones.apache.org):
> 2012-12-16T13:01:00, stalled for 28447s at:
> TestFunctionQuery.testBooleanFunctions
> [junit4:junit4] HEARTBEAT J0 PID(81...@lucene.zones.apache.org):
> 2012-12-16T13:02:00, stalled for 28507s at:
> TestFunctionQuery.testBooleanFunctions
> [junit4:junit4] HEARTBEAT J0 PID(81...@lucene.zones.apache.org):
> 2012-12-16T13:03:00, stalled for 28567s at:
> TestFunctionQuery.testBooleanFunctions
> [junit4:junit4] JVM J0: stdout was not empty, see:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Clover-trunk/solr/build/solr-core/test/temp/junit4-J0-20121216_044733_583.sysout
> [junit4:junit4] >>> JVM J0: stdout (verbatim) ----
> [junit4:junit4] 2012-12-16 13:03:49
> [junit4:junit4] Full thread dump OpenJDK 64-Bit Server VM (20.0-b12 mixed
> mode):
> [junit4:junit4]
> [junit4:junit4] "searcherExecutor-2577-thread-1" prio=5
> tid=0x000000085eb67000 nid=0x61c105b waiting on condition [0x00007ffff0b0d000]
> [junit4:junit4] java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x00000008178c9c40> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4] at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000840ce2800
> nid=0x61c0aa2 runnable [0x00007ffff9496000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
> [junit4:junit4] at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> [junit4:junit4] at
> java.net.ServerSocket.implAccept(ServerSocket.java:470)
> [junit4:junit4] at java.net.ServerSocket.accept(ServerSocket.java:438)
> [junit4:junit4] at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:387)
> [junit4:junit4] at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI Scheduler(0)" daemon prio=5 tid=0x0000000840ce1000
> nid=0x61c0969 waiting on condition [0x00007ffff0f11000]
> [junit4:junit4] java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000814f12f88> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4] at
> java.util.concurrent.DelayQueue.take(DelayQueue.java:189)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "GC Daemon" daemon prio=5 tid=0x0000000840ce0000
> nid=0x61c0949 in Object.wait() [0x00007ffff6d6f000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at sun.misc.GC$Daemon.run(GC.java:117)
> [junit4:junit4] - locked <0x0000000814f1c428> (a
> sun.misc.GC$LatencyLock)
> [junit4:junit4]
> [junit4:junit4] "RMI Reaper" prio=5 tid=0x0000000840cdf800 nid=0x61c0947 in
> Object.wait() [0x00007ffff7577000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
> [junit4:junit4] - locked <0x0000000814f1ad08> (a
> java.lang.ref.ReferenceQueue$Lock)
> [junit4:junit4] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
> [junit4:junit4] at
> sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:350)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000840cde800
> nid=0x61c0936 runnable [0x00007ffff6e70000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
> [junit4:junit4] at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> [junit4:junit4] at
> java.net.ServerSocket.implAccept(ServerSocket.java:470)
> [junit4:junit4] at java.net.ServerSocket.accept(ServerSocket.java:438)
> [junit4:junit4] at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:387)
> [junit4:junit4] at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000840cde000
> nid=0x61c0934 runnable [0x00007ffff494b000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
> [junit4:junit4] at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> [junit4:junit4] at
> java.net.ServerSocket.implAccept(ServerSocket.java:470)
> [junit4:junit4] at java.net.ServerSocket.accept(ServerSocket.java:438)
> [junit4:junit4] at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:387)
> [junit4:junit4] at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "TimeLimitedCollector timer thread" daemon prio=5
> tid=0x0000000848e69800 nid=0x61c096d waiting on condition [0x00007ffff3a3c000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:267)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-2" daemon prio=5
> tid=0x000000084c8c6800 nid=0x61c087d runnable [0x00007ffffdfe1000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000811568070> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at
> java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-1" daemon prio=5
> tid=0x000000084c8c5000 nid=0x61c087b runnable [0x00007ffffe1e3000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000811568070> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at
> java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "Low Memory Detector" daemon prio=5 tid=0x00000008011c4800
> nid=0x61c0876 runnable [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread1" daemon prio=5 tid=0x00000008011c3800
> nid=0x61c0875 waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread0" daemon prio=5 tid=0x00000008011c3000
> nid=0x61c0874 waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Signal Dispatcher" daemon prio=5 tid=0x00000008011c2000
> nid=0x61c0873 waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Finalizer" daemon prio=5 tid=0x00000008011c1800
> nid=0x61c0872 in Object.wait() [0x00007ffffebed000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
> [junit4:junit4] - locked <0x0000000810cec8f8> (a
> java.lang.ref.ReferenceQueue$Lock)
> [junit4:junit4] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
> [junit4:junit4] at
> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
> [junit4:junit4]
> [junit4:junit4] "Reference Handler" daemon prio=5 tid=0x00000008011c0800
> nid=0x61c0871 in Object.wait() [0x00007ffffecee000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.Object.wait(Object.java:502)
> [junit4:junit4] at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> [junit4:junit4] - locked <0x0000000810ce8b80> (a
> java.lang.ref.Reference$Lock)
> [junit4:junit4]
> [junit4:junit4] "main" prio=5 tid=0x00000008011c0000 nid=0x61c0672 runnable
> [0x00007fffffbfd000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.io.FileInputStream.readBytes(Native Method)
> [junit4:junit4] at
> java.io.FileInputStream.read(FileInputStream.java:236)
> [junit4:junit4] at
> java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> [junit4:junit4] at
> java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> [junit4:junit4] - locked <0x0000000810ce22e0> (a
> java.io.BufferedInputStream)
> [junit4:junit4] at
> sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
> [junit4:junit4] at
> sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
> [junit4:junit4] - locked <0x0000000810ce22c8> (a
> java.io.InputStreamReader)
> [junit4:junit4] at
> java.io.InputStreamReader.read(InputStreamReader.java:184)
> [junit4:junit4] at java.io.BufferedReader.fill(BufferedReader.java:153)
> [junit4:junit4] at
> java.io.BufferedReader.readLine(BufferedReader.java:316)
> [junit4:junit4] - locked <0x0000000810ce22c8> (a
> java.io.InputStreamReader)
> [junit4:junit4] at
> java.io.BufferedReader.readLine(BufferedReader.java:379)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:31)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:13)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.Iterators$5.hasNext(Iterators.java:539)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:150)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:255)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
> [junit4:junit4]
> [junit4:junit4] "VM Thread" prio=5 tid=0x000000080121e800 nid=0x61c0870
> runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#0 (ParallelGC)" prio=5 tid=0x0000000801215800
> nid=0x61c0863 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#1 (ParallelGC)" prio=5 tid=0x0000000801216800
> nid=0x61c0864 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#2 (ParallelGC)" prio=5 tid=0x0000000801217000
> nid=0x61c0865 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#3 (ParallelGC)" prio=5 tid=0x0000000801217800
> nid=0x61c0866 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#4 (ParallelGC)" prio=5 tid=0x0000000801218000
> nid=0x61c0867 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#5 (ParallelGC)" prio=5 tid=0x0000000801219000
> nid=0x61c0868 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#6 (ParallelGC)" prio=5 tid=0x0000000801219800
> nid=0x61c0869 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#7 (ParallelGC)" prio=5 tid=0x000000080121a000
> nid=0x61c086a runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#8 (ParallelGC)" prio=5 tid=0x000000080121a800
> nid=0x61c086b runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#9 (ParallelGC)" prio=5 tid=0x000000080121b800
> nid=0x61c086c runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#10 (ParallelGC)" prio=5
> tid=0x000000080121c000 nid=0x61c086d runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#11 (ParallelGC)" prio=5
> tid=0x000000080121c800 nid=0x61c086e runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#12 (ParallelGC)" prio=5
> tid=0x000000080121d000 nid=0x61c086f runnable
> [junit4:junit4]
> [junit4:junit4] "VM Periodic Task Thread" prio=5 tid=0x000000080121f000
> nid=0x61c0877 waiting on condition
> [junit4:junit4]
> [junit4:junit4] JNI global references: 1183
> [junit4:junit4]
> [junit4:junit4] Heap
> [junit4:junit4] PSYoungGen total 212224K, used 110789K
> [0x0000000830220000, 0x000000083d1c0000, 0x0000000840220000)
> [junit4:junit4] eden space 211840K, 52% used
> [0x0000000830220000,0x0000000836e516f0,0x000000083d100000)
> [junit4:junit4] from space 384K, 0% used
> [0x000000083d160000,0x000000083d160000,0x000000083d1c0000)
> [junit4:junit4] to space 384K, 0% used
> [0x000000083d100000,0x000000083d100000,0x000000083d160000)
> [junit4:junit4] PSOldGen total 524288K, used 127127K
> [0x0000000810220000, 0x0000000830220000, 0x0000000830220000)
> [junit4:junit4] object space 524288K, 24% used
> [0x0000000810220000,0x0000000817e45ef0,0x0000000830220000)
> [junit4:junit4] PSPermGen total 83968K, used 83967K
> [0x000000080b020000, 0x0000000810220000, 0x0000000810220000)
> [junit4:junit4] object space 83968K, 99% used
> [0x000000080b020000,0x000000081021fd98,0x0000000810220000)
> [junit4:junit4] <<< JVM J0: EOF ----
> {noformat}
> https://builds.apache.org/job/Lucene-Solr-Clover-4.x/77/consoleFull (today):
> {noformat}
> [junit4:junit4] HEARTBEAT J0 PID(2...@lucene.zones.apache.org):
> 2012-12-16T22:11:23, stalled for 13809s at: QueryEqualityTest (suite)
> [junit4:junit4] HEARTBEAT J0 PID(2...@lucene.zones.apache.org):
> 2012-12-16T22:12:23, stalled for 13869s at: QueryEqualityTest (suite)
> [junit4:junit4] HEARTBEAT J0 PID(2...@lucene.zones.apache.org):
> 2012-12-16T22:13:23, stalled for 13929s at: QueryEqualityTest (suite)
> [junit4:junit4] HEARTBEAT J0 PID(2...@lucene.zones.apache.org):
> 2012-12-16T22:14:23, stalled for 13989s at: QueryEqualityTest (suite)
> [junit4:junit4] JVM J0: stdout was not empty, see:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Clover-4.x/solr/build/solr-core/test/temp/junit4-J0-20121216_175857_638.sysout
> [junit4:junit4] >>> JVM J0: stdout (verbatim) ----
> [junit4:junit4] 2012-12-16 22:14:25
> [junit4:junit4] Full thread dump OpenJDK 64-Bit Server VM (20.0-b12 mixed
> mode):
> [junit4:junit4]
> [junit4:junit4] "TimeLimitedCollector timer thread" daemon prio=5
> tid=0x00000008460dc800 nid=0x61c9fa3 waiting on condition [0x00007ffffd8da000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:267)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-2" daemon prio=5
> tid=0x000000084b749800 nid=0x61c9f96 runnable [0x00007ffffe0e2000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000810dd3f20> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at
> java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-1" daemon prio=5
> tid=0x000000084b749000 nid=0x61c9f95 runnable [0x00007ffffe1e3000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000810dd3f20> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at
> java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "Low Memory Detector" daemon prio=5 tid=0x00000008011c4800
> nid=0x61c9f90 runnable [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread1" daemon prio=5 tid=0x00000008011c3800
> nid=0x61c9f8f waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread0" daemon prio=5 tid=0x00000008011c3000
> nid=0x61c9f8e waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Signal Dispatcher" daemon prio=5 tid=0x00000008011c2000
> nid=0x61c9f8d waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Finalizer" daemon prio=5 tid=0x00000008011c1800
> nid=0x61c9f8c in Object.wait() [0x00007ffffebed000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
> [junit4:junit4] - locked <0x00000008109642d8> (a
> java.lang.ref.ReferenceQueue$Lock)
> [junit4:junit4] at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
> [junit4:junit4] at
> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
> [junit4:junit4]
> [junit4:junit4] "Reference Handler" daemon prio=5 tid=0x00000008011c0800
> nid=0x61c9f8b in Object.wait() [0x00007ffffecee000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.Object.wait(Object.java:502)
> [junit4:junit4] at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> [junit4:junit4] - locked <0x0000000810967160> (a
> java.lang.ref.Reference$Lock)
> [junit4:junit4]
> [junit4:junit4] "main" prio=5 tid=0x00000008011c0000 nid=0x61c9d83 runnable
> [0x00007fffffbfd000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.io.FileInputStream.readBytes(Native Method)
> [junit4:junit4] at
> java.io.FileInputStream.read(FileInputStream.java:236)
> [junit4:junit4] at
> java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> [junit4:junit4] at
> java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> [junit4:junit4] - locked <0x000000081098f1f8> (a
> java.io.BufferedInputStream)
> [junit4:junit4] at
> sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
> [junit4:junit4] at
> sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
> [junit4:junit4] - locked <0x000000081098f1e0> (a
> java.io.InputStreamReader)
> [junit4:junit4] at
> java.io.InputStreamReader.read(InputStreamReader.java:184)
> [junit4:junit4] at java.io.BufferedReader.fill(BufferedReader.java:153)
> [junit4:junit4] at
> java.io.BufferedReader.readLine(BufferedReader.java:316)
> [junit4:junit4] - locked <0x000000081098f1e0> (a
> java.io.InputStreamReader)
> [junit4:junit4] at
> java.io.BufferedReader.readLine(BufferedReader.java:379)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:31)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:13)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.Iterators$5.hasNext(Iterators.java:539)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:150)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:255)
> [junit4:junit4] at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
> [junit4:junit4]
> [junit4:junit4] "VM Thread" prio=5 tid=0x000000080121e800 nid=0x61c9f8a
> runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#0 (ParallelGC)" prio=5 tid=0x0000000801215800
> nid=0x61c9f7d runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#1 (ParallelGC)" prio=5 tid=0x0000000801216800
> nid=0x61c9f7e runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#2 (ParallelGC)" prio=5 tid=0x0000000801217000
> nid=0x61c9f7f runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#3 (ParallelGC)" prio=5 tid=0x0000000801217800
> nid=0x61c9f80 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#4 (ParallelGC)" prio=5 tid=0x0000000801218000
> nid=0x61c9f81 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#5 (ParallelGC)" prio=5 tid=0x0000000801219000
> nid=0x61c9f82 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#6 (ParallelGC)" prio=5 tid=0x0000000801219800
> nid=0x61c9f83 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#7 (ParallelGC)" prio=5 tid=0x000000080121a000
> nid=0x61c9f84 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#8 (ParallelGC)" prio=5 tid=0x000000080121a800
> nid=0x61c9f85 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#9 (ParallelGC)" prio=5 tid=0x000000080121b800
> nid=0x61c9f86 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#10 (ParallelGC)" prio=5
> tid=0x000000080121c000 nid=0x61c9f87 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#11 (ParallelGC)" prio=5
> tid=0x000000080121c800 nid=0x61c9f88 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#12 (ParallelGC)" prio=5
> tid=0x000000080121d000 nid=0x61c9f89 runnable
> [junit4:junit4]
> [junit4:junit4] "VM Periodic Task Thread" prio=5 tid=0x000000080121f000
> nid=0x61c9f91 waiting on condition
> [junit4:junit4]
> [junit4:junit4] JNI global references: 1271
> [junit4:junit4]
> [junit4:junit4] Heap
> [junit4:junit4] PSYoungGen total 229120K, used 170968K
> [0x0000000830220000, 0x000000083eb60000, 0x0000000840220000)
> [junit4:junit4] eden space 228992K, 74% used
> [0x0000000830220000,0x000000083a8fe360,0x000000083e1c0000)
> [junit4:junit4] from space 128K, 75% used
> [0x000000083eb40000,0x000000083eb58000,0x000000083eb60000)
> [junit4:junit4] to space 384K, 0% used
> [0x000000083eaa0000,0x000000083eaa0000,0x000000083eb00000)
> [junit4:junit4] PSOldGen total 524288K, used 179710K
> [0x0000000810220000, 0x0000000830220000, 0x0000000830220000)
> [junit4:junit4] object space 524288K, 34% used
> [0x0000000810220000,0x000000081b19fa98,0x0000000830220000)
> [junit4:junit4] PSPermGen total 83968K, used 83967K
> [0x000000080b020000, 0x0000000810220000, 0x0000000810220000)
> [junit4:junit4] object space 83968K, 99% used
> [0x000000080b020000,0x000000081021fc68,0x0000000810220000)
> [junit4:junit4] <<< JVM J0: EOF ----
> {noformat}
> To me it looks like there is something wrong with the test runner. In other
> cases this totally-hanging threads were caused by permgen-errors, but nothing
> like this in the logs. It looks like the slave is waiting forever to get some
> messages from the master or vice versa.
> You can imitate clover tests on ASF Jenkins by running:
> {noformat}
> ANT_OPTS="-Xmx1536M" ant -Dtests.jettyConnector=Socket jenkins-clover
> {noformat}
> (the ANT_OPTS are just for the generate-clover-task using lots of memory -
> totally unrelated. This is just needed after the tests ran successfully).
> jenkins-clover calls run-clover task which automatically gives more memory to
> the child JVMs, so maybe this is related and needs more memory. You can find
> that in the main common-build.xml file, where you will find a property
> definition if="run.clover" giving more heap and code cache to childs:
> {code:xml}
> <condition property="tests.heapsize" value="768M">
> <isset property="run.clover"/>
> </condition>
> <property name="tests.heapsize" value="512M"/>
>
> <condition property="tests.clover.args"
> value="-XX:ReservedCodeCacheSize=128m">
> <isset property="run.clover"/>
> </condition>
> <property name="tests.clover.args" value=""/>
> {code}
> (maybe tweak that if Solr needs more memory - Lucene clover passes fine).
> I disabled clover tests for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
