Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-main/5553/

1 tests failed.
FAILED:  org.apache.solr.core.TestCoreDiscovery.testTooManyTransientCores

Error Message:
org.apache.solr.common.SolrException: There should only be 3 cores loaded, 
coreLOS and two coreT? cores

Stack Trace:
org.apache.solr.common.SolrException: There should only be 3 cores loaded, 
coreLOS and two coreT? cores
        at 
__randomizedtesting.SeedInfo.seed([B130DE82F9552C41:9BE49E26B453645D]:0)
        at 
app//org.apache.solr.common.util.RetryUtil.retryUntil(RetryUtil.java:84)
        at 
app//org.apache.solr.core.TestCoreDiscovery.testTooManyTransientCores(TestCoreDiscovery.java:336)
        at 
[email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
 Method)
        at 
[email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
[email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
        at 
app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at 
app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
        at 
app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at 
app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
        at 
app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at 
app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at 
app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
        at 
app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
        at 
app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
        at 
app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
        at 
app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at 
app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at 
app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at 
app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at 
app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
        at 
app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at 
app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at 
app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at 
app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at 
app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
        at 
app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
        at [email protected]/java.lang.Thread.run(Thread.java:829)




Build Log:
[...truncated 1958 lines...]
#   2022-12-11 12:43:51
#   Full thread dump OpenJDK 64-Bit Server VM (11.0.16.1+1 mixed mode):
#
#   Threads class SMR info:
#   _java_thread_list=0x00007f87ec002730, length=32, elements={
#   0x00007f88302c1000, 0x00007f88302c5800, 0x00007f88302da800, 
0x00007f88302dc800,
#   0x00007f88302de800, 0x00007f88302e0800, 0x00007f88302e2800, 
0x00007f883036a000,
#   0x00007f88309a8800, 0x00007f8830ba8000, 0x00007f8830bc8000, 
0x00007f8830bc9800,
#   0x00007f8830bcb800, 0x00007f8830bcd000, 0x00007f8830bce800, 
0x00007f8830bd0800,
#   0x00007f8830bd3000, 0x00007f8830bd4800, 0x00007f8830bd6800, 
0x00007f8830bd8000,
#   0x00007f883141a800, 0x00007f8831ab6000, 0x00007f8831aba800, 
0x00007f8831b11000,
#   0x00007f8831b12000, 0x00007f8831b17000, 0x00007f8831b1c000, 
0x00007f8832055800,
#   0x00007f883252a800, 0x00007f883252d000, 0x00007f8830029000, 
0x00007f87ec001800
#   }
#
#   "Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.85ms elapsed=184.52s 
tid=0x00007f88302c1000 nid=0x2831b2 waiting on condition  [0x00007f8818084000]
#      java.lang.Thread.State: RUNNABLE
#       at 
java.lang.ref.Reference.waitForReferencePendingList([email protected]/Native 
Method)
#       at 
java.lang.ref.Reference.processPendingReferences([email protected]/Reference.java:241)
#       at 
java.lang.ref.Reference$ReferenceHandler.run([email protected]/Reference.java:213)
#
#   "Finalizer" #3 daemon prio=8 os_prio=0 cpu=0.35ms elapsed=184.52s 
tid=0x00007f88302c5800 nid=0x2831b3 in Object.wait()  [0x00007f8818043000]
#      java.lang.Thread.State: WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <0x00000000e04013f0> (a java.lang.ref.ReferenceQueue$Lock)
#       at 
java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
#       - waiting to re-lock in wait() <0x00000000e04013f0> (a 
java.lang.ref.ReferenceQueue$Lock)
#       at 
java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:176)
#       at 
java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:170)
#
#   "Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.30ms elapsed=184.51s 
tid=0x00007f88302da800 nid=0x2831b4 runnable  [0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#
#   "Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.08ms elapsed=184.50s 
tid=0x00007f88302dc800 nid=0x2831b5 runnable  [0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#
#   "C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=2774.47ms 
elapsed=184.50s tid=0x00007f88302de800 nid=0x2831b6 waiting on condition  
[0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#      No compile task
#
#   "C1 CompilerThread0" #8 daemon prio=9 os_prio=0 cpu=1144.87ms 
elapsed=184.48s tid=0x00007f88302e0800 nid=0x2831b8 waiting on condition  
[0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#      No compile task
#
#   "Sweeper thread" #9 daemon prio=9 os_prio=0 cpu=4.17ms elapsed=184.48s 
tid=0x00007f88302e2800 nid=0x2831b9 runnable  [0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#
#   "Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=1.52ms elapsed=184.43s 
tid=0x00007f883036a000 nid=0x2831ba in Object.wait()  [0x00007f88134fd000]
#      java.lang.Thread.State: TIMED_WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <no object reference available>
#       at 
java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
#       - waiting to re-lock in wait() <0x00000000e04018c0> (a 
java.lang.ref.ReferenceQueue$Lock)
#       at 
jdk.internal.ref.CleanerImpl.run([email protected]/CleanerImpl.java:148)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#       at 
jdk.internal.misc.InnocuousThread.run([email protected]/InnocuousThread.java:161)
#
#   "Log4j2-TF-3-AsyncLoggerConfig-1" #12 daemon prio=5 os_prio=0 cpu=467.50ms 
elapsed=183.49s tid=0x00007f88309a8800 nid=0x2831c6 in Object.wait()  
[0x00007f88134bc000]
#      java.lang.Thread.State: TIMED_WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <no object reference available>
#       at java.lang.Object.wait([email protected]/Object.java:462)
#       at 
org.apache.logging.log4j.core.async.TimeoutBlockingWaitStrategy.awaitNanos(TimeoutBlockingWaitStrategy.java:130)
#       at 
org.apache.logging.log4j.core.async.TimeoutBlockingWaitStrategy.waitFor(TimeoutBlockingWaitStrategy.java:86)
#       - waiting to re-lock in wait() <0x00000000e04037b8> (a java.lang.Object)
#       at 
com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
#       at 
com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159)
#       at 
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "ShutdownMonitor" #14 daemon prio=5 os_prio=0 cpu=2.17ms elapsed=182.93s 
tid=0x00007f8830ba8000 nid=0x2831cb runnable  [0x00007f881347b000]
#      java.lang.Thread.State: RUNNABLE
#       at java.net.PlainSocketImpl.socketAccept([email protected]/Native 
Method)
#       at 
java.net.AbstractPlainSocketImpl.accept([email protected]/AbstractPlainSocketImpl.java:474)
#       at 
java.net.ServerSocket.implAccept([email protected]/ServerSocket.java:565)
#       at 
java.net.ServerSocket.accept([email protected]/ServerSocket.java:533)
#       at 
org.eclipse.jetty.server.ShutdownMonitor$ShutdownMonitorRunnable.run(ShutdownMonitor.java:344)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-15" #15 prio=5 os_prio=0 cpu=0.53ms elapsed=182.91s 
tid=0x00007f8830bc8000 nid=0x2831cc runnable  [0x00007f87da9c4000]
#      java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000fb72cff8> (a sun.nio.ch.Util$2)
#       - locked <0x00000000fb72cea8> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-16" #16 prio=5 os_prio=0 cpu=0.54ms elapsed=182.90s 
tid=0x00007f8830bc9800 nid=0x2831cd waiting on condition  [0x00007f87da983000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-17" #17 prio=5 os_prio=0 cpu=0.61ms elapsed=182.90s 
tid=0x00007f8830bcb800 nid=0x2831ce waiting on condition  [0x00007f87da123000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-18" #18 prio=5 os_prio=0 cpu=0.41ms elapsed=182.90s 
tid=0x00007f8830bcd000 nid=0x2831cf waiting on condition  [0x00007f87da0e2000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-19-acceptor-0@284a54ec-ServerConnector@70eecdc2{HTTP/1.1, 
(http/1.1, h2c)}{127.0.0.1:7574}" #19 prio=5 os_prio=0 cpu=0.76ms 
elapsed=182.90s tid=0x00007f8830bce800 nid=0x2831d0 runnable  
[0x00007f87da0a1000]
#      java.lang.Thread.State: RUNNABLE
#       at 
sun.nio.ch.ServerSocketChannelImpl.accept0([email protected]/Native Method)
#       at 
sun.nio.ch.ServerSocketChannelImpl.accept([email protected]/ServerSocketChannelImpl.java:533)
#       at 
sun.nio.ch.ServerSocketChannelImpl.accept([email protected]/ServerSocketChannelImpl.java:285)
#       at 
org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
#       at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-20" #20 prio=5 os_prio=0 cpu=0.39ms elapsed=182.90s 
tid=0x00007f8830bd0800 nid=0x2831d1 waiting on condition  [0x00007f87da060000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-21" #21 prio=5 os_prio=0 cpu=0.47ms elapsed=182.90s 
tid=0x00007f8830bd3000 nid=0x2831d2 waiting on condition  [0x00007f87da01f000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-22" #22 prio=5 os_prio=0 cpu=0.16ms elapsed=182.90s 
tid=0x00007f8830bd4800 nid=0x2831d3 runnable  [0x00007f87d9fde000]
#      java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000fb72d350> (a sun.nio.ch.Util$2)
#       - locked <0x00000000fb72d200> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-23" #23 prio=5 os_prio=0 cpu=0.40ms elapsed=182.90s 
tid=0x00007f8830bd6800 nid=0x2831d4 waiting on condition  [0x00007f87d9f9d000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "qtp1215951090-24" #24 prio=5 os_prio=0 cpu=0.24ms elapsed=182.89s 
tid=0x00007f8830bd8000 nid=0x2831d5 waiting on condition  [0x00007f87d9f5c000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "Session-HouseKeeper-51827393-1" #25 prio=5 os_prio=0 cpu=0.50ms 
elapsed=182.63s tid=0x00007f883141a800 nid=0x2831d6 waiting on condition  
[0x00007f87d9d1b000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000f9790460> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
#       at 
java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "httpShardExecutor-5-thread-1" #26 prio=5 os_prio=0 cpu=0.83ms 
elapsed=181.12s tid=0x00007f8831ab6000 nid=0x283262 runnable  
[0x00007f87d8e93000]
#      java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f99e7b50> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f99e7a00> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "httpShardExecutor-5-thread-2" #27 prio=5 os_prio=0 cpu=0.16ms 
elapsed=181.12s tid=0x00007f8831aba800 nid=0x283263 runnable  
[0x00007f87d8e52000]
#      java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f99e8098> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f99e7f48> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "Connection evictor" #28 daemon prio=5 os_prio=0 cpu=1.73ms elapsed=181.04s 
tid=0x00007f8831b11000 nid=0x283264 waiting on condition  [0x00007f87d8c11000]
#      java.lang.Thread.State: TIMED_WAITING (sleeping)
#       at java.lang.Thread.sleep([email protected]/Native Method)
#       at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "Connection evictor" #29 daemon prio=5 os_prio=0 cpu=0.45ms elapsed=181.04s 
tid=0x00007f8831b12000 nid=0x283265 waiting on condition  [0x00007f87d8bd0000]
#      java.lang.Thread.State: TIMED_WAITING (sleeping)
#       at java.lang.Thread.sleep([email protected]/Native Method)
#       at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "h2sc-7-thread-1" #30 prio=5 os_prio=0 cpu=0.14ms elapsed=181.04s 
tid=0x00007f8831b17000 nid=0x283266 runnable  [0x00007f87d8b8f000]
#      java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f97b3b20> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f97b3ac8> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "h2sc-7-thread-2" #31 prio=5 os_prio=0 cpu=0.17ms elapsed=181.04s 
tid=0x00007f8831b1c000 nid=0x283267 runnable  [0x00007f87d8b4e000]
#      java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f97b5450> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f97b5300> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "CloserThread" #32 prio=5 os_prio=0 cpu=0.19ms elapsed=180.83s 
tid=0x00007f8832055800 nid=0x283268 in Object.wait()  [0x00007f87d8b0d000]
#      java.lang.Thread.State: WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <0x00000000f9676228> (a java.lang.Object)
#       at java.lang.Object.wait([email protected]/Object.java:328)
#       at org.apache.solr.core.CloserThread.run(CoreContainer.java:2517)
#       - waiting to re-lock in wait() <0x00000000f9676228> (a java.lang.Object)
#
#   "org.eclipse.jetty.util.RolloverFileOutputStream" #34 daemon prio=5 
os_prio=0 cpu=0.15ms elapsed=180.10s tid=0x00007f883252a800 nid=0x28326c 
waiting on condition  [0x00007f87d848b000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000fb89ea10> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
#       at 
java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
#   "AsyncRequestLogWriter@49122853" #35 prio=5 os_prio=0 cpu=1.61ms 
elapsed=180.10s tid=0x00007f883252d000 nid=0x28326d waiting on condition  
[0x00007f87d844a000]
#      java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100e4a8> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.server.AsyncRequestLogWriter$WriterThread.run(AsyncRequestLogWriter.java:66)
#
#   "DestroyJavaVM" #36 prio=5 os_prio=0 cpu=3366.58ms elapsed=180.09s 
tid=0x00007f8830029000 nid=0x2831a6 waiting on condition  [0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#
#   "Attach Listener" #37 daemon prio=9 os_prio=0 cpu=0.27ms elapsed=0.10s 
tid=0x00007f87ec001800 nid=0x283ada waiting on condition  [0x0000000000000000]
#      java.lang.Thread.State: RUNNABLE
#
#   "VM Thread" os_prio=0 cpu=34.94ms elapsed=184.52s tid=0x00007f88302b9000 
nid=0x2831b1 runnable
#
#   "GC Thread#0" os_prio=0 cpu=52.22ms elapsed=184.57s tid=0x00007f8830041800 
nid=0x2831ac runnable
#
#   "GC Thread#1" os_prio=0 cpu=47.27ms elapsed=183.82s tid=0x00007f8800001000 
nid=0x2831c1 runnable
#
#   "GC Thread#2" os_prio=0 cpu=49.92ms elapsed=183.82s tid=0x00007f8800002800 
nid=0x2831c2 runnable
#
#   "GC Thread#3" os_prio=0 cpu=40.41ms elapsed=183.82s tid=0x00007f8800004000 
nid=0x2831c3 runnable
#
#   "G1 Main Marker" os_prio=0 cpu=1.10ms elapsed=184.56s 
tid=0x00007f883004e800 nid=0x2831ad runnable
#
#   "G1 Conc#0" os_prio=0 cpu=41.92ms elapsed=184.56s tid=0x00007f8830050800 
nid=0x2831ae runnable
#
#   "G1 Refine#0" os_prio=0 cpu=1.97ms elapsed=184.54s tid=0x00007f88301f1000 
nid=0x2831af runnable
#
#   "G1 Young RemSet Sampling" os_prio=0 cpu=43.75ms elapsed=184.54s 
tid=0x00007f88301f2800 nid=0x2831b0 runnable
#   "VM Periodic Task Thread" os_prio=0 cpu=74.02ms elapsed=184.40s 
tid=0x00007f88303b6800 nid=0x2831bb waiting on condition
#
#   JNI global refs: 22, weak refs: 0
#
#   Solr process 2634146 is still running; forcefully killing it now.
#   Killed process 2634146
#   Sending stop command to Solr running on port 8983 ... waiting up to 180 
seconds to allow Jetty process 2634033 to stop gracefully.
#    [|]   [/]   [-]   [\]      
# --
#
# Please find the SOLR_HOME snapshot for failed test #1 at: 
/home/jenkins/jenkins-slave/workspace/Solr/Solr-Check-main/solr/packaging/build/test-output/failure-snapshots/32-1
# Last output:
# Sending stop command to Solr running on port 7574 ... waiting up to 180 
seconds to allow Jetty process 2634146 to stop gracefully.
#  [|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]   [|]   [/]  
 [-]   [\]   [|]   [/]   [-]   [\]   
[|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]   [|]   [/]  
 [-]   [\]   [|]   [/]   [-]   [\]   
[|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]   [|]   [/]  
 [-]   [\]   [|]   [/]   [-]   [\]   
[|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]   [|]   [/]  
 [-]   [\]   [|]   [/]   [-]   [\]   
[|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]   [|]   [/]  
 [-]   [\]   [|]   [/]   [-]   [\]   
[|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]   [|]   [/]  
 [-]   [\]   [|]   [/]   [-]   [\]   
[|]   [/]   [-]   [\]   [|]   [/]   [-]  
 [\]   [|]   [/]   [-]   [\]   [|]   
[/]   [-]   [\]   [|]   [/]   [-]   [\]  
 [|]   [/]   [-]   [\]   [|]   [/]   
[-]   [\]   [|]   [/]   [-]   [\]   [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]   [-]   [\]      Solr process 
2634146 is still running; jstacking it now.
# 2022-12-11 12:43:51
# Full thread dump OpenJDK 64-Bit Server VM (11.0.16.1+1 mixed mode):
#
# Threads class SMR info:
# _java_thread_list=0x00007f87ec002730, length=32, elements={
# 0x00007f88302c1000, 0x00007f88302c5800, 0x00007f88302da800, 
0x00007f88302dc800,
# 0x00007f88302de800, 0x00007f88302e0800, 0x00007f88302e2800, 
0x00007f883036a000,
# 0x00007f88309a8800, 0x00007f8830ba8000, 0x00007f8830bc8000, 
0x00007f8830bc9800,
# 0x00007f8830bcb800, 0x00007f8830bcd000, 0x00007f8830bce800, 
0x00007f8830bd0800,
# 0x00007f8830bd3000, 0x00007f8830bd4800, 0x00007f8830bd6800, 
0x00007f8830bd8000,
# 0x00007f883141a800, 0x00007f8831ab6000, 0x00007f8831aba800, 
0x00007f8831b11000,
# 0x00007f8831b12000, 0x00007f8831b17000, 0x00007f8831b1c000, 
0x00007f8832055800,
# 0x00007f883252a800, 0x00007f883252d000, 0x00007f8830029000, 0x00007f87ec001800
# }
#
# "Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.85ms elapsed=184.52s 
tid=0x00007f88302c1000 nid=0x2831b2 waiting on condition  [0x00007f8818084000]
#    java.lang.Thread.State: RUNNABLE
#       at 
java.lang.ref.Reference.waitForReferencePendingList([email protected]/Native 
Method)
#       at 
java.lang.ref.Reference.processPendingReferences([email protected]/Reference.java:241)
#       at 
java.lang.ref.Reference$ReferenceHandler.run([email protected]/Reference.java:213)
#
# "Finalizer" #3 daemon prio=8 os_prio=0 cpu=0.35ms elapsed=184.52s 
tid=0x00007f88302c5800 nid=0x2831b3 in Object.wait()  [0x00007f8818043000]
#    java.lang.Thread.State: WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <0x00000000e04013f0> (a java.lang.ref.ReferenceQueue$Lock)
#       at 
java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
#       - waiting to re-lock in wait() <0x00000000e04013f0> (a 
java.lang.ref.ReferenceQueue$Lock)
#       at 
java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:176)
#       at 
java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:170)
#
# "Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.30ms elapsed=184.51s 
tid=0x00007f88302da800 nid=0x2831b4 runnable  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#
# "Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.08ms elapsed=184.50s 
tid=0x00007f88302dc800 nid=0x2831b5 runnable  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#
# "C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=2774.47ms elapsed=184.50s 
tid=0x00007f88302de800 nid=0x2831b6 waiting on condition  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#    No compile task
#
# "C1 CompilerThread0" #8 daemon prio=9 os_prio=0 cpu=1144.87ms elapsed=184.48s 
tid=0x00007f88302e0800 nid=0x2831b8 waiting on condition  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#    No compile task
#
# "Sweeper thread" #9 daemon prio=9 os_prio=0 cpu=4.17ms elapsed=184.48s 
tid=0x00007f88302e2800 nid=0x2831b9 runnable  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#
# "Common-Cleaner" #10 daemon prio=8 os_prio=0 cpu=1.52ms elapsed=184.43s 
tid=0x00007f883036a000 nid=0x2831ba in Object.wait()  [0x00007f88134fd000]
#    java.lang.Thread.State: TIMED_WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <no object reference available>
#       at 
java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
#       - waiting to re-lock in wait() <0x00000000e04018c0> (a 
java.lang.ref.ReferenceQueue$Lock)
#       at 
jdk.internal.ref.CleanerImpl.run([email protected]/CleanerImpl.java:148)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#       at 
jdk.internal.misc.InnocuousThread.run([email protected]/InnocuousThread.java:161)
#
# "Log4j2-TF-3-AsyncLoggerConfig-1" #12 daemon prio=5 os_prio=0 cpu=467.50ms 
elapsed=183.49s tid=0x00007f88309a8800 nid=0x2831c6 in Object.wait()  
[0x00007f88134bc000]
#    java.lang.Thread.State: TIMED_WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <no object reference available>
#       at java.lang.Object.wait([email protected]/Object.java:462)
#       at 
org.apache.logging.log4j.core.async.TimeoutBlockingWaitStrategy.awaitNanos(TimeoutBlockingWaitStrategy.java:130)
#       at 
org.apache.logging.log4j.core.async.TimeoutBlockingWaitStrategy.waitFor(TimeoutBlockingWaitStrategy.java:86)
#       - waiting to re-lock in wait() <0x00000000e04037b8> (a java.lang.Object)
#       at 
com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
#       at 
com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159)
#       at 
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "ShutdownMonitor" #14 daemon prio=5 os_prio=0 cpu=2.17ms elapsed=182.93s 
tid=0x00007f8830ba8000 nid=0x2831cb runnable  [0x00007f881347b000]
#    java.lang.Thread.State: RUNNABLE
#       at java.net.PlainSocketImpl.socketAccept([email protected]/Native 
Method)
#       at 
java.net.AbstractPlainSocketImpl.accept([email protected]/AbstractPlainSocketImpl.java:474)
#       at 
java.net.ServerSocket.implAccept([email protected]/ServerSocket.java:565)
#       at 
java.net.ServerSocket.accept([email protected]/ServerSocket.java:533)
#       at 
org.eclipse.jetty.server.ShutdownMonitor$ShutdownMonitorRunnable.run(ShutdownMonitor.java:344)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-15" #15 prio=5 os_prio=0 cpu=0.53ms elapsed=182.91s 
tid=0x00007f8830bc8000 nid=0x2831cc runnable  [0x00007f87da9c4000]
#    java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000fb72cff8> (a sun.nio.ch.Util$2)
#       - locked <0x00000000fb72cea8> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-16" #16 prio=5 os_prio=0 cpu=0.54ms elapsed=182.90s 
tid=0x00007f8830bc9800 nid=0x2831cd waiting on condition  [0x00007f87da983000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-17" #17 prio=5 os_prio=0 cpu=0.61ms elapsed=182.90s 
tid=0x00007f8830bcb800 nid=0x2831ce waiting on condition  [0x00007f87da123000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-18" #18 prio=5 os_prio=0 cpu=0.41ms elapsed=182.90s 
tid=0x00007f8830bcd000 nid=0x2831cf waiting on condition  [0x00007f87da0e2000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-19-acceptor-0@284a54ec-ServerConnector@70eecdc2{HTTP/1.1, 
(http/1.1, h2c)}{127.0.0.1:7574}" #19 prio=5 os_prio=0 cpu=0.76ms 
elapsed=182.90s tid=0x00007f8830bce800 nid=0x2831d0 runnable  
[0x00007f87da0a1000]
#    java.lang.Thread.State: RUNNABLE
#       at 
sun.nio.ch.ServerSocketChannelImpl.accept0([email protected]/Native Method)
#       at 
sun.nio.ch.ServerSocketChannelImpl.accept([email protected]/ServerSocketChannelImpl.java:533)
#       at 
sun.nio.ch.ServerSocketChannelImpl.accept([email protected]/ServerSocketChannelImpl.java:285)
#       at 
org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
#       at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-20" #20 prio=5 os_prio=0 cpu=0.39ms elapsed=182.90s 
tid=0x00007f8830bd0800 nid=0x2831d1 waiting on condition  [0x00007f87da060000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-21" #21 prio=5 os_prio=0 cpu=0.47ms elapsed=182.90s 
tid=0x00007f8830bd3000 nid=0x2831d2 waiting on condition  [0x00007f87da01f000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-22" #22 prio=5 os_prio=0 cpu=0.16ms elapsed=182.90s 
tid=0x00007f8830bd4800 nid=0x2831d3 runnable  [0x00007f87d9fde000]
#    java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000fb72d350> (a sun.nio.ch.Util$2)
#       - locked <0x00000000fb72d200> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:933)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1077)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-23" #23 prio=5 os_prio=0 cpu=0.40ms elapsed=182.90s 
tid=0x00007f8830bd6800 nid=0x2831d4 waiting on condition  [0x00007f87d9f9d000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "qtp1215951090-24" #24 prio=5 os_prio=0 cpu=0.24ms elapsed=182.89s 
tid=0x00007f8830bd8000 nid=0x2831d5 waiting on condition  [0x00007f87d9f5c000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100ca00> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1014)
#       at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1064)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "Session-HouseKeeper-51827393-1" #25 prio=5 os_prio=0 cpu=0.50ms 
elapsed=182.63s tid=0x00007f883141a800 nid=0x2831d6 waiting on condition  
[0x00007f87d9d1b000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000f9790460> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
#       at 
java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "httpShardExecutor-5-thread-1" #26 prio=5 os_prio=0 cpu=0.83ms 
elapsed=181.12s tid=0x00007f8831ab6000 nid=0x283262 runnable  
[0x00007f87d8e93000]
#    java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f99e7b50> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f99e7a00> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "httpShardExecutor-5-thread-2" #27 prio=5 os_prio=0 cpu=0.16ms 
elapsed=181.12s tid=0x00007f8831aba800 nid=0x283263 runnable  
[0x00007f87d8e52000]
#    java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f99e8098> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f99e7f48> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "Connection evictor" #28 daemon prio=5 os_prio=0 cpu=1.73ms elapsed=181.04s 
tid=0x00007f8831b11000 nid=0x283264 waiting on condition  [0x00007f87d8c11000]
#    java.lang.Thread.State: TIMED_WAITING (sleeping)
#       at java.lang.Thread.sleep([email protected]/Native Method)
#       at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "Connection evictor" #29 daemon prio=5 os_prio=0 cpu=0.45ms elapsed=181.04s 
tid=0x00007f8831b12000 nid=0x283265 waiting on condition  [0x00007f87d8bd0000]
#    java.lang.Thread.State: TIMED_WAITING (sleeping)
#       at java.lang.Thread.sleep([email protected]/Native Method)
#       at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "h2sc-7-thread-1" #30 prio=5 os_prio=0 cpu=0.14ms elapsed=181.04s 
tid=0x00007f8831b17000 nid=0x283266 runnable  [0x00007f87d8b8f000]
#    java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f97b3b20> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f97b3ac8> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "h2sc-7-thread-2" #31 prio=5 os_prio=0 cpu=0.17ms elapsed=181.04s 
tid=0x00007f8831b1c000 nid=0x283267 runnable  [0x00007f87d8b4e000]
#    java.lang.Thread.State: RUNNABLE
#       at sun.nio.ch.EPoll.wait([email protected]/Native Method)
#       at 
sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
#       at 
sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
#       - locked <0x00000000f97b5450> (a sun.nio.ch.Util$2)
#       - locked <0x00000000f97b5300> (a sun.nio.ch.EPollSelectorImpl)
#       at 
sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
#       at 
org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
#       at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
#       at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248)
#       at 
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:199)
#       at 
org.eclipse.jetty.io.ManagedSelector$$Lambda$401/0x00000001004a1c40.run(Unknown 
Source)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
#       at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$402/0x00000001004a1440.run(Unknown
 Source)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "CloserThread" #32 prio=5 os_prio=0 cpu=0.19ms elapsed=180.83s 
tid=0x00007f8832055800 nid=0x283268 in Object.wait()  [0x00007f87d8b0d000]
#    java.lang.Thread.State: WAITING (on object monitor)
#       at java.lang.Object.wait([email protected]/Native Method)
#       - waiting on <0x00000000f9676228> (a java.lang.Object)
#       at java.lang.Object.wait([email protected]/Object.java:328)
#       at org.apache.solr.core.CloserThread.run(CoreContainer.java:2517)
#       - waiting to re-lock in wait() <0x00000000f9676228> (a java.lang.Object)
#
# "org.eclipse.jetty.util.RolloverFileOutputStream" #34 daemon prio=5 os_prio=0 
cpu=0.15ms elapsed=180.10s tid=0x00007f883252a800 nid=0x28326c waiting on 
condition  [0x00007f87d848b000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000fb89ea10> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
#       at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
#       at 
java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
#       at 
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
#       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
#       at java.lang.Thread.run([email protected]/Thread.java:829)
#
# "AsyncRequestLogWriter@49122853" #35 prio=5 os_prio=0 cpu=1.61ms 
elapsed=180.10s tid=0x00007f883252d000 nid=0x28326d waiting on condition  
[0x00007f87d844a000]
#    java.lang.Thread.State: TIMED_WAITING (parking)
#       at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
#       - parking to wait for  <0x00000000e100e4a8> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
#       at 
java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
#       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
#       at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
#       at 
org.eclipse.jetty.server.AsyncRequestLogWriter$WriterThread.run(AsyncRequestLogWriter.java:66)
#
# "DestroyJavaVM" #36 prio=5 os_prio=0 cpu=3366.58ms elapsed=180.09s 
tid=0x00007f8830029000 nid=0x2831a6 waiting on condition  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#
# "Attach Listener" #37 daemon prio=9 os_prio=0 cpu=0.27ms elapsed=0.10s 
tid=0x00007f87ec001800 nid=0x283ada waiting on condition  [0x0000000000000000]
#    java.lang.Thread.State: RUNNABLE
#
# "VM Thread" os_prio=0 cpu=34.94ms elapsed=184.52s tid=0x00007f88302b9000 
nid=0x2831b1 runnable
#
# "GC Thread#0" os_prio=0 cpu=52.22ms elapsed=184.57s tid=0x00007f8830041800 
nid=0x2831ac runnable
#
# "GC Thread#1" os_prio=0 cpu=47.27ms elapsed=183.82s tid=0x00007f8800001000 
nid=0x2831c1 runnable
#
# "GC Thread#2" os_prio=0 cpu=49.92ms elapsed=183.82s tid=0x00007f8800002800 
nid=0x2831c2 runnable
#
# "GC Thread#3" os_prio=0 cpu=40.41ms elapsed=183.82s tid=0x00007f8800004000 
nid=0x2831c3 runnable
#
# "G1 Main Marker" os_prio=0 cpu=1.10ms elapsed=184.56s tid=0x00007f883004e800 
nid=0x2831ad runnable
#
# "G1 Conc#0" os_prio=0 cpu=41.92ms elapsed=184.56s tid=0x00007f8830050800 
nid=0x2831ae runnable
#
# "G1 Refine#0" os_prio=0 cpu=1.97ms elapsed=184.54s tid=0x00007f88301f1000 
nid=0x2831af runnable
#
# "G1 Young RemSet Sampling" os_prio=0 cpu=43.75ms elapsed=184.54s 
tid=0x00007f88301f2800 nid=0x2831b0 runnable
# "VM Periodic Task Thread" os_prio=0 cpu=74.02ms elapsed=184.40s 
tid=0x00007f88303b6800 nid=0x2831bb waiting on condition
#
# JNI global refs: 22, weak refs: 0
#
# Solr process 2634146 is still running; forcefully killing it now.
# Killed process 2634146
# Sending stop command to Solr running on port 8983 ... waiting up to 180 
seconds to allow Jetty process 2634033 to stop gracefully.
#  [|]   [/]   [-]   [\]      

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.core.TestCoreDiscovery.testTooManyTransientCores 
(:solr:core)
    Test output: 
/home/jenkins/jenkins-slave/workspace/Solr/Solr-Check-main/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.core.TestCoreDiscovery.txt
    Reproduce with: gradlew :solr:core:test --tests 
"org.apache.solr.core.TestCoreDiscovery.testTooManyTransientCores" 
-Ptests.jvms=4 -Ptests.haltonfailure=false 
"-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC 
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" 
-Ptests.seed=B130DE82F9552C41 -Ptests.multiplier=2 -Ptests.badapples=false 
-Ptests.file.encoding=ISO-8859-1


FAILURE: Build failed with an exception.

* Where:
Build file 
'/home/jenkins/jenkins-slave/workspace/Solr/Solr-Check-main/solr/packaging/build.gradle'
 line: 226

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command 
> '/home/jenkins/jenkins-slave/workspace/Solr/Solr-Check-main/.gradle/node/packaging/node_modules/bats/bin/bats''
>  finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with 
Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings 
and determine if they come from your own scripts or plugins.

See 
https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 55m 1s
652 actionable tasks: 652 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to