Ah, I see it fail intermittently, but can reproduce. Thanks, Dawid,
I'll see if I can find why. Hopefully a bad assumption in the test,
we'll see!

On Wed, Feb 3, 2021 at 3:14 AM Dawid Weiss <[email protected]> wrote:
>
> This reproduces for me. (Mike, help! :).
>
> gradlew test --tests TestKnnGraph.testMergeProducesSameGraph
> -Dtests.seed=CE4157E04A67AF88 -Dtests.multiplier=2 -Dtests.slow=true
> -Dtests.locale=ln-AO -Dtests.timezone=Indian/Chagos
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>
> On Wed, Feb 3, 2021 at 1:32 AM Apache Jenkins Server
> <[email protected]> wrote:
> >
> > Build: 
> > https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Check-master/1518/
> >
> > 7 tests failed.
> > FAILED:  org.apache.lucene.index.TestKnnGraph.testMergeProducesSameGraph
> >
> > Error Message:
> > difference at ord=8: array lengths differed, expected.length=10 
> > actual.length=9; arrays first differed at element [7]; expected:<82> but 
> > was:<89>
> >
> > Stack Trace:
> > difference at ord=8: array lengths differed, expected.length=10 
> > actual.length=9; arrays first differed at element [7]; expected:<82> but 
> > was:<89>
> >         at 
> > __randomizedtesting.SeedInfo.seed([CE4157E04A67AF88:3824B749E6BCA0B3]:0)
> >         at 
> > org.junit.internal.ComparisonCriteria.arrayEquals(ComparisonCriteria.java:78)
> >         at 
> > org.junit.internal.ComparisonCriteria.arrayEquals(ComparisonCriteria.java:28)
> >         at org.junit.Assert.internalArrayEquals(Assert.java:534)
> >         at org.junit.Assert.assertArrayEquals(Assert.java:418)
> >         at 
> > org.apache.lucene.index.TestKnnGraph.assertGraphEquals(TestKnnGraph.java:152)
> >         at 
> > org.apache.lucene.index.TestKnnGraph.testMergeProducesSameGraph(TestKnnGraph.java:146)
> >         at 
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> > Method)
> >         at 
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >         at 
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> >         at 
> > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> >         at 
> > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >         at 
> > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> >         at 
> > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >         at 
> > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> >         at 
> > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:51)
> >         at 
> > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >         at 
> > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >         at 
> > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
> >         at java.base/java.lang.Thread.run(Thread.java:834)
> > Caused by: java.lang.AssertionError: expected:<82> but was:<89>
> >         at org.junit.Assert.fail(Assert.java:89)
> >         at org.junit.Assert.failNotEquals(Assert.java:835)
> >         at org.junit.Assert.assertEquals(Assert.java:120)
> >         at org.junit.Assert.assertEquals(Assert.java:146)
> >         at 
> > org.junit.internal.ExactComparisonCriteria.assertElementsEqual(ExactComparisonCriteria.java:8)
> >         at 
> > org.junit.internal.ComparisonCriteria.arrayEquals(ComparisonCriteria.java:76)
> >         ... 43 more
> >
> >
> > FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test
> >
> > Error Message:
> > org.apache.solr.cloud.ZkController$NotInClusterStateException: Number of 
> > replicas in the state does not match what we set:1 vs 3
> >
> > Stack Trace:
> > org.apache.solr.cloud.ZkController$NotInClusterStateException: Number of 
> > replicas in the state does not match what we set:1 vs 3
> >         at 
> > __randomizedtesting.SeedInfo.seed([CE4157E04A67AF88:4615683AE49BC270]:0)
> >         at 
> > org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForActiveReplicaCount(AbstractFullDistribZkTestBase.java:624)
> >         at 
> > org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:563)
> >         at 
> > org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:360)
> >         at 
> > org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1085)
> >         at 
> > org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1058)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >         at 
> > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> >         at 
> > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >         at 
> > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> >         at 
> > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >         at 
> > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> >         at 
> > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >         at 
> > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:51)
> >         at 
> > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >         at 
> > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >         at 
> > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >         at 
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> >         at 
> > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
> >         at java.base/java.lang.Thread.run(Thread.java:834)
> >
> >
> > FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.classMethod
> >
> > Error Message:
> > com.carrotsearch.randomizedtesting.ThreadLeakError: 42 threads leaked from 
> > SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:
> >    1) Thread[id=5317, name=qtp6697611-5317, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    2) Thread[id=5318, name=qtp6697611-5318, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    3) Thread[id=5405, name=SolrRrdBackendFactory-3723-thread-2, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    4) Thread[id=5384, name=SolrRrdBackendFactory-3755-thread-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    5) Thread[id=5308, name=Session-HouseKeeper-4c13bc80-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    6) Thread[id=5351, name=SolrRrdBackendFactory-3723-thread-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    7) Thread[id=5307, name=qtp276720161-5307, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    8) Thread[id=5366, name=CloserThread, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/java.lang.Object.wait(Native Method)
> >         at [email protected]/java.lang.Object.wait(Object.java:328)
> >         at 
> > app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2246)
> >    9) Thread[id=5331, name=qtp1877464605-5331, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   10) Thread[id=5306, name=qtp276720161-5306, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   11) Thread[id=5352, name=MetricsHistoryHandler-3724-thread-1, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   12) Thread[id=5313, name=qtp6697611-5313, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   13) Thread[id=5315, 
> > name=qtp6697611-5315-acceptor-0@cb72a22-ServerConnector@459a65f{HTTP/1.1, 
> > (http/1.1, h2c)}{127.0.0.1:44053}, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
> >         at 
> > app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
> >         at 
> > app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   14) Thread[id=5303, name=qtp276720161-5303, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   15) Thread[id=5314, name=qtp6697611-5314, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   16) Thread[id=5316, name=qtp6697611-5316, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   17) Thread[id=5333, name=qtp1877464605-5333, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   18) Thread[id=5305, name=qtp276720161-5305, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   19) Thread[id=5403, name=Connector-Scheduler-1441ecf6-1, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   20) Thread[id=5332, name=qtp1877464605-5332, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   21) Thread[id=5395, name=Connector-Scheduler-459a65f-1, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   22) Thread[id=5302, 
> > name=qtp276720161-5302-acceptor-0@79faa8c9-ServerConnector@1441ecf6{HTTP/1.1,
> >  (http/1.1, h2c)}{127.0.0.1:35719}, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
> >         at 
> > app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
> >         at 
> > app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   23) Thread[id=5376, name=CloserThread, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/java.lang.Object.wait(Native Method)
> >         at [email protected]/java.lang.Object.wait(Object.java:328)
> >         at 
> > app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2246)
> >   24) Thread[id=5326, name=qtp1877464605-5326, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   25) Thread[id=5330, name=qtp1877464605-5330, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   26) Thread[id=5375, name=MetricsHistoryHandler-3746-thread-1, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   27) Thread[id=5386, name=CloserThread, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/java.lang.Object.wait(Native Method)
> >         at [email protected]/java.lang.Object.wait(Object.java:328)
> >         at 
> > app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2246)
> >   28) Thread[id=5321, name=Session-HouseKeeper-77cf65f8-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   29) Thread[id=5301, name=qtp276720161-5301, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   30) Thread[id=5304, name=qtp276720161-5304, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   31) Thread[id=5327, name=qtp1877464605-5327, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   32) Thread[id=5319, name=qtp6697611-5319, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   33) Thread[id=5406, name=SolrRrdBackendFactory-3745-thread-2, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   34) Thread[id=5328, 
> > name=qtp1877464605-5328-acceptor-0@73e4b35a-ServerConnector@6bd85f39{HTTP/1.1,
> >  (http/1.1, h2c)}{127.0.0.1:34101}, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
> >         at 
> > app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
> >         at 
> > app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   35) Thread[id=5320, name=qtp6697611-5320, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   36) Thread[id=5407, name=SolrRrdBackendFactory-3755-thread-2, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   37) Thread[id=5374, name=SolrRrdBackendFactory-3745-thread-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   38) Thread[id=5329, name=qtp1877464605-5329, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   39) Thread[id=5404, name=Connector-Scheduler-6bd85f39-1, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   40) Thread[id=5300, name=qtp276720161-5300, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   41) Thread[id=5334, name=Session-HouseKeeper-75e4eea2-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   42) Thread[id=5385, name=MetricsHistoryHandler-3756-thread-1, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >
> > Stack Trace:
> > com.carrotsearch.randomizedtesting.ThreadLeakError: 42 threads leaked from 
> > SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:
> >    1) Thread[id=5317, name=qtp6697611-5317, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    2) Thread[id=5318, name=qtp6697611-5318, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    3) Thread[id=5405, name=SolrRrdBackendFactory-3723-thread-2, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    4) Thread[id=5384, name=SolrRrdBackendFactory-3755-thread-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    5) Thread[id=5308, name=Session-HouseKeeper-4c13bc80-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    6) Thread[id=5351, name=SolrRrdBackendFactory-3723-thread-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    7) Thread[id=5307, name=qtp276720161-5307, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >    8) Thread[id=5366, name=CloserThread, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/java.lang.Object.wait(Native Method)
> >         at [email protected]/java.lang.Object.wait(Object.java:328)
> >         at 
> > app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2246)
> >    9) Thread[id=5331, name=qtp1877464605-5331, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   10) Thread[id=5306, name=qtp276720161-5306, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   11) Thread[id=5352, name=MetricsHistoryHandler-3724-thread-1, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   12) Thread[id=5313, name=qtp6697611-5313, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   13) Thread[id=5315, 
> > name=qtp6697611-5315-acceptor-0@cb72a22-ServerConnector@459a65f{HTTP/1.1, 
> > (http/1.1, h2c)}{127.0.0.1:44053}, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
> >         at 
> > app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
> >         at 
> > app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   14) Thread[id=5303, name=qtp276720161-5303, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   15) Thread[id=5314, name=qtp6697611-5314, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   16) Thread[id=5316, name=qtp6697611-5316, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   17) Thread[id=5333, name=qtp1877464605-5333, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   18) Thread[id=5305, name=qtp276720161-5305, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   19) Thread[id=5403, name=Connector-Scheduler-1441ecf6-1, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   20) Thread[id=5332, name=qtp1877464605-5332, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   21) Thread[id=5395, name=Connector-Scheduler-459a65f-1, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   22) Thread[id=5302, 
> > name=qtp276720161-5302-acceptor-0@79faa8c9-ServerConnector@1441ecf6{HTTP/1.1,
> >  (http/1.1, h2c)}{127.0.0.1:35719}, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
> >         at 
> > app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
> >         at 
> > app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   23) Thread[id=5376, name=CloserThread, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/java.lang.Object.wait(Native Method)
> >         at [email protected]/java.lang.Object.wait(Object.java:328)
> >         at 
> > app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2246)
> >   24) Thread[id=5326, name=qtp1877464605-5326, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   25) Thread[id=5330, name=qtp1877464605-5330, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   26) Thread[id=5375, name=MetricsHistoryHandler-3746-thread-1, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   27) Thread[id=5386, name=CloserThread, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/java.lang.Object.wait(Native Method)
> >         at [email protected]/java.lang.Object.wait(Object.java:328)
> >         at 
> > app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2246)
> >   28) Thread[id=5321, name=Session-HouseKeeper-77cf65f8-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   29) Thread[id=5301, name=qtp276720161-5301, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   30) Thread[id=5304, name=qtp276720161-5304, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   31) Thread[id=5327, name=qtp1877464605-5327, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> >         at 
> > app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
> >         at 
> > app//org.eclipse.jetty.io.ManagedSelector$$Lambda$246/0x0000000100515840.run(Unknown
> >  Source)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   32) Thread[id=5319, name=qtp6697611-5319, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   33) Thread[id=5406, name=SolrRrdBackendFactory-3745-thread-2, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   34) Thread[id=5328, 
> > name=qtp1877464605-5328-acceptor-0@73e4b35a-ServerConnector@6bd85f39{HTTP/1.1,
> >  (http/1.1, h2c)}{127.0.0.1:34101}, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
> >         at 
> > [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
> >         at 
> > app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
> >         at 
> > app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   35) Thread[id=5320, name=qtp6697611-5320, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   36) Thread[id=5407, name=SolrRrdBackendFactory-3755-thread-2, 
> > state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   37) Thread[id=5374, name=SolrRrdBackendFactory-3745-thread-1, 
> > state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   38) Thread[id=5329, name=qtp1877464605-5329, state=TIMED_WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
> >         at 
> > app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:842)
> >         at 
> > app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:892)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   39) Thread[id=5404, name=Connector-Scheduler-6bd85f39-1, state=WAITING, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
> >         at 
> > [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
> >         at 
> > [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
> >         at 
> > [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
> >         at 
> > [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >         at [email protected]/java.lang.Thread.run(Thread.java:834)
> >   40) Thread[id=5300, name=qtp276720161-5300, state=RUNNABLE, 
> > group=TGRP-ChaosMonkeyNothingIsSafeTest]
> >         at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
> >         at 
> > [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
> >         at 
> > [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
> >         at [email protected]/sun.nio.ch.SelectorImpl.select(SelectorI
> >
> > [...truncated too long message...]
> >
> > ePacket(SSLSocketInputRecord.java:70)
> >         at 
> > java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1308)
> >         at 
> > java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:949)
> >         at 
> > org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
> >         at 
> > org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
> >         at 
> > org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
> >         at 
> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
> >         at 
> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> >         at 
> > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> >         at 
> > org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> >         at 
> > org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
> >         at 
> > org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> >         at 
> > org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> >         at 
> > org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> >         at 
> > org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
> >         at 
> > org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
> >         at 
> > org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> >         at 
> > org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> >         at 
> > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
> >         at 
> > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> >         at 
> > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:582)
> >         ... 54 more
> >
> >
> >
> >
> > Build Log:
> > [...truncated 31597 lines...]
> > ERROR: The following test(s) have failed:
> >   - 
> > org.apache.solr.client.solrj.impl.CloudHttp2SolrClientTest.testRetryUpdatesWhenClusterStateIsStale
> >  (:solr:solrj)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/solr/solrj/build/test-results/test/outputs/OUTPUT-org.apache.solr.client.solrj.impl.CloudHttp2SolrClientTest.txt
> >     Reproduce with: gradlew :solr:solrj:test --tests 
> > "org.apache.solr.client.solrj.impl.CloudHttp2SolrClientTest.testRetryUpdatesWhenClusterStateIsStale"
> >  -Ptests.jvms=4 -Ptests.haltonfailure=false 
> > -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=CE4157E04A67AF88 
> > -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
> >
> >   - org.apache.solr.cloud.CollectionsAPISolrJTest.testSplitShard 
> > (:solr:core)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.CollectionsAPISolrJTest.txt
> >     Reproduce with: gradlew :solr:core:test --tests 
> > "org.apache.solr.cloud.CollectionsAPISolrJTest.testSplitShard" 
> > -Ptests.jvms=4 -Ptests.haltonfailure=false 
> > -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=CE4157E04A67AF88 
> > -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
> >
> >   - org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test (:solr:core)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.txt
> >     Reproduce with: gradlew :solr:core:test --tests 
> > "org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test" -Ptests.jvms=4 
> > -Ptests.haltonfailure=false -Ptests.jvmargs=-XX:TieredStopAtLevel=1 
> > -Ptests.seed=CE4157E04A67AF88 -Ptests.multiplier=2 -Ptests.badapples=false 
> > -Ptests.file.encoding=UTF-8
> >
> >   - org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.classMethod 
> > (:solr:core)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.txt
> >     Reproduce with: gradlew :solr:core:test --tests 
> > "org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.classMethod" 
> > -Ptests.jvms=4 -Ptests.haltonfailure=false 
> > -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=CE4157E04A67AF88 
> > -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
> >
> >   - org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.classMethod 
> > (:solr:core)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.txt
> >     Reproduce with: gradlew :solr:core:test --tests 
> > "org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.classMethod" 
> > -Ptests.jvms=4 -Ptests.haltonfailure=false 
> > -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=CE4157E04A67AF88 
> > -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
> >
> >   - org.apache.solr.cloud.LeaderElectionContextKeyTest.classMethod 
> > (:solr:core)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.LeaderElectionContextKeyTest.txt
> >     Reproduce with: gradlew :solr:core:test --tests 
> > "org.apache.solr.cloud.LeaderElectionContextKeyTest.classMethod" 
> > -Ptests.jvms=4 -Ptests.haltonfailure=false 
> > -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=CE4157E04A67AF88 
> > -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
> >
> >   - org.apache.lucene.index.TestKnnGraph.testMergeProducesSameGraph 
> > (:lucene:core)
> >     Test output: 
> > /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-Check-master/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.TestKnnGraph.txt
> >     Reproduce with: gradlew :lucene:core:test --tests 
> > "org.apache.lucene.index.TestKnnGraph.testMergeProducesSameGraph" 
> > -Ptests.jvms=4 -Ptests.haltonfailure=false 
> > -Ptests.jvmargs=-XX:TieredStopAtLevel=1 -Ptests.seed=CE4157E04A67AF88 
> > -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.file.encoding=UTF-8
> >
> >
> > Deprecated Gradle features were used in this build, making it incompatible 
> > with Gradle 7.0.
> > Use '--warning-mode all' to show the individual deprecation warnings.
> > See 
> > https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
> >
> > BUILD SUCCESSFUL in 1h 12m 11s
> > 930 actionable tasks: 930 executed
> > Build step 'Invoke Gradle script' changed build result to SUCCESS
> > Archiving artifacts
> > java.lang.InterruptedException: no matches found within 10000
> >         at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2826)
> >         at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2705)
> >         at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2686)
> >         at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3073)
> > Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
> >                 at 
> > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
> >                 at 
> > hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
> >                 at hudson.remoting.Channel.call(Channel.java:998)
> >                 at hudson.FilePath.act(FilePath.java:1069)
> >                 at hudson.FilePath.act(FilePath.java:1058)
> >                 at hudson.FilePath.validateAntFileMask(FilePath.java:2684)
> >                 at 
> > hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:265)
> >                 at 
> > hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
> >                 at 
> > hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
> >                 at 
> > hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
> >                 at 
> > hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
> >                 at hudson.model.Build$BuildExecution.post2(Build.java:186)
> >                 at 
> > hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
> >                 at hudson.model.Run.execute(Run.java:1905)
> >                 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >                 at 
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> >                 at hudson.model.Executor.run(Executor.java:428)
> > Caused: hudson.FilePath$TunneledInterruptedException
> >         at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3075)
> >         at hudson.remoting.UserRequest.perform(UserRequest.java:211)
> >         at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> >         at hudson.remoting.Request$2.run(Request.java:369)
> >         at 
> > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >         at 
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >         at 
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >         at java.lang.Thread.run(Thread.java:748)
> > Caused: java.lang.InterruptedException: java.lang.InterruptedException: no 
> > matches found within 10000
> >         at hudson.FilePath.act(FilePath.java:1071)
> >         at hudson.FilePath.act(FilePath.java:1058)
> >         at hudson.FilePath.validateAntFileMask(FilePath.java:2684)
> >         at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:265)
> >         at 
> > hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
> >         at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
> >         at 
> > hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
> >         at 
> > hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
> >         at hudson.model.Build$BuildExecution.post2(Build.java:186)
> >         at 
> > hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
> >         at hudson.model.Run.execute(Run.java:1905)
> >         at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> >         at 
> > hudson.model.ResourceController.execute(ResourceController.java:97)
> >         at hudson.model.Executor.run(Executor.java:428)
> > No artifacts found that match the file pattern 
> > "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
> > Recording test results
> > Build step 'Publish JUnit test result report' changed build result to 
> > UNSTABLE
> > Email was triggered for: Unstable (Test Failures)
> > Sending email for trigger: Unstable (Test Failures)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to