Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3223/
2 tests failed. FAILED: org.apache.solr.cloud.OverseerTest.testOverseerFailure Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([BB0261892F804B90]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.OverseerTest Error Message: Suite timeout exceeded (>= 7200000 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). at __randomizedtesting.SeedInfo.seed([BB0261892F804B90]:0) Build Log: [...truncated 15766 lines...] [junit4] Suite: org.apache.solr.cloud.OverseerTest [junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.OverseerTest_BB0261892F804B90-001/init-core-data-001 [junit4] 2> 3102160 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom [junit4] 2> 3102160 WARN (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3 [junit4] 2> 3102160 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false [junit4] 2> 3102178 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN) [junit4] 2> 3102179 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 3102208 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 3102208 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 3102306 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:42498 [junit4] 2> 3102306 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:42498 [junit4] 2> 3102306 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 42498 [junit4] 2> 3102323 INFO (zkConnectionManagerCallback-9941-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3102380 INFO (zkConnectionManagerCallback-9943-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3102392 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore [junit4] 2> 3102392 INFO (SUITE-OverseerTest-seed#[BB0261892F804B90]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end [junit4] 2> 3102408 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset [junit4] 2> 3103127 INFO (zkConnectionManagerCallback-9947-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3103227 INFO (zkCallback-9946-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3103292 INFO (zkCallback-9942-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3103320 INFO (zkConnectionManagerCallback-9954-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3103324 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3103325 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3103327 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3103328 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@b8a3b85[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3103342 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3103343 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3103375 INFO (OverseerStateUpdate-74394069969469441-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3103389 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3103659 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3103685 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469442-node1_core1-n_0000000000 [junit4] 2> 3103694 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3103695 INFO (OverseerAutoScalingTriggerThread-74394069969469441-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3103696 INFO (OverseerStateUpdate-74394069969469441-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3103731 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3103732 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3103733 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=74394069969469441-127.0.0.1:42498_solr-n_0000000000) am exiting, but I'm still the leader [junit4] 2> 3103733 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3103745 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3103746 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000001) starting [junit4] 2> 3103749 INFO (OverseerStateUpdate-74394069969469441-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3103749 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3103765 INFO (zkCallback-9946-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3103782 INFO (zkCallback-9953-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3103786 INFO (zkCallback-9942-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3103841 INFO (closeThreadPool-9959-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469441-127.0.0.1:42498_solr-n_0000000001) closing [junit4] 2> 3103841 INFO (OverseerStateUpdate-74394069969469441-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3103841 WARN (ScheduledTrigger-9242-thread-1) [ ] o.a.s.c.a.TriggerBase Exception getting trigger state '/autoscaling/triggerState/.scheduled_maintenance' [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/triggerState/.scheduled_maintenance [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/triggerState/.scheduled_maintenance [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:59) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerBase.restoreState(TriggerBase.java:266) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:620) ~[java/:?] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3103841 WARN (ScheduledTrigger-9242-thread-1) [ ] o.a.s.c.a.TriggerBase Exception updating trigger state '/autoscaling/triggerState/.scheduled_maintenance' [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/triggerState/.scheduled_maintenance [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/triggerState/.scheduled_maintenance [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:59) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerBase.saveState(TriggerBase.java:246) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:642) ~[java/:?] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3103875 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=74394069969469441-127.0.0.1:42498_solr-n_0000000001) am exiting, but I'm still the leader [junit4] 2> 3103877 WARN (OverseerAutoScalingTriggerThread-74394069969469441-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread Error removing old markers [junit4] 2> => org.apache.solr.common.AlreadyClosedException [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74) [junit4] 2> org.apache.solr.common.AlreadyClosedException: null [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:262) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3104047 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset [junit4] 2> 3104081 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testExternalClusterStateChangeBehavior [junit4] 2> 3104370 INFO (zkConnectionManagerCallback-9963-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3104407 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3104407 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@32c8223b[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3104409 INFO (zkConnectionManagerCallback-9970-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3104434 INFO (zkConnectionManagerCallback-9975-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3104436 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3104440 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3104441 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469444-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3104485 INFO (OverseerStateUpdate-74394069969469444-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3104486 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3104763 INFO (OverseerStateUpdate-74394069969469444-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"addreplica", [junit4] 2> "collection":"test", [junit4] 2> "shard":"x", [junit4] 2> "base_url":"http://127.0.0.1/solr", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "node_name":"node1", [junit4] 2> "core":"core1", [junit4] 2> "state":"down"} [junit4] 2> 3104864 INFO (zkCallback-9942-thread-5) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0]) [junit4] 2> 3104865 ERROR (OverseerStateUpdate-74394069969469444-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception in Overseer main queue loop [junit4] 2> => org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /clusterstate.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) [junit4] 2> org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /clusterstate.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1336) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$setData$6(SolrZkClient.java:370) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.setData(SolrZkClient.java:370) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:220) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:259) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3104929 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1084d0331ed0004, likely client has closed socket [junit4] 2> 3104950 INFO (closeThreadPool-9977-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469444-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3104950 INFO (OverseerStateUpdate-74394069969469444-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3104950 INFO (OverseerAutoScalingTriggerThread-74394069969469444-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3104961 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3105021 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testExternalClusterStateChangeBehavior [junit4] 2> 3105067 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRemovalOfLastReplica [junit4] 2> 3105183 INFO (zkConnectionManagerCallback-9981-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3105299 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=1024039526 [junit4] 2> 3105300 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3105301 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@43a43cd[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3105309 INFO (zkConnectionManagerCallback-9988-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3105315 INFO (zkConnectionManagerCallback-9993-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3105330 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3105331 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3105332 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469447-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3105347 WARN (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3105348 INFO (OverseerStateUpdate-74394069969469447-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3106034 INFO (closeThreadPool-9995-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469447-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3106035 INFO (OverseerStateUpdate-74394069969469447-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3106035 INFO (OverseerAutoScalingTriggerThread-74394069969469447-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3106037 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3106131 INFO (TEST-OverseerTest.testRemovalOfLastReplica-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testRemovalOfLastReplica [junit4] 2> 3106149 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testShardAssignment [junit4] 2> 3106338 INFO (zkConnectionManagerCallback-9997-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3106340 WARN (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3106341 WARN (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@fd7b4d8[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3106355 INFO (zkConnectionManagerCallback-10004-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3106357 INFO (zkConnectionManagerCallback-10009-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3106359 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3106361 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3106362 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469450-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3106388 WARN (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3106388 INFO (OverseerStateUpdate-74394069969469450-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3106406 INFO (zkConnectionManagerCallback-10013-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3106423 INFO (zkCallback-9942-thread-4) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3106432 INFO (zkCallback-10008-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3106432 INFO (zkCallback-9996-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3106432 INFO (zkCallback-10012-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3106552 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3106557 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469453-127.0.0.1_core1-n_0000000000 [junit4] 2> 3106801 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2 [junit4] 2> 3106817 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/74394069969469453-127.0.0.1_core2-n_0000000000 [junit4] 2> 3106971 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard3 [junit4] 2> 3106976 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard3/leader after winning as /collections/collection1/leader_elect/shard3/election/74394069969469453-127.0.0.1_core3-n_0000000000 [junit4] 2> 3107095 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3107201 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2 [junit4] 2> 3107309 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard3 [junit4] 2> 3107321 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. [junit4] 2> 3107322 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. [junit4] 2> 3107322 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. [junit4] 2> 3107322 WARN (zkCallback-10012-thread-1) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader [junit4] 2> 3107323 WARN (zkCallback-10012-thread-1) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader [junit4] 2> 3107324 WARN (zkCallback-10012-thread-1) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader [junit4] 2> 3107324 INFO (zkCallback-10008-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3107324 INFO (zkCallback-9996-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3107325 INFO (zkCallback-10012-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3107351 INFO (zkCallback-9942-thread-4) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3107776 INFO (closeThreadPool-10017-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469450-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3107777 INFO (OverseerAutoScalingTriggerThread-74394069969469450-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3107777 INFO (OverseerStateUpdate-74394069969469450-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3107920 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3108089 INFO (TEST-OverseerTest.testShardAssignment-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testShardAssignment [junit4] 2> 3108094 INFO (TEST-OverseerTest.testLatchWatcher-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLatchWatcher [junit4] 2> 3108199 INFO (TEST-OverseerTest.testLatchWatcher-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testLatchWatcher [junit4] 2> 3108213 INFO (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testExceptionWhenFlushClusterState [junit4] 2> 3108419 INFO (zkConnectionManagerCallback-10023-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3108434 WARN (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3108434 WARN (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@40b8ced1[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3108436 INFO (zkConnectionManagerCallback-10030-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3108464 INFO (zkConnectionManagerCallback-10035-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3108528 INFO (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3108542 INFO (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3108542 INFO (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469454-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3108599 INFO (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3108601 WARN (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3108635 WARN (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception when process message = { [junit4] 2> "operation":"create", [junit4] 2> "name":"collection1", [junit4] 2> "replicationFactor":"1", [junit4] 2> "numShards":"1", [junit4] 2> "stateFormat":"2", [junit4] 2> "createNodeSet":""}, consider as bad message and poll out from the queue [junit4] 2> 3108638 ERROR (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception in Overseer when process message from work queue, retrying [junit4] 2> => org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) [junit4] 2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:204) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.enqueueUpdate(ZkStateWriter.java:130) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processQueueItem(Overseer.java:315) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:189) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3108849 ERROR (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception in Overseer main queue loop [junit4] 2> => org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) [junit4] 2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:204) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:259) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3108851 WARN (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception when process message = { [junit4] 2> "operation":"create", [junit4] 2> "name":"collection1", [junit4] 2> "replicationFactor":"1", [junit4] 2> "numShards":"1", [junit4] 2> "stateFormat":"2", [junit4] 2> "createNodeSet":""}, consider as bad message and poll out from the queue [junit4] 2> 3108852 ERROR (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception in Overseer when process message from work queue, retrying [junit4] 2> => org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) [junit4] 2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:204) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.enqueueUpdate(ZkStateWriter.java:130) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processQueueItem(Overseer.java:315) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:189) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3109100 INFO (closeThreadPool-10037-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469454-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3109100 INFO (OverseerStateUpdate-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3109121 INFO (OverseerAutoScalingTriggerThread-74394069969469454-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3109122 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3110217 INFO (TEST-OverseerTest.testExceptionWhenFlushClusterState-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testExceptionWhenFlushClusterState [junit4] 2> 3110306 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDownNodeFailover [junit4] 2> 3110624 INFO (zkConnectionManagerCallback-10039-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3110626 WARN (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3110626 WARN (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@fad07d3[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3110628 INFO (zkConnectionManagerCallback-10046-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3110630 INFO (zkConnectionManagerCallback-10051-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3110644 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3110697 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3110698 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469457-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3110739 INFO (OverseerStateUpdate-74394069969469457-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3110757 WARN (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3110761 INFO (zkConnectionManagerCallback-10055-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3110818 INFO (zkCallback-10038-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3110819 INFO (zkCallback-10054-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3110822 INFO (zkConnectionManagerCallback-10062-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3110826 INFO (zkCallback-9942-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3110875 INFO (zkCallback-10050-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3110875 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3110876 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3111050 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection0/leaders/shard1 [junit4] 2> 3111051 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection0/leaders/shard1/leader after winning as /collections/collection0/leader_elect/shard1/election/74394069969469460-127.0.0.1_core1-n_0000000000 [junit4] 2> 3111192 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3111194 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469460-127.0.0.1_core1-n_0000000000 [junit4] 2> 3111315 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard1 [junit4] 2> 3111345 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard1/leader after winning as /collections/collection2/leader_elect/shard1/election/74394069969469460-127.0.0.1_core1-n_0000000000 [junit4] 2> 3111454 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection3/leaders/shard1 [junit4] 2> 3111467 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection3/leaders/shard1/leader after winning as /collections/collection3/leader_elect/shard1/election/74394069969469460-127.0.0.1_core1-n_0000000000 [junit4] 2> 3111599 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection4/leaders/shard1 [junit4] 2> 3111601 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection4/leaders/shard1/leader after winning as /collections/collection4/leader_elect/shard1/election/74394069969469460-127.0.0.1_core1-n_0000000000 [junit4] 2> 3111737 INFO (zkConnectionManagerCallback-10064-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3111737 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469457-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3111737 INFO (OverseerStateUpdate-74394069969469457-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3111737 INFO (OverseerAutoScalingTriggerThread-74394069969469457-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3111761 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3111775 WARN (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3111776 WARN (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@6e08a621[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3111798 INFO (zkConnectionManagerCallback-10071-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3111829 INFO (zkConnectionManagerCallback-10076-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3111830 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3111844 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3111845 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3111846 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469462-127.0.0.1:42498_solr-n_0000000001) starting [junit4] 2> 3111857 WARN (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3111871 INFO (OverseerStateUpdate-74394069969469462-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3111873 INFO (OverseerStateUpdate-74394069969469462-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3111890 INFO (zkCallback-10061-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3111890 INFO (zkCallback-10054-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3111890 INFO (zkCallback-9942-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3111904 INFO (zkCallback-10050-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3111908 INFO (zkCallback-10063-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3111961 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1084d0331ed0014, likely client has closed socket [junit4] 2> 3111968 INFO (zkCallback-10075-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3112010 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1084d0331ed0012, likely client has closed socket [junit4] 2> 3112022 INFO (closeThreadPool-10078-thread-2) [ ] o.a.s.c.Overseer Overseer (id=74394069969469462-127.0.0.1:42498_solr-n_0000000001) closing [junit4] 2> 3112022 INFO (OverseerStateUpdate-74394069969469462-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3112023 INFO (OverseerAutoScalingTriggerThread-74394069969469462-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3112038 INFO (closeThreadPool-10078-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469457-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3112058 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3112175 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDownNodeFailover [junit4] 2> 3112199 INFO (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testStateChange [junit4] 2> 3112370 INFO (zkConnectionManagerCallback-10082-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3112372 WARN (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3112373 WARN (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@64b97982[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3112386 INFO (zkConnectionManagerCallback-10089-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3112389 INFO (zkConnectionManagerCallback-10094-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3112416 INFO (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3112435 INFO (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3112436 INFO (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469465-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3112455 INFO (OverseerStateUpdate-74394069969469465-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3112471 WARN (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3112927 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1084d0331ed001b, likely client has closed socket [junit4] 2> 3112990 INFO (closeThreadPool-10096-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469465-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3112990 INFO (OverseerStateUpdate-74394069969469465-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3112990 INFO (OverseerAutoScalingTriggerThread-74394069969469465-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3113354 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3113783 INFO (TEST-OverseerTest.testStateChange-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStateChange [junit4] 2> 3113856 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadQueueItem [junit4] 2> 3114282 INFO (zkConnectionManagerCallback-10098-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3114303 WARN (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3114304 WARN (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@108995d1[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3114319 INFO (zkConnectionManagerCallback-10105-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3114359 INFO (zkConnectionManagerCallback-10110-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3114360 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3114362 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3114362 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469468-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3114466 INFO (OverseerStateUpdate-74394069969469468-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3114501 WARN (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3114752 INFO (zkConnectionManagerCallback-10114-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3115370 INFO (zkCallback-9942-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3115405 INFO (zkCallback-10109-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3115442 INFO (zkCallback-10097-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3115453 INFO (zkCallback-10113-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3116031 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3116045 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469471-127.0.0.1_core1-n_0000000000 [junit4] 2> 3116230 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2 [junit4] 2> 3116284 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/74394069969469471-127.0.0.1_core2-n_0000000000 [junit4] 2> 3116429 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard3 [junit4] 2> 3116447 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard3/leader after winning as /collections/collection1/leader_elect/shard3/election/74394069969469471-127.0.0.1_core3-n_0000000000 [junit4] 2> 3116701 ERROR (OverseerStateUpdate-74394069969469468-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified: { [junit4] 2> "operation":"state", [junit4] 2> "state":"active", [junit4] 2> "node_name":"127.0.0.1", [junit4] 2> "core":"core0", [junit4] 2> "core_node_name":"node0", [junit4] 2> "collection":"", [junit4] 2> "shard":"shard1", [junit4] 2> "numShards":"1", [junit4] 2> "base_url":"http://127.0.0.1/solr/"} [junit4] 2> 3116753 ERROR (OverseerStateUpdate-74394069969469468-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified: { [junit4] 2> "operation":"deletecore", [junit4] 2> "node_name":"127.0.0.1", [junit4] 2> "core":"core0", [junit4] 2> "core_node_name":"node0", [junit4] 2> "collection":""} [junit4] 2> 3117011 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard1 [junit4] 2> 3117048 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard1/leader after winning as /collections/collection2/leader_elect/shard1/election/74394069969469471-127.0.0.1_core1-n_0000000000 [junit4] 2> 3117321 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard2 [junit4] 2> 3117406 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard2/leader after winning as /collections/collection2/leader_elect/shard2/election/74394069969469471-127.0.0.1_core2-n_0000000000 [junit4] 2> 3117569 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard3 [junit4] 2> 3117570 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard3/leader after winning as /collections/collection2/leader_elect/shard3/election/74394069969469471-127.0.0.1_core3-n_0000000000 [junit4] 2> 3117752 INFO (zkCallback-9942-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3117752 INFO (zkCallback-10109-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3117752 INFO (zkCallback-10097-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) [junit4] 2> 3117876 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1084d0331ed001e, likely client has closed socket [junit4] 2> 3118130 INFO (closeThreadPool-10118-thread-1) [ ] o.a.s.c.Overseer Overseer (id=74394069969469468-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3118131 INFO (OverseerStateUpdate-74394069969469468-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3118138 INFO (OverseerAutoScalingTriggerThread-74394069969469468-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3118250 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3118449 INFO (TEST-OverseerTest.testBadQueueItem-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem [junit4] 2> 3118616 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=2013987747 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=1086989557 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=1236901821 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=949151302 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=405077228 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=165191313 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=203009120 [junit4] 2> 3118650 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=597085043 [junit4] 2> 3118688 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerFailure [junit4] 2> 3119233 INFO (zkConnectionManagerCallback-10122-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3119307 INFO (zkCallback-9942-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3119324 INFO (zkCallback-10121-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3119324 INFO (zkConnectionManagerCallback-10126-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3119359 WARN (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3119360 WARN (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@3edd78be[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3119384 INFO (zkConnectionManagerCallback-10133-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3119444 INFO (zkConnectionManagerCallback-10138-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3119476 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3119477 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3119478 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3119496 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469473-127.0.0.1:42498_solr-n_0000000000) starting [junit4] 2> 3119552 INFO (OverseerStateUpdate-74394069969469473-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3119553 INFO (OverseerStateUpdate-74394069969469473-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3119577 WARN (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3119886 INFO (zkConnectionManagerCallback-10143-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3119905 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3119906 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3120070 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3120140 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469472-node1_core1-n_0000000000 [junit4] 2> 3120267 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3120268 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469472-node1_core1-n_0000000001 [junit4] 2> 3120756 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 [junit4] 2> 3121246 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/74394069969469472-node1_core1-n_0000000002 [junit4] 2> 3121302 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1084d0331ed0021, likely client has closed socket [junit4] 2> 3121383 INFO (zkConnectionManagerCallback-10145-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3121384 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469473-127.0.0.1:42498_solr-n_0000000000) closing [junit4] 2> 3121391 INFO (OverseerAutoScalingTriggerThread-74394069969469473-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting. [junit4] 2> 3121598 INFO (OverseerStateUpdate-74394069969469473-127.0.0.1:42498_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42498_solr [junit4] 2> 3122113 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) ~[java/:?] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 3122131 WARN (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2 [junit4] 2> 3122149 WARN (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@38fad75c[provider=null,keyStore=null,trustStore=null] [junit4] 2> 3122169 INFO (zkConnectionManagerCallback-10152-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3122246 INFO (zkConnectionManagerCallback-10157-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 3122263 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3122264 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42498/solr ready [junit4] 2> 3122265 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42498_solr [junit4] 2> 3122266 INFO (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Overseer (id=74394069969469477-127.0.0.1:42498_solr-n_0000000001) starting [junit4] 2> 3122332 INFO (OverseerStateUpdate-74394069969469477-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42498_solr [junit4] 2> 3122333 INFO (OverseerStateUpdate-74394069969469477-127.0.0.1:42498_solr-n_0000000001) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 3122422 WARN (TEST-OverseerTest.testOverseerFailure-seed#[BB0261892F804B90]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState. [junit4] 2> 3136095 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=476741745 [junit4] 2> 3179238 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=1368107387 [junit4] 2> mar 23, 2019 6:13:13 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate [junit4] 2> WARNING: Suite execution timed out: org.apache.solr.cloud.OverseerTest [junit4] 2> ==== jstack at approximately timeout time ==== [junit4] 2> "ScheduledTrigger-9292-thread-4" ID=27752 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] 2> at java.lang.Thread.run(Thread.java:748) [junit4] 2> [junit4] 2> "ScheduledTrigger-9292-thread-3" ID=27751 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] 2> at java.lang.Thread.run(Thread.java:748) [junit4] 2> [junit4] 2> "ScheduledTrigger-9292-thread-2" ID=27750 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> - timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] 2> at java.lang.Thread.run(Thread.java:748) [junit4] 2> [junit4] 2> "ScheduledTrigger-9292-thread-1" ID=27749 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a542aee [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] [...truncated too long message...] ail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml resolve: jar-checksums: [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1111392919 [copy] Copying 240 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1111392919 [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1111392919 check-working-copy: [ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working [ivy:cachepath] confs: [default] [ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public [ivy:cachepath] found com.jcraft#jsch;0.1.53 in public [ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public [ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public [ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public [ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public [ivy:cachepath] found commons-codec#commons-codec;1.6 in public [ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public [ivy:cachepath] :: resolution report :: resolve 73ms :: artifacts dl 1ms --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 8 | 0 | 0 | 0 || 8 | 0 | --------------------------------------------------------------------- [wc-checker] Initializing working copy... [wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". [wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation [wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. [wc-checker] Checking working copy status... -jenkins-base: BUILD SUCCESSFUL Total time: 315 minutes 7 seconds Archiving artifacts java.lang.InterruptedException: no matches found within 10000 at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2847) at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2726) at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2707) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:955) at hudson.FilePath.act(FilePath.java:1072) at hudson.FilePath.act(FilePath.java:1061) at hudson.FilePath.validateAntFileMask(FilePath.java:2705) at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690) at hudson.model.Build$BuildExecution.post2(Build.java:186) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635) at hudson.model.Run.execute(Run.java:1835) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused: hudson.FilePath$TunneledInterruptedException at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3088) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000 at hudson.FilePath.act(FilePath.java:1074) at hudson.FilePath.act(FilePath.java:1061) at hudson.FilePath.validateAntFileMask(FilePath.java:2705) at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690) at hudson.model.Build$BuildExecution.post2(Build.java:186) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635) at hudson.model.Run.execute(Run.java:1835) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error? Recording test results Build step 'Publish JUnit test result report' changed build result to UNSTABLE Email was triggered for: Unstable (Test Failures) Sending email for trigger: Unstable (Test Failures)
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org