Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20608/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry

Error Message:
Captured an uncaught exception in thread: Thread[id=7095, 
name=OverseerAutoScalingTriggerThread-98773144295571458-dummy.host.com:8984_solr-n_0000000000,
 state=RUNNABLE, group=Overseer autoscaling triggers]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=7095, 
name=OverseerAutoScalingTriggerThread-98773144295571458-dummy.host.com:8984_solr-n_0000000000,
 state=RUNNABLE, group=Overseer autoscaling triggers]
        at 
__randomizedtesting.SeedInfo.seed([FC5CF73E695F42E6:2A83D6645F42517E]:0)
Caused by: org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
        at __randomizedtesting.SeedInfo.seed([FC5CF73E695F42E6]:0)
        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:107)
        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:44)
        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:398)
        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:149)
        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:220)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /autoscaling/events/.auto_add_replicas
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:105)
        ... 5 more




Build Log:
[...truncated 12396 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestLeaderElectionZkExpiry
   [junit4]   2> 1024382 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_FC5CF73E695F42E6-001/init-core-data-001
   [junit4]   2> 1024383 WARN  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1024383 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1024385 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1024386 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderElectionWithZkExpiry
   [junit4]   2> 1024390 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@76602c3f, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1024418 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 
'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@76602c3f
   [junit4]   2> 1024424 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 
'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@76602c3f
   [junit4]   2> 1024424 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@76602c3f
   [junit4]   2> 1024425 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_FC5CF73E695F42E6-001/testLeaderElectionWithZkExpiry-solr-001
   [junit4]   2> 1024426 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1024426 INFO  (Thread-1600) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1024426 INFO  (Thread-1600) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1024427 ERROR (Thread-1600) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1024526 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [    ] o.a.s.c.ZkTestServer start zk server on port:40295
   [junit4]   2> 1024582 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1024583 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerElectionContext I am going to 
be the leader dummy.host.com:8984_solr
   [junit4]   2> 1024583 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=98773144295571458-dummy.host.com:8984_solr-n_0000000000) starting
   [junit4]   2> 1024588 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1024589 INFO  
(OverseerStateUpdate-98773144295571458-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1024595 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Exception causing close of session 0x15ee9a6c3160002 due 
to java.nio.channels.CancelledKeyException
   [junit4]   2> 1024695 WARN  
(zkCallback-954-thread-2-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1024695 WARN  
(zkCallback-954-thread-2-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1026086 WARN  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6]-SendThread(127.0.0.1:40295))
 [n:dummy.host.com:8984_solr    ] o.a.z.ClientCnxn Unable to reconnect to 
ZooKeeper service, session 0x15ee9a6c3160002 has expired
   [junit4]   2> 1026087 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1026087 WARN  
(OverseerStateUpdate-98773144295571458-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Solr cannot talk to ZK, 
exiting Overseer main queue loop
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer/queue
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:313)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:290)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstElement(ZkDistributedQueue.java:406)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:165)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:142)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:185)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1026087 INFO  
(OverseerStateUpdate-98773144295571458-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1026087 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 1026087 WARN  
(OverseerCollectionConfigSetProcessor-98773144295571458-dummy.host.com:8984_solr-n_0000000000)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerTaskProcessor Overseer cannot 
talk to ZK
   [junit4]   2> 1026087 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=98773144295571458-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 1026088 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr 
   ] o.a.s.c.Overseer could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:299)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1026195 WARN  
(zkCallback-954-thread-1-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch 
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for 
/live_nodes]
   [junit4]   2> okt. 05, 2017 12:09:35 AM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: 
Thread[OverseerAutoScalingTriggerThread-98773144295571458-dummy.host.com:8984_solr-n_0000000000,5,Overseer
 autoscaling triggers]
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
__randomizedtesting.SeedInfo.seed([FC5CF73E695F42E6]:0)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:107)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:44)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:398)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:149)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:220)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:105)
   [junit4]   2>        ... 5 more
   [junit4]   2> 
   [junit4]   2> 1026199 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection 
expired - starting a new one...
   [junit4]   2> 1026200 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connection with 
ZooKeeper reestablished.
   [junit4]   2> 1026200 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController ZooKeeper session 
re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1026200 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (1) -> (0)
   [junit4]   2> 1026201 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerElectionContext I am going to 
be the leader dummy.host.com:8984_solr
   [junit4]   2> 1026202 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=98773144295571459-dummy.host.com:8984_solr-n_0000000001) starting
   [junit4]   2> 1026203 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1026204 INFO  
(zkCallback-954-thread-1-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1026204 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy 
Reconnected to ZooKeeper
   [junit4]   2> 1026204 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 1026308 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1026308 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1027589 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:40295))
 [n:dummy.host.com:8984_solr    ] o.a.z.ClientCnxn Unable to reconnect to 
ZooKeeper service, session 0x15ee9a6c3160003 has expired
   [junit4]   2> 1027589 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1027589 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 1027589 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=98773144295571459-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 1027589 WARN  
(OverseerAutoScalingTriggerThread-98773144295571459-dummy.host.com:8984_solr-n_0000000001)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.a.OverseerTriggerThread 
OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1027590 INFO  
(OverseerStateUpdate-98773144295571459-dummy.host.com:8984_solr-n_0000000001) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer According to ZK I 
(id=98773144295571459-dummy.host.com:8984_solr-n_0000000001) am no longer a 
leader.
   [junit4]   2> 1027591 INFO  
(OverseerStateUpdate-98773144295571459-dummy.host.com:8984_solr-n_0000000001) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1027591 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection 
expired - starting a new one...
   [junit4]   2> 1027591 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr 
   ] o.a.s.c.Overseer could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:299)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1027592 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connection with 
ZooKeeper reestablished.
   [junit4]   2> 1027592 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController ZooKeeper session 
re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1027693 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1027693 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1028756 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:40295))
 [n:dummy.host.com:8984_solr    ] o.a.z.ClientCnxn Unable to reconnect to 
ZooKeeper service, session 0x15ee9a6c3160004 has expired
   [junit4]   2> 1028756 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1028756 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 1028756 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection 
expired - starting a new one...
   [junit4]   2> 1028758 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connection with 
ZooKeeper reestablished.
   [junit4]   2> 1028758 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController ZooKeeper session 
re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1028856 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch 
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for 
/live_nodes]
   [junit4]   2> 1028867 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1028867 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1030769 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:40295))
 [n:dummy.host.com:8984_solr    ] o.a.z.ClientCnxn Unable to reconnect to 
ZooKeeper service, session 0x15ee9a6c3160005 has expired
   [junit4]   2> 1030769 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1030769 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 1030769 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection 
expired - starting a new one...
   [junit4]   2> 1030770 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connection with 
ZooKeeper reestablished.
   [junit4]   2> 1030770 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController ZooKeeper session 
re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1030869 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Error reading cluster 
properties from zookeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /clusterprops.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.loadClusterProperties(ZkStateReader.java:948)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:432)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 1030877 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1030877 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1032762 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:40295))
 [n:dummy.host.com:8984_solr    ] o.a.z.ClientCnxn Unable to reconnect to 
ZooKeeper service, session 0x15ee9a6c3160006 has expired
   [junit4]   2> 1032762 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1032762 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 1032762 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection 
expired - starting a new one...
   [junit4]   2> 1032763 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connection with 
ZooKeeper reestablished.
   [junit4]   2> 1032764 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController ZooKeeper session 
re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1032863 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController 
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
= Session expired for /live_nodes
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:716)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:433)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 1032863 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /live_nodes
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:716)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:433)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2>        ... 5 more
   [junit4]   2> 
   [junit4]   2> 1032863 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed
   [junit4]   2> 1032864 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 1032867 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1032867 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient has 
disconnected
   [junit4]   2> 1034758 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:40295))
 [n:dummy.host.com:8984_solr    ] o.a.z.ClientCnxn Unable to reconnect to 
ZooKeeper service, session 0x15ee9a6c3160007 has expired
   [junit4]   2> 1034758 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6814e774 name: 
ZooKeeperConnection Watcher:127.0.0.1:40295/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1034758 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 1034758 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection 
expired - starting a new one...
   [junit4]   2> 1034759 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connection with 
ZooKeeper reestablished.
   [junit4]   2> 1034759 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController ZooKeeper session 
re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1034858 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController 
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
= Session expired for /clusterstate.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:426)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 1034859 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /clusterstate.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:426)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:322)
   [junit4]   2>        ... 5 more
   [junit4]   2> 
   [junit4]   2> 1034859 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (1) -> (0)
   [junit4]   2> 1034859 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed
   [junit4]   2> 1034859 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 1034860 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerElectionContext I am going to 
be the leader dummy.host.com:8984_solr
   [junit4]   2> 1034861 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=98773144295571465-dummy.host.com:8984_solr-n_0000000002) starting
   [junit4]   2> 1034861 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.TestLeaderElectionZkExpiry 
Time=3077433598591527 Overseer leader is = dummy.host.com:8984_solr
   [junit4]   2> 1034862 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=98773144295571465-dummy.host.com:8984_solr-n_0000000002) closing
   [junit4]   2> 1034862 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1034862 INFO  
(OverseerStateUpdate-98773144295571465-dummy.host.com:8984_solr-n_0000000002) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer According to ZK I 
(id=98773144295571465-dummy.host.com:8984_solr-n_0000000002) am no longer a 
leader.
   [junit4]   2> 1034862 INFO  
(OverseerStateUpdate-98773144295571465-dummy.host.com:8984_solr-n_0000000002) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1034862 WARN  
(OverseerAutoScalingTriggerThread-98773144295571465-dummy.host.com:8984_solr-n_0000000002)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at java.lang.Object.wait(Native Method)
   [junit4]   2>        at java.lang.Object.wait(Object.java:502)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:185)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:171)
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:116)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1034863 INFO  
(zkCallback-954-thread-3-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1034866 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy 
Reconnected to ZooKeeper
   [junit4]   2> 1034866 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 1034862 INFO  
(OverseerCollectionConfigSetProcessor-98773144295571465-dummy.host.com:8984_solr-n_0000000002)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerTaskProcessor According to ZK 
I (id=98773144295571465-dummy.host.com:8984_solr-n_0000000002) am no longer a 
leader.
   [junit4]   2> 1034867 ERROR 
(OverseerCollectionConfigSetProcessor-98773144295571465-dummy.host.com:8984_solr-n_0000000002)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerTaskProcessor Unable to 
prioritize overseer 
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2>        at java.lang.Object.wait(Native Method)
   [junit4]   2>        at java.lang.Object.wait(Object.java:502)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1097)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:61)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1034867 WARN  (OverseerExitThread) [n:dummy.host.com:8984_solr 
   ] o.a.s.c.Overseer I'm exiting, but I'm still the leader
   [junit4]   2> 1034867 WARN  
(zkCallback-954-thread-5-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch 
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for 
/live_nodes]
   [junit4]   2> 1034867 WARN  
(zkCallback-954-thread-3-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch 
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for 
/live_nodes]
   [junit4]   2> 1034867 WARN  
(zkCallback-954-thread-6-processing-n:dummy.host.com:8984_solr) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch 
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for 
/live_nodes]
   [junit4]   2> 1034867 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.CoreContainer Shutting down 
CoreContainer instance=519072380
   [junit4]   2> 1034867 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.node, tag=null
   [junit4]   2> 1034867 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@662d9bd4: rootName = null, 
domain = solr.node, service url = null, agent id = null] for registry solr.node 
/ com.codahale.metrics.MetricRegistry@11f99fa3
   [junit4]   2> 1034870 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1034870 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@5637ad4f: rootName = null, 
domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / 
com.codahale.metrics.MetricRegistry@5432f2f8
   [junit4]   2> 1034878 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1034878 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@cb7cd1e: rootName = null, 
domain = solr.jetty, service url = null, agent id = null] for registry 
solr.jetty / com.codahale.metrics.MetricRegistry@500d7ca6
   [junit4]   2> 1034878 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.TransientSolrCoreCacheDefault 
Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1034878 ERROR 
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is 
not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN 
server state changes
   [junit4]   2> 1034879 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.c.ZkTestServer connecting to 
127.0.0.1:40295 40295
   [junit4]   2> 1036367 WARN  (OverseerExitThread) [n:dummy.host.com:8984_solr 
   ] o.a.s.c.Overseer Unable to rejoinElection 
   [junit4]   2> org.apache.solr.common.SolrException: Unable to rejoin election
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1924)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:324)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /overseer_elect
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:772)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerElectionContext.copy(ElectionContext.java:815)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:385)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1921)
   [junit4]   2>        ... 2 more
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /overseer_elect
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerElectionContext.<init>(ElectionContext.java:770)
   [junit4]   2>        ... 5 more
   [junit4]   2> 1036367 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController 
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
= Session expired for /overseer_elect/election
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 1036367 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController 
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
= Session expired for /overseer_elect/election
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 
   [junit4]   2> 1036368 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /overseer_elect/election
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2>        ... 5 more
   [junit4]   2> 
   [junit4]   2> 1036369 ERROR 
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:407)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /overseer_elect/election
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:347)
   [junit4]   2>        ... 5 more
   [junit4]   2> 
   [junit4]   2> 1036369 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed
   [junit4]   2> 1036369 WARN  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed
   [junit4]   2> 1036369 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 1036369 INFO  
(zkCallback-954-thread-4-processing-n:dummy.host.com:8984_solr-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 1041875 INFO  (Thread-1600) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:40295 40295
   [junit4]   2> 1041875 WARN  (Thread-1600) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/aliases.json
   [junit4]   2>        5       /solr/security.json
   [junit4]   2>        5       /solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/clusterstate.json
   [junit4]   2>        2       /solr/autoscaling.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/live_nodes
   [junit4]   2>        5       /solr/collections
   [junit4]   2>        2       /solr/overseer/queue
   [junit4]   2>        2       /solr/overseer/collection-queue-work
   [junit4]   2>        2       /solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> 1041875 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[FC5CF73E695F42E6])
 [n:dummy.host.com:8984_solr    ] o.a.s.SolrTestCaseJ4 ###Ending 
testLeaderElectionWithZkExpiry
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestLeaderElectionZkExpiry 
-Dtests.method=testLeaderElectionWithZkExpiry -Dtests.seed=FC5CF73E695F42E6 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lv-LV 
-Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] ERROR   17.5s J0 | 
TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=7095, 
name=OverseerAutoScalingTriggerThread-98773144295571458-dummy.host.com:8984_solr-n_0000000000,
 state=RUNNABLE, group=Overseer autoscaling triggers]
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([FC5CF73E695F42E6:2A83D6645F42517E]:0)
   [junit4]    > Caused by: org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([FC5CF73E695F42E6]:0)
   [junit4]    >        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:107)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:44)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:398)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:149)
   [junit4]    >        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:220)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]    >        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
   [junit4]    >        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:105)
   [junit4]    >        ... 5 more
   [junit4]   2> 1041876 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) 
[n:dummy.host.com:8984_solr    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1041876 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.CoreContainer Shutting down 
CoreContainer instance=519072380
   [junit4]   2> 1041877 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) 
[n:dummy.host.com:8984_solr    ] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.node, tag=null
   [junit4]   2> 1041877 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) 
[n:dummy.host.com:8984_solr    ] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1041877 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) 
[n:dummy.host.com:8984_solr    ] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1041877 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[FC5CF73E695F42E6]-worker) 
[n:dummy.host.com:8984_solr    ] o.a.s.SolrTestCaseJ4 
------------------------------------------------------- Done waiting for 
tracked resources to be released
   [junit4]   2> NOTE: test params are: codec=Lucene70, 
sim=RandomSimilarity(queryNorm=true): {}, locale=lv-LV, 
timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Linux 4.10.0-33-generic amd64/Oracle Corporation 
1.8.0_144 (64-bit)/cpus=8,threads=1,free=257228352,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [PluginInfoTest, 
PeerSyncTest, HttpTriggerListenerTest, FieldMutatingUpdateProcessorTest, 
LeaderFailoverAfterPartitionTest, TestSolrQueryParser, TestCSVLoader, 
TestEmbeddedSolrServerSchemaAPI, TestReplicationHandler, TestBinaryField, 
ShardSplitTest, TestFastOutputStream, PeerSyncWithIndexFingerprintCachingTest, 
OverseerRolesTest, BasicZkTest, TestQueryWrapperFilter, 
AddSchemaFieldsUpdateProcessorFactoryTest, SolrPluginUtilsTest, 
CdcrBootstrapTest, InfixSuggestersTest, TestIndexingPerformance, 
TestExclusionRuleCollectionAccess, TestBulkSchemaAPI, RequestHandlersTest, 
TestSolr4Spatial2, TestSearcherReuse, TestMergePolicyConfig, 
AnalyticsQueryTest, ConcurrentDeleteAndCreateCollectionTest, 
SpellCheckCollatorTest, CdcrRequestHandlerTest, DistributedFacetPivotSmallTest, 
TestNumericTerms64, TestPhraseSuggestions, TestCollationField, 
TestReloadAndDeleteDocs, TestCSVResponseWriter, TestInPlaceUpdatesStandalone, 
DocumentAnalysisRequestHandlerTest, TestRandomFaceting, UUIDFieldTest, 
TestLazyCores, TestConfigSetProperties, TestAuthenticationFramework, 
HLLUtilTest, TestReRankQParserPlugin, ClassificationUpdateProcessorFactoryTest, 
IndexBasedSpellCheckerTest, UninvertDocValuesMergePolicyTest, TestRealTimeGet, 
TestSSLRandomization, ParsingFieldUpdateProcessorsTest, HdfsSyncSliceTest, 
SuggesterWFSTTest, TestPivotHelperCode, TestPrepRecovery, 
SolrRequestParserTest, TestRetrieveFieldsOptimizer, 
DistributedQueryComponentOptimizationTest, TestBlendedInfixSuggestions, 
SolrGangliaReporterTest, AutoAddReplicasIntegrationTest, 
TestReplicationHandlerBackup, TestExtendedDismaxParser, 
TestOverriddenPrefixQueryForCustomFieldType, ExternalFileFieldSortTest, 
TestComponentsName, TestBackupRepositoryFactory, 
BinaryUpdateRequestHandlerTest, TestLeaderElectionZkExpiry]
   [junit4] Completed [278/745 (1!)] on J0 in 17.51s, 1 test, 1 error <<< 
FAILURES!

[...truncated 48411 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:826: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:706: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:693: Source checkout 
is dirty (unversioned/missing files) after running tests!!! Offending files:
* lucene/licenses/morfologik-ukrainian-search-3.7.5.jar.sha1

Total time: 82 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to