Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/115/

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationDistributedZkTest

Error Message:
9 threads leaked from SUITE scope at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest:     1) Thread[id=24299, 
name=cdcr-scheduler-3455-thread-1, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
         at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=24370, 
name=cdcr-replicator-3456-thread-8, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    3) Thread[id=24331, 
name=cdcr-replicator-3456-thread-7, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    4) Thread[id=24330, 
name=cdcr-replicator-3456-thread-6, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    5) Thread[id=24327, 
name=cdcr-replicator-3456-thread-5, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    6) Thread[id=24313, 
name=cdcr-replicator-3456-thread-2, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    7) Thread[id=24300, 
name=cdcr-replicator-3456-thread-1, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    8) Thread[id=24320, 
name=cdcr-replicator-3456-thread-3, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    9) Thread[id=24321, 
name=cdcr-replicator-3456-thread-4, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 9 threads leaked from SUITE 
scope at org.apache.solr.cloud.CdcrReplicationDistributedZkTest: 
   1) Thread[id=24299, name=cdcr-scheduler-3455-thread-1, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=24370, name=cdcr-replicator-3456-thread-8, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=24331, name=cdcr-replicator-3456-thread-7, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=24330, name=cdcr-replicator-3456-thread-6, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=24327, name=cdcr-replicator-3456-thread-5, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=24313, name=cdcr-replicator-3456-thread-2, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=24300, name=cdcr-replicator-3456-thread-1, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=24320, name=cdcr-replicator-3456-thread-3, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=24321, name=cdcr-replicator-3456-thread-4, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([D6671974DEEF3170]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=24299, name=cdcr-scheduler-3455-thread-1, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
         at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=24370, 
name=cdcr-replicator-3456-thread-8, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    3) Thread[id=24331, 
name=cdcr-replicator-3456-thread-7, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    4) Thread[id=24330, 
name=cdcr-replicator-3456-thread-6, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    5) Thread[id=24327, 
name=cdcr-replicator-3456-thread-5, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    6) Thread[id=24313, 
name=cdcr-replicator-3456-thread-2, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    7) Thread[id=24300, 
name=cdcr-replicator-3456-thread-1, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    8) Thread[id=24320, 
name=cdcr-replicator-3456-thread-3, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    9) Thread[id=24321, 
name=cdcr-replicator-3456-thread-4, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=24299, name=cdcr-scheduler-3455-thread-1, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=24370, name=cdcr-replicator-3456-thread-8, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=24331, name=cdcr-replicator-3456-thread-7, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=24330, name=cdcr-replicator-3456-thread-6, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=24327, name=cdcr-replicator-3456-thread-5, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=24313, name=cdcr-replicator-3456-thread-2, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=24300, name=cdcr-replicator-3456-thread-1, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=24320, name=cdcr-replicator-3456-thread-3, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=24321, name=cdcr-replicator-3456-thread-4, state=WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([D6671974DEEF3170]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:37506/nl_xy/gp: Error CREATEing SolrCore 
'halfcollection_shard1_replica1': Unable to create core 
[halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired 
for /overseer/queue/qn-

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:37506/nl_xy/gp: Error CREATEing SolrCore 
'halfcollection_shard1_replica1': Unable to create core 
[halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired 
for /overseer/queue/qn-
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11911 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/init-core-data-001
   [junit4]   2> 1148874 INFO  
(SUITE-CdcrReplicationDistributedZkTest-seed#[D6671974DEEF3170]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1148874 INFO  
(SUITE-CdcrReplicationDistributedZkTest-seed#[D6671974DEEF3170]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: 
/wjgbk/d
   [junit4]   2> 1148880 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1148880 INFO  (Thread-18169) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1148880 INFO  (Thread-18169) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1148980 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.ZkTestServer start zk server on port:55015
   [junit4]   2> 1148981 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1148994 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1149001 INFO  (zkCallback-705-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5d8d481e 
name:ZooKeeperConnection Watcher:127.0.0.1:55015 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1149005 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1149005 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1149005 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1149011 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1149013 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1149019 INFO  (zkCallback-706-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6924f66f 
name:ZooKeeperConnection Watcher:127.0.0.1:55015/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1149020 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1149020 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1149020 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1149027 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1149029 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1149029 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1149031 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1149031 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1149032 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1149033 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1149034 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1149034 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1149035 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1149035 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1149036 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1149037 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1149038 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1149038 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1149039 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1149039 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1149040 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1149041 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1149042 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1149042 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1149043 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1149043 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1149045 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1149045 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1149047 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testReplicationAfterRestart
   [junit4]   2> 1149146 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1
   [junit4]   2> 1149147 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1149171 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@7b15605e{/wjgbk/d,null,AVAILABLE}
   [junit4]   2> 1149173 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.e.j.s.ServerConnector Started 
ServerConnector@19b99cf5{HTTP/1.1,[http/1.1]}{127.0.0.1:49096}
   [junit4]   2> 1149173 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.e.j.s.Server Started @1153183ms
   [junit4]   2> 1149173 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, 
hostContext=/wjgbk/d, hostPort=49096, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores}
   [junit4]   2> 1149173 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1149173 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001'
   [junit4]   2> 1149174 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr 
(NoInitialContextEx)
   [junit4]   2> 1149174 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not 
find system property or JNDI)
   [junit4]   2> 1149174 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1149176 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1149176 INFO  (zkCallback-707-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7f54adc8 
name:ZooKeeperConnection Watcher:127.0.0.1:55015/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1149176 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1149176 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1149178 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1149178 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/solr.xml
   [junit4]   2> 1149184 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores
   [junit4]   2> 1149184 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.CoreContainer New CoreContainer 107768317
   [junit4]   2> 1149184 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001]
   [junit4]   2> 1149184 WARN  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/lib
   [junit4]   2> 1149187 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 
90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 
20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 
2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : 
false,useRetries : false,
   [junit4]   2> 1149188 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1149189 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1149189 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1149189 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55015/solr
   [junit4]   2> 1149189 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1149189 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1149190 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1149191 INFO  (zkCallback-710-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@647db108 
name:ZooKeeperConnection Watcher:127.0.0.1:55015 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1149191 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1149192 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1149196 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$$Lambda$35/1318240705@33e88e16
   [junit4]   2> 1149202 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ConnectionManager Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 1149202 INFO  
(zkCallback-711-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4eaef895 
name:ZooKeeperConnection Watcher:127.0.0.1:55015/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1149202 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ConnectionManager Client is 
connected to ZooKeeper
   [junit4]   2> 1149203 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue
   [junit4]   2> 1149205 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 1149207 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 1149208 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 1149209 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 1149210 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1149211 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/aliases.json
   [junit4]   2> 1149212 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/clusterstate.json
   [junit4]   2> 1149213 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/security.json
   [junit4]   2> 1149214 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader Updating cluster 
state from ZooKeeper... 
   [junit4]   2> 1149214 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader Loaded empty cluster 
properties
   [junit4]   2> 1149215 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (0)
   [junit4]   2> 1149217 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect
   [junit4]   2> 1149218 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 1149219 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1149219 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.LeaderElector Joined leadership 
election with path: 
/overseer_elect/election/96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000
   [junit4]   2> 1149219 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.OverseerElectionContext I am going 
to be the leader 127.0.0.1:49096_wjgbk%2Fd
   [junit4]   2> 1149219 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 1149220 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.Overseer Overseer 
(id=96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) starting
   [junit4]   2> 1149222 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue-work
   [junit4]   2> 1149225 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.OverseerAutoReplicaFailoverThread 
Starting OverseerAutoReplicaFailoverThread 
autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1149233 INFO  
(OverseerCollectionConfigSetProcessor-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000)
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.OverseerTaskProcessor Process 
current queue of overseer operations
   [junit4]   2> 1149233 INFO  
(OverseerStateUpdate-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.Overseer Starting to work on the main 
queue
   [junit4]   2> 1149235 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.ZkController Register node as live 
in ZooKeeper:/live_nodes/127.0.0.1:49096_wjgbk%2Fd
   [junit4]   2> 1149235 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:49096_wjgbk%2Fd
   [junit4]   2> 1149236 INFO  
(OverseerStateUpdate-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (0)
   [junit4]   2> 1149237 INFO  
(zkCallback-711-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader A live node change: 
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], 
has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1149238 INFO  
(zkCallback-711-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1149248 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.CoreContainer Security conf doesn't 
exist. Skipping setup for authorization module.
   [junit4]   2> 1149248 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.CoreContainer No authentication 
plugin used.
   [junit4]   2> 1149249 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.CorePropertiesLocator Looking for 
core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores
   [junit4]   2> 1149249 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.CoreDescriptor Created 
CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, 
schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, 
collection=tmp_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1149249 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.CorePropertiesLocator Found core 
collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1
   [junit4]   2> 1149250 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions
   [junit4]   2> 1149250 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1149250 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 1149251 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController Collection tmp_collection not visible yet, but flagging it 
so a watch is registered when it becomes visible
   [junit4]   2> 1149252 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Deleting data for [tmp_collection]
   [junit4]   2> 1149252 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 1149252 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[D6671974DEEF3170])
 [n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 1149253 INFO  
(OverseerStateUpdate-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49096/wjgbk/d";,
   [junit4]   2>   "node_name":"127.0.0.1:49096_wjgbk%2Fd",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1149253 INFO  
(OverseerStateUpdate-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49096/wjgbk/d";,
   [junit4]   2>   "node_name":"127.0.0.1:49096_wjgbk%2Fd",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1149253 INFO  
(OverseerStateUpdate-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: tmp_collection
   [junit4]   2> 1149253 INFO  
(OverseerStateUpdate-96212485109383172-127.0.0.1:49096_wjgbk%2Fd-n_0000000000) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard2
   [junit4]   2> 1149254 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1149254 INFO  
(zkCallback-711-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state 
change: [WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1150254 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1150254 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController Check for collection zkNode:tmp_collection
   [junit4]   2> 1150255 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 1150255 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController Looking for collection configName
   [junit4]   2> 1150256 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ZkController Only one config set found in zk - using it:conf1
   [junit4]   2> 1150256 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection
   [junit4]   2> 1150256 INFO  
(zkCallback-711-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd    ] o.a.s.c.c.ZkStateReader A collections change: 
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], 
has occurred - updating...
   [junit4]   2> 1150257 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/tmp_collection]
   [junit4]   2> 1150257 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/tmp_collection] [configName]=[conf1] 
specified config exists in ZooKeeper
   [junit4]   2> 1150257 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1'
   [junit4]   2> 1150258 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1150258 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1150259 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] o.a.s.c.Config 
loaded config solrconfig.xml with version 0 
   [junit4]   2> 1150263 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1150266 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 1150272 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1150277 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1150340 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1150341 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1150341 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1150344 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1150368 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 1150368 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection tmp_collection
   [junit4]   2> 1150368 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1150368 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data/]
   [junit4]   2> 1150368 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d62a0ad
   [junit4]   2> 1150370 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data
   [junit4]   2> 1150371 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore New index directory detected: old=null 
new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data/index/
   [junit4]   2> 1150371 WARN  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data/index'
 doesn't exist. Creating new index...
   [junit4]   2> 1150371 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data/index
   [junit4]   2> 1150371 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.14336671117941605]
   [junit4]   2> 1150371 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: 
num=1
   [junit4]   2>        commit{dir=MockDirectoryWrapper(RAMDirectory@3f84c373 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33493806),segFN=segments_1,generation=1}
   [junit4]   2> 1150371 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1150372 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup 
for core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data/
   [junit4]   2> 1150372 WARN  
(OldIndexDirectoryCleanupThreadForCore-collection1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.DirectoryFactory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_D6671974DEEF3170-001/jetty-001/cores/collection1/data/
 does not point to a valid data directory; skipping clean-up of old index 
directories.
   [junit4]   2> 1150376 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 1150388 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as 
default, creating implicit default
   [junit4]   2> 1150389 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150390 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150391 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150392 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1150401 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/cdcr,/update,/admin/file,/terms
   [junit4]   2> 1150401 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1150402 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 1150402 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
numVersionBuckets=65536
   [junit4]   2> 1150402 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1150402 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1150403 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=50, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=0.982421875, 
floorSegmentMB=2.140625, forceMergeDeletesPctAllowed=10.449872342200138, 
segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.12558099461100528
   [junit4]   2> 1150403 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: 
num=1
   [junit4]   2>        commit{dir=MockDirectoryWrapper(RAMDirectory@3f84c373 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33493806),segFN=segments_1,generation=1}
   [junit4]   2> 1150403 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1150403 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening 
[Searcher@24d26c49[collection1] main]
   [junit4]   2> 1150403 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/tmp_collection]
   [junit4]   2> 1150404 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/tmp_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1150404 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based 
storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1150404 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 1150405 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1150405 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1150405 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 1150405 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1150405 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1150405 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1150409 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.c.SolrZkClient makePath: 
/collections/tmp_collection/cdcr/state
   [junit4]   2> 1150411 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.CdcrBufferStateManager Created znode 
/collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 1150415 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.h.CdcrProcessStateManager Created znode 
/collections/tmp_collection/cdcr/state/process
   [junit4]   2> 1150427 INFO  
(searcherExecutor-2494-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd 
x:collection1 s:shard2 c:tmp_collection r:core_node1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@24d26c49[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1150427 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed 
version buckets
   [junit4]   2> 1150427 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 
65536 version buckets from index
   [junit4]   2> 1150427 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed 
version bucket highest value from index
   [junit4]   2> 1150427 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent 
updates, using new clock 1539399763371229184
   [junit4]   2> 1150429 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with 
highest version 1539399763371229184
   [junit4]   2> 1150429 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1150430 INFO  
(coreLoadExecutor-2493-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1150430 INFO  
(coreZkRegister-2487-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd 
x:collection1 s:shard2 c:tmp_collection r:core_node1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.ZkController Register replica - core:collection1 
address:http://127.0.0.1:49096/wjgbk/d collection:tmp_collection shard:shard2
   [junit4]   2> 1150431 INFO  
(coreZkRegister-2487-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd 
x:collection1 s:shard2 c:tmp_collection r:core_node1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.c.SolrZkClient makePath: 
/collections/tmp_collection/leader_elect/shard2/election
   [junit4]   2> 1150433 INFO  
(coreZkRegister-2487-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd 
x:collection1 s:shard2 c:tmp_collection r:core_node1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/tmp_collection/leader_elect/shard2/election/96212485109383172-core_node1-n_0000000000
   [junit4]   2> 1150434 INFO  
(coreZkRegister-2487-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd 
x:collection1 s:shard2 c:tmp_collection r:core_node1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process 
for shard=shard2 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1150435 INFO  
(coreZkRegister-2487-thread-1-processing-n:127.0.0.1:49096_wjgbk%2Fd 
x:collection1 s:shard2 c:tmp_collection r:core_node1) 
[n:127.0.0.1:49096_wjgbk%2Fd c:tmp_collection s:shard2 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to 
continue.
   [junit4]   2> 1150435 INFO  (coreZkRegister

[...truncated too long message...]

he.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
   [junit4]    >        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: 
Timeout occured while waiting response from server at: 
http://127.0.0.1:52518/nl_xy/gp
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
   [junit4]    >        ... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    >        at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    >        at 
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]    >        at 
java.net.SocketInputStream.read(SocketInputStream.java:170)
   [junit4]    >        at 
java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]    >        at 
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    >        at 
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    >        at 
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    >        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    >        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    >        at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    >        at 
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    >        at 
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    >        at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    >        at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    >        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    >        at 
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    >        at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    >        at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    >        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    >        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    >        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
   [junit4]    >        ... 11 more
   [junit4]   2> 5893658 INFO  
(SUITE-CollectionsAPIDistributedZkTest-seed#[D6671974DEEF3170]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_D6671974DEEF3170-001
   [junit4]   2> Jul 09, 2016 6:59:20 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{other_tl1=PostingsFormat(name=Asserting), 
range_facet_l_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), 
_version_=PostingsFormat(name=Asserting), 
multiDefault=PostingsFormat(name=Memory doPackFST= false), 
a_t=PostingsFormat(name=Memory doPackFST= false), 
intDefault=PostingsFormat(name=Asserting), a_i1=PostingsFormat(name=Memory 
doPackFST= false), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), 
range_facet_i_dv=PostingsFormat(name=Memory doPackFST= false), 
text=PostingsFormat(name=LuceneFixedGap), 
range_facet_l=PostingsFormat(name=Memory doPackFST= false), 
timestamp=PostingsFormat(name=Memory doPackFST= false)}, 
docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), 
range_facet_i_dv=DocValuesFormat(name=Lucene54), 
timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=1667, 
maxMBSortInHeap=6.377993961922069, sim=ClassicSimilarity, locale=hi-IN, 
timezone=Asia/Tokyo
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_74 (64-bit)/cpus=4,threads=1,free=136362504,total=499646464
   [junit4]   2> NOTE: All tests run in this JVM: [XmlUpdateRequestHandlerTest, 
TestFileDictionaryLookup, SuggesterTest, TestGeoJSONResponseWriter, 
TestSuggestSpellingConverter, TestCoreDiscovery, 
TestDistributedStatsComponentCardinality, ExitableDirectoryReaderTest, 
DocValuesMultiTest, SpatialHeatmapFacetsTest, RegexBoostProcessorTest, 
TestDefaultStatsCache, TestRawTransformer, TestRestoreCore, 
OverriddenZkACLAndCredentialsProvidersTest, AnalyticsQueryTest, 
TestSubQueryTransformerDistrib, SaslZkACLProviderTest, 
TestClassicSimilarityFactory, CollectionsAPISolrJTest, TestFastWriter, 
CircularListTest, TestFieldCacheSort, TestPostingsSolrHighlighter, 
QueryParsingTest, ClassificationUpdateProcessorFactoryTest, TestIndexSearcher, 
SolrCoreCheckLockOnStartupTest, TestMaxScoreQueryParser, 
TestSolrCloudWithKerberosAlt, TestImplicitCoreProperties, 
TestSolrCLIRunExample, TestUseDocValuesAsStored, TestConfigSetsAPIExclusivity, 
DeleteInactiveReplicaTest, TestUtils, TestGroupingSearch, 
SharedFSAutoReplicaFailoverUtilsTest, TestGraphTermsQParserPlugin, 
TestRecovery, TestOnReconnectListenerSupport, NoCacheHeaderTest, 
TriLevelCompositeIdRoutingTest, TestNumericTerms64, TestPerFieldSimilarity, 
SampleTest, TestJsonRequest, TestSchemaResource, MigrateRouteKeyTest, 
UpdateParamsTest, HdfsWriteToMultipleCollectionsTest, 
DistributedQueryComponentOptimizationTest, BinaryUpdateRequestHandlerTest, 
TestOrdValues, TestDynamicFieldResource, TestFieldTypeCollectionResource, 
HdfsRecoveryZkTest, TestCloudDeleteByQuery, 
StatelessScriptUpdateProcessorFactoryTest, TestHdfsCloudBackupRestore, 
TestTolerantUpdateProcessorCloud, TestExactStatsCache, 
BlockJoinFacetDistribTest, VersionInfoTest, GraphQueryTest, 
TestCSVResponseWriter, HardAutoCommitTest, SpatialRPTFieldTypeTest, 
DeleteShardTest, TestLocalFSCloudBackupRestore, SolrCloudExampleTest, 
LeaderFailoverAfterPartitionTest, BlockCacheTest, 
IgnoreCommitOptimizeUpdateProcessorFactoryTest, ClusterStateTest, 
AutoCommitTest, SimpleFacetsTest, TestReplicaProperties, TestManagedSchema, 
TestUniqueKeyFieldResource, RecoveryZkTest, TestCollationField, 
RecoveryAfterSoftCommitTest, DebugComponentTest, ShowFileRequestHandlerTest, 
TestRestManager, TestNamedUpdateProcessors, TestExtendedDismaxParser, 
TestSSLRandomization, CleanupOldIndexTest, TestTrackingShardHandlerFactory, 
DirectUpdateHandlerOptimizeTest, PluginInfoTest, TestCharFilters, 
RequestHandlersTest, TestChildDocTransformer, HdfsThreadLeakTest, 
TestJettySolrRunner, TestCloudPseudoReturnFields, BasicDistributedZkTest, 
BasicDistributedZk2Test, UnloadDistributedZkTest, 
CollectionsAPIDistributedZkTest]
   [junit4] Completed [621/621 (2!)] on J2 in 4064.84s, 1 test, 1 error <<< 
FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J2: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160709_172102_512.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid18841.hprof
 ...
   [junit4] Heap dump file created [685747688 bytes in 8.495 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 11 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:530:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984:
 There were test failures: 621 suites (10 ignored), 2642 tests, 2 suite-level 
errors, 1 error, 1303 ignored (17 assumptions) [seed: D6671974DEEF3170]

Total time: 311 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to