Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/806/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test

Error Message:
There are still nodes recoverying - waited for 30 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 30 
seconds
        at 
__randomizedtesting.SeedInfo.seed([D39ADD34EFA34F8F:5BCEE2EE415F2277]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:836)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1393)
        at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:74)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=212176, 
name=testExecutor-7119-thread-10, state=RUNNABLE, 
group=TGRP-HdfsUnloadDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=212176, name=testExecutor-7119-thread-10, 
state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:56452: Error CREATEing SolrCore 'multiunload9': 
Unable to create core [multiunload9] Caused by: Direct buffer memory
        at __randomizedtesting.SeedInfo.seed([D39ADD34EFA34F8F]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:550)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:47807/xivv/n: Could not load collection 
from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:47807/xivv/n: Could not load collection from 
ZK:halfcollectionblocker
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=1583, name=collection1, 
state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=1583, name=collection1, state=RUNNABLE, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:53223/a_: Could not find collection : 
awholynewstresscollection_collection1_0
        at __randomizedtesting.SeedInfo.seed([D39ADD34EFA34F8F]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 9878 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/init-core-data-001
   [junit4]   2> 0    INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 38   INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /a_/
   [junit4]   2> 1610 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your 
platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3026 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3497 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
org.mortbay.log.Slf4jLog
   [junit4]   2> 3507 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3709 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 3785 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_51550_hdfs____j228ta/webapp
   [junit4]   2> 4097 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 5503 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51550
   [junit4]   2> 7708 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 7731 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 7774 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_41349_datanode____2dlbya/webapp
   [junit4]   2> 8103 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 8906 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41349
   [junit4]   2> 9464 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 9466 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 9517 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_58551_datanode____.p1wtfv/webapp
   [junit4]   2> 9693 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 10410 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58551
   [junit4]   2> 11386 INFO  (IPC Server handler 8 on 53919) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-55e5395a-0ce6-4634-903a-e175bd310048 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=4e47f273-2de8-4274-8c07-6757c6ee20e9, infoPort=41349, 
ipcPort=55601, storageInfo=lv=-56;cid=testClusterID;nsid=399587516;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 2 msecs
   [junit4]   2> 11393 INFO  (IPC Server handler 3 on 53919) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-09ee5ec1-2f7f-41da-9ec6-01950d59ec1a node DatanodeRegistration(127.0.0.1, 
datanodeUuid=9daa3185-756c-4889-a89a-99340dfb59ad, infoPort=58551, 
ipcPort=33329, storageInfo=lv=-56;cid=testClusterID;nsid=399587516;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 8 msecs
   [junit4]   2> 11393 INFO  (IPC Server handler 8 on 53919) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-32678de0-56e0-4435-b8a1-9f04cd5ae3fd node DatanodeRegistration(127.0.0.1, 
datanodeUuid=4e47f273-2de8-4274-8c07-6757c6ee20e9, infoPort=41349, 
ipcPort=55601, storageInfo=lv=-56;cid=testClusterID;nsid=399587516;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 11405 INFO  (IPC Server handler 3 on 53919) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-dba6a977-055c-48c5-8b24-653ded3f323d node DatanodeRegistration(127.0.0.1, 
datanodeUuid=9daa3185-756c-4889-a89a-99340dfb59ad, infoPort=58551, 
ipcPort=33329, storageInfo=lv=-56;cid=testClusterID;nsid=399587516;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 1 msecs
   [junit4]   2> 12092 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 12101 INFO  (Thread-111) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 12101 INFO  (Thread-111) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 12201 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:52003
   [junit4]   2> 12229 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 12371 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12466 INFO  (zkCallback-1-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5ce7d9a4 
name:ZooKeeperConnection Watcher:127.0.0.1:52003 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12466 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12470 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12472 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 12521 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 12539 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12554 INFO  (zkCallback-2-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1e3e01a0 
name:ZooKeeperConnection Watcher:127.0.0.1:52003/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12554 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12555 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12562 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 12574 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 12581 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 12586 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 12592 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 12639 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 12657 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 12658 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 12661 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 12662 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 12666 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 12666 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 12670 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 12670 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 12674 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 12675 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 12679 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 12679 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 12683 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 12683 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 12686 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 12687 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 12699 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 12699 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 12703 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 12703 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 12708 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 12721 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12737 INFO  (zkCallback-3-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7953a79f 
name:ZooKeeperConnection Watcher:127.0.0.1:52003 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12738 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12738 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12738 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 12758 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 12765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 12777 INFO  (zkCallback-4-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@72ba1e7b 
name:ZooKeeperConnection Watcher:127.0.0.1:52003/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 12779 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 12779 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 12780 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf2/solrconfig.xml
   [junit4]   2> 12780 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 12793 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf2/schema.xml
   [junit4]   2> 12793 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 12797 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf2/enumsConfig.xml
   [junit4]   2> 12797 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 12800 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 12800 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 12807 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf2/stopwords.txt
   [junit4]   2> 12807 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 12811 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf2/protwords.txt
   [junit4]   2> 12812 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 12815 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf2/currency.xml
   [junit4]   2> 12815 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 12818 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 12818 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 12821 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 12822 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 12824 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf2/old_synonyms.txt
   [junit4]   2> 12825 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 12827 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf2/synonyms.txt
   [junit4]   2> 12828 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 12830 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml
 to /configs/conf2/elevate.xml
   [junit4]   2> 12831 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 13382 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores/collection1
   [junit4]   2> 13419 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.u.log Logging initialized @16846ms
   [junit4]   2> 13571 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 13712 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@41b5d11{/a_,null,AVAILABLE}
   [junit4]   2> 13723 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@4895ec7a{HTTP/1.1}{127.0.0.1:53223}
   [junit4]   2> 13724 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.s.Server Started @17151ms
   [junit4]   2> 13724 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:53919/hdfs__localhost_53919__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001_tempDir-002_control_data,
 hostContext=/a_, hostPort=53223, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores}
   [junit4]   2> 13732 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 13755 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/'
   [junit4]   2> 13933 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 13934 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 13940 INFO  (zkCallback-5-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@75ca9b72 
name:ZooKeeperConnection Watcher:127.0.0.1:52003/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 13940 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 13941 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 13945 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 13947 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/solr.xml
   [junit4]   2> 14178 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores
   [junit4]   2> 14250 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 835558640
   [junit4]   2> 14250 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/]
   [junit4]   2> 14250 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/lib
   [junit4]   2> 14251 WARN  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/lib).
   [junit4]   2> 14292 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 14488 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 14494 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 14558 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 14560 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52003/solr
   [junit4]   2> 14577 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 14577 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 14579 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 14588 INFO  (zkCallback-7-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@14201367 
name:ZooKeeperConnection Watcher:127.0.0.1:52003 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 14592 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 14593 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 14633 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 14653 INFO  
(zkCallback-8-thread-1-processing-n:127.0.0.1:53223_a_) [n:127.0.0.1:53223_a_   
 ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6b60bfde 
name:ZooKeeperConnection Watcher:127.0.0.1:52003/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 14653 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 14666 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 14676 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 14682 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 14687 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 14690 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 14702 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 14707 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 14709 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 14711 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 14713 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:53223_a_
   [junit4]   2> 14716 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:53223_a_
   [junit4]   2> 14724 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 14727 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 14743 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 14747 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/94591931139031046-127.0.0.1:53223_a_-n_0000000000
   [junit4]   2> 14758 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:53223_a_
   [junit4]   2> 14758 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 14762 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer Overseer 
(id=94591931139031046-127.0.0.1:53223_a_-n_0000000000) starting
   [junit4]   2> 14781 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 14826 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 14840 INFO  
(OverseerCollectionConfigSetProcessor-94591931139031046-127.0.0.1:53223_a_-n_0000000000)
 [n:127.0.0.1:53223_a_    ] o.a.s.c.OverseerTaskProcessor Process current queue 
of overseer operations
   [junit4]   2> 14853 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 14868 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 14986 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 14986 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 15021 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.CoresLocator Looking for core definitions 
underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores
   [junit4]   2> 15070 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 15071 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.CoresLocator Found core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores/collection1/
   [junit4]   2> 15081 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 15106 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 15106 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) 
[n:127.0.0.1:53223_a_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 15107 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 15108 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 15117 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53223/a_";,
   [junit4]   2>   "node_name":"127.0.0.1:53223_a_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 15124 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 15128 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53223/a_";,
   [junit4]   2>   "node_name":"127.0.0.1:53223_a_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 15130 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 15140 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 15145 INFO  
(zkCallback-8-thread-1-processing-n:127.0.0.1:53223_a_) [n:127.0.0.1:53223_a_   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 16126 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 16126 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 16128 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 16128 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 16131 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 16133 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores/collection1/'
   [junit4]   2> 16225 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.c.Config 
loaded config solrconfig.xml with version 0 
   [junit4]   2> 16299 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
current version of requestparams : -1
   [junit4]   2> 16332 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 16496 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.c.Config 
Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 16515 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 16669 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 17735 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Initialized with 
rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 17805 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
default search field in schema is text
   [junit4]   2> 17806 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
unique key field: id
   [junit4]   2> 18075 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 18091 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 18096 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 18098 WARN  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 18099 WARN  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 18099 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 18110 WARN  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 18110 WARN  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 18114 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 18116 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 18121 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 18150 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:53919/solr_hdfs_home
   [junit4]   2> 18150 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 18170 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 18176 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17f8a76a
   [junit4]   2> 18227 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 18476 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsLocalityReporter Registering direcotry 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data for 
locality metrics.
   [junit4]   2> 18480 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 18489 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 18600 WARN  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:53919/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 18650 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 18681 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsLocalityReporter Registering direcotry 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data/index 
for locality metrics.
   [junit4]   2> 18681 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 18683 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.12109375, 
noCFSRatio=0.17127012241870715]
   [junit4]   2> 19491 INFO  (IPC Server handler 9 on 53919) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47593 is 
added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-55e5395a-0ce6-4634-903a-e175bd310048:NORMAL:127.0.0.1:47593|RBW],
 
ReplicaUnderConstruction[[DISK]DS-dba6a977-055c-48c5-8b24-653ded3f323d:NORMAL:127.0.0.1:43766|RBW]]}
 size 71
   [junit4]   2> 19493 INFO  (IPC Server handler 0 on 53919) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43766 is 
added to blk_1073741825_1001 size 71
   [junit4]   2> 19917 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        commit{dir=NRTCachingDirectory(HdfsDirectory@b86cd4f9 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d7136bf; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 19918 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 19926 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:53919/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 20257 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"nodistrib"
   [junit4]   2> 20302 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"dedupe"
   [junit4]   2> 20313 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 20313 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"stored_sig"
   [junit4]   2> 20314 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 20314 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-explicit"
   [junit4]   2> 20495 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 20495 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 20495 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating 
implicit default
   [junit4]   2> 20572 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 20580 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 20582 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 20584 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 20645 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 20646 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 20674 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 20675 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 20698 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 20698 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 20702 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=7, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=83.923828125, 
floorSegmentMB=1.580078125, forceMergeDeletesPctAllowed=3.0388882378422535, 
segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.8817959224396658
   [junit4]   2> 20825 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        commit{dir=NRTCachingDirectory(HdfsDirectory@b86cd4f9 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d7136bf; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 20825 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 20869 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening Searcher@4dafe08d[collection1] main
   [junit4]   2> 20894 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 20900 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 20901 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the 
RestManager with znodeBase: /configs/conf1
   [junit4]   2> 20905 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 20905 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 20948 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 20953 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 20953 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 20953 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 21005 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 21012 INFO  
(searcherExecutor-8-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@4dafe08d[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 21031 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 21033 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version 
buckets from index
   [junit4]   2> 21035 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket 
highest value from index
   [junit4]   2> 21037 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1513470907530608640
   [junit4]   2> 21039 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Took 6.0ms to seed version buckets with highest version 
1513470907530608640
   [junit4]   2> 21041 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 21043 INFO  
(coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:53223_a_) 
[n:127.0.0.1:53223_a_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 21050 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:http://127.0.0.1:53223/a_ 
collection:control_collection shard:shard1
   [junit4]   2> 21059 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 21068 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/94591931139031046-core_node1-n_0000000000
   [junit4]   2> 21070 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 21073 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 21079 INFO  
(zkCallback-8-thread-1-processing-n:127.0.0.1:53223_a_) [n:127.0.0.1:53223_a_   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 21092 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 21092 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 21092 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:53223/a_/collection1/
   [junit4]   2> 21093 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 21093 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:53223/a_/collection1/ has no replicas
   [junit4]   2> 21093 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:53223/a_/collection1/ shard1
   [junit4]   2> 21094 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 21106 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 21126 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53223/a_";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 21147 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 21155 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 21159 INFO  (zkCallback-10-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1c3e89b9 
name:ZooKeeperConnection Watcher:127.0.0.1:52003/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 21159 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 21159 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 21159 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 21181 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 21233 INFO  (zkCallback-10-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 21234 INFO  
(zkCallback-8-thread-1-processing-n:127.0.0.1:53223_a_) [n:127.0.0.1:53223_a_   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 21277 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are http://127.0.0.1:53223/a_/collection1/ and leader is 
http://127.0.0.1:53223/a_/collection1/
   [junit4]   2> 21279 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53223/a_
   [junit4]   2> 21279 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 21279 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 21279 INFO  
(coreZkRegister-1-thread-1-processing-n:127.0.0.1:53223_a_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53223_a_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 21282 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53223/a_";,
   [junit4]   2>   "node_name":"127.0.0.1:53223_a_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 21285 INFO  
(OverseerStateUpdate-94591931139031046-127.0.0.1:53223_a_-n_0000000000) 
[n:127.0.0.1:53223_a_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53223/a_";,
   [junit4]   2>   "node_name":"127.0.0.1:53223_a_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 21465 INFO  (zkCallback-10-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 21465 INFO  
(zkCallback-8-thread-1-processing-n:127.0.0.1:53223_a_) [n:127.0.0.1:53223_a_   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 21755 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/shard-1-001/cores/collection1
   [junit4]   2> 21756 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_D39ADD34EFA34F8F-001/shard-1-001
   [junit4]   2> 21759 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 21778 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[D39ADD34EFA34F8F]) [    ] 
o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7a

[...truncated too long message...]

 org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.teardownClass(HdfsUnloadDistributedZkTest.java:47)
   [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2>        ... 53 more
   [junit4]   2> 4599486 INFO  
(SUITE-HdfsUnloadDistributedZkTest-seed#[D39ADD34EFA34F8F]-worker) 
[n:127.0.0.1:41390_ c:unloadcollection s:shard1 r:core_node4 
x:unloadcollection4] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 27, 2015 10:08:32 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, 
sim=ClassicSimilarity, locale=es_UY, timezone=Asia/Bahrain
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=4,threads=2,free=373624360,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: 
[HdfsCollectionsAPIDistributedZkTest, TestFieldSortValues, 
TestManagedSynonymFilterFactory, ParsingFieldUpdateProcessorsTest, 
SuggesterTSTTest, TestSolrQueryParser, DirectUpdateHandlerTest, TestSort, 
SuggesterWFSTTest, LeaderElectionTest, TestHighFrequencyDictionaryFactory, 
MultiThreadedOCPTest, TestFastWriter, TestSolrDynamicMBean, 
TestSolrIndexConfig, SuggesterTest, HdfsDirectoryFactoryTest, TestFastLRUCache, 
AnalyticsQueryTest, TestLeaderInitiatedRecoveryThread, 
AsyncMigrateRouteKeyTest, FullSolrCloudDistribCmdsTest, TestCloudManagedSchema, 
DebugComponentTest, TestXIncludeConfig, HdfsBasicDistributedZkTest, 
TestSystemIdResolver, CursorMarkTest, TestSolrCLIRunExample, 
CloudMLTQParserTest, RecoveryAfterSoftCommitTest, SearchHandlerTest, 
QueryResultKeyTest, TestSearchPerf, TestHashQParserPlugin, 
TestImplicitCoreProperties, TestConfigSetProperties, 
BinaryUpdateRequestHandlerTest, TestGroupingSearch, 
TestFieldTypeCollectionResource, CloudExitableDirectoryReaderTest, 
DistributedFacetPivotSmallTest, HdfsSyncSliceTest, 
FieldMutatingUpdateProcessorTest, UnloadDistributedZkTest, 
BasicAuthIntegrationTest, TestManagedSchemaDynamicFieldResource, 
TestConfigReload, SecurityConfHandlerTest, 
StatelessScriptUpdateProcessorFactoryTest, TestZkChroot, UUIDFieldTest, 
QueryParsingTest, TestDocBasedVersionConstraints, TestUniqueKeyFieldResource, 
AssignTest, SegmentsInfoRequestHandlerTest, SolrXmlInZkTest, 
TestExtendedDismaxParser, RequestHandlersTest, 
OverseerCollectionConfigSetProcessorTest, NoCacheHeaderTest, 
TestSolrDeletionPolicy1, TestAuthorizationFramework, 
TestFuzzyAnalyzedSuggestions, TestWriterPerf, TestOrdValues, 
SharedFSAutoReplicaFailoverUtilsTest, DataDrivenBlockJoinTest, 
TestExpandComponent, HighlighterConfigTest, CdcrUpdateLogTest, 
LukeRequestHandlerTest, CollectionReloadTest, TestManagedResourceStorage, 
TestFieldResource, TestAddFieldRealTimeGet, TestBulkSchemaConcurrent, 
ResponseHeaderTest, TestDistributedStatsComponentCardinality, 
BasicDistributedZk2Test, TestStressReorder, TestSmileRequest, 
TestSearcherReuse, SimpleFacetsTest, TestPerFieldSimilarity, 
SolrCoreCheckLockOnStartupTest, TestTrieFacet, StatsComponentTest, 
ClusterStateUpdateTest, ZkSolrClientTest, AutoCommitTest, 
TestDistributedGrouping, TestRecovery, TestHashPartitioner, 
DistributedSpellCheckComponentTest, ZkControllerTest, TestJoin, TestReload, 
DistributedTermsComponentTest, PeerSyncTest, ConvertedLegacyTest, 
TestBadConfig, TestIndexSearcher, ShowFileRequestHandlerTest, 
DistributedQueryElevationComponentTest, SimplePostToolTest, TestCoreDiscovery, 
SolrRequestParserTest, TestTrie, SpatialFilterTest, 
WordBreakSolrSpellCheckerTest, DirectUpdateHandlerOptimizeTest, 
SortByFunctionTest, DocValuesMultiTest, DisMaxRequestHandlerTest, 
TestReversedWildcardFilterFactory, TestQueryTypes, XmlUpdateRequestHandlerTest, 
TestValueSourceCache, TermVectorComponentTest, RequiredFieldsTest, 
TestArbitraryIndexDir, IndexSchemaRuntimeFieldTest, IndexSchemaTest, 
TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, 
PingRequestHandlerTest, UpdateParamsTest, TestQuerySenderNoQuery, 
CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, 
TestStressRecovery, TestMergePolicyConfig, TestSolrDeletionPolicy2, SampleTest, 
TestDocSet, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, 
OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, 
TestPostingsSolrHighlighter, DirectSolrConnectionTest, 
NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, 
TestCharFilters, TestCodecSupport, EchoParamsTest, 
TestSweetSpotSimilarityFactory, TestDefaultSimilarityFactory, 
TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, 
OpenExchangeRatesOrgProviderTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, 
TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, 
TestTolerantSearch, AliasIntegrationTest, CollectionTooManyReplicasTest, 
DistribCursorPagingTest, OverriddenZkACLAndCredentialsProvidersTest, 
OverseerRolesTest, SharedFSAutoReplicaFailoverTest, TestCollectionAPI, 
TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestDownShardTolerantSearch, 
TestLeaderElectionZkExpiry, TestMiniSolrCloudClusterBase, 
TestReplicaProperties, TestSolrCloudWithKerberosAlt, HdfsThreadLeakTest, 
HdfsUnloadDistributedZkTest]
   [junit4] Completed [448/546] on J2 in 104.73s, 1 test, 1 error <<< FAILURES!

[...truncated 322 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:514:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
 There were test failures: 546 suites (8 ignored), 2145 tests, 3 errors, 1 
failure, 29 ignored (17 assumptions) [seed: D39ADD34EFA34F8F]

Total time: 195 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to