Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/799/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:
Server refused connection at: http://127.0.0.1:36470/ef_deq/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Server refused connection at: 
http://127.0.0.1:36470/ef_deq/collection1
        at 
__randomizedtesting.SeedInfo.seed([E3BD047C4F547756:6BE93BA6E1A81AAE]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:583)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
        at 
org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:366)
        at 
org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
        at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at 
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
        at 
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
        at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
        at 
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
        at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
        at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
        ... 48 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=2799, name=collection4, 
state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=2799, name=collection4, state=RUNNABLE, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:38212/lf/s: Could not find collection : 
awholynewstresscollection_collection4_0
        at __randomizedtesting.SeedInfo.seed([E3BD047C4F547756]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 9851 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/init-core-data-001
   [junit4]   2> 415781 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: 
/ef_deq/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 415886 WARN  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 415921 WARN  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 415923 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 415936 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_43562_hdfs____.yurqmc/webapp
   [junit4]   2> 416275 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 417001 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43562
   [junit4]   2> 417275 WARN  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 417276 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 417300 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_43193_datanode____.617ap9/webapp
   [junit4]   2> 417457 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 417839 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43193
   [junit4]   2> 417948 WARN  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 417950 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 417976 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_41549_datanode____.isel6k/webapp
   [junit4]   2> 418090 INFO  (IPC Server handler 0 on 57930) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-c4010aae-5732-4d51-9f34-b9c55cd7e765 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=bdc0c8e4-9850-453a-81ff-13cbb38df544, infoPort=43193, 
ipcPort=48950, storageInfo=lv=-56;cid=testClusterID;nsid=1355030368;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 6 msecs
   [junit4]   2> 418091 INFO  (IPC Server handler 0 on 57930) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-2c343400-8c25-4ed6-b18d-32f894a75b5c node DatanodeRegistration(127.0.0.1, 
datanodeUuid=bdc0c8e4-9850-453a-81ff-13cbb38df544, infoPort=43193, 
ipcPort=48950, storageInfo=lv=-56;cid=testClusterID;nsid=1355030368;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 418191 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 418884 INFO  
(SUITE-HdfsBasicDistributedZk2Test-seed#[E3BD047C4F547756]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41549
   [junit4]   2> 419094 INFO  (IPC Server handler 6 on 57930) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-865ae032-ecb7-46de-b3c2-53cfb995547e node DatanodeRegistration(127.0.0.1, 
datanodeUuid=ed515bb2-9833-41df-9e6a-c37016955a6e, infoPort=41549, 
ipcPort=34239, storageInfo=lv=-56;cid=testClusterID;nsid=1355030368;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 419095 INFO  (IPC Server handler 6 on 57930) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-21581e0c-84f6-4c2c-ba6c-217e1972f2ef node DatanodeRegistration(127.0.0.1, 
datanodeUuid=ed515bb2-9833-41df-9e6a-c37016955a6e, infoPort=41549, 
ipcPort=34239, storageInfo=lv=-56;cid=testClusterID;nsid=1355030368;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 419236 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 419236 INFO  (Thread-899) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 419237 INFO  (Thread-899) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 419336 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:47992
   [junit4]   2> 419337 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 419337 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 419340 INFO  (zkCallback-233-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@236b6d61 
name:ZooKeeperConnection Watcher:127.0.0.1:47992 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 419344 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 419345 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 419345 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 419348 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 419350 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 419357 INFO  (zkCallback-234-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@395c6a6f 
name:ZooKeeperConnection Watcher:127.0.0.1:47992/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 419358 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 419358 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 419358 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 419360 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 419361 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 419362 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 419364 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 419364 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 419366 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 419366 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 419368 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 419368 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 419370 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 419370 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 419371 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 419372 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 419373 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 419373 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 419374 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 419375 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 419376 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 419377 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 419379 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 419379 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 419381 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 419381 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 419382 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 419382 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 419735 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores/collection1
   [junit4]   2> 419737 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 419739 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@2a218db{/ef_deq,null,AVAILABLE}
   [junit4]   2> 419740 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@6def54c1{HTTP/1.1}{127.0.0.1:38948}
   [junit4]   2> 419740 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.Server Started @423204ms
   [junit4]   2> 419740 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:57930/hdfs__localhost_57930__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001_tempDir-002_control_data,
 hostContext=/ef_deq, hostPort=38948, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores}
   [junit4]   2> 419741 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 419741 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/'
   [junit4]   2> 419756 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 419762 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 419763 INFO  (zkCallback-235-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3a7f062a 
name:ZooKeeperConnection Watcher:127.0.0.1:47992/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 419763 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 419763 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 419769 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 419769 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/solr.xml
   [junit4]   2> 419774 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores
   [junit4]   2> 419774 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 1523604709
   [junit4]   2> 419774 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/]
   [junit4]   2> 419775 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/lib
   [junit4]   2> 419775 WARN  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/lib).
   [junit4]   2> 419783 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 419785 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 419785 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 419785 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 419786 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47992/solr
   [junit4]   2> 419786 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 419786 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 419787 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 419794 INFO  (zkCallback-237-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@22d89cb8 
name:ZooKeeperConnection Watcher:127.0.0.1:47992 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 419795 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 419796 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 419798 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ConnectionManager Waiting for client 
to connect to ZooKeeper
   [junit4]   2> 419800 INFO  
(zkCallback-238-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7b6a0ebf 
name:ZooKeeperConnection Watcher:127.0.0.1:47992/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 419801 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ConnectionManager Client is connected 
to ZooKeeper
   [junit4]   2> 419802 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 419805 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 419808 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 419811 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 419813 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 419816 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 419817 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 419819 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/clusterstate.json
   [junit4]   2> 419821 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 419822 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:38948_ef_deq
   [junit4]   2> 419822 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:38948_ef_deq
   [junit4]   2> 419824 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 419831 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 419832 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 419833 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000
   [junit4]   2> 419834 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:38948_ef_deq
   [junit4]   2> 419834 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 419837 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer Overseer 
(id=94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) starting
   [junit4]   2> 419840 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue-work
   [junit4]   2> 419848 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.OverseerAutoReplicaFailoverThread 
Starting OverseerAutoReplicaFailoverThread 
autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 419849 INFO  
(OverseerCollectionConfigSetProcessor-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000)
 [n:127.0.0.1:38948_ef_deq    ] o.a.s.c.OverseerTaskProcessor Process current 
queue of overseer operations
   [junit4]   2> 419850 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer Starting to work on the main 
queue
   [junit4]   2> 419851 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ZkStateReader Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 419876 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.CoreContainer Security conf doesn't 
exist. Skipping setup for authorization module.
   [junit4]   2> 419876 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.CoreContainer No authentication plugin 
used.
   [junit4]   2> 419877 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.CoresLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores
   [junit4]   2> 419877 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 419877 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.CoresLocator Found core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores/collection1/
   [junit4]   2> 419878 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 419879 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 419880 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 419882 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 419882 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 419882 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38948/ef_deq";,
   [junit4]   2>   "node_name":"127.0.0.1:38948_ef_deq",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 419883 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 419883 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38948/ef_deq";,
   [junit4]   2>   "node_name":"127.0.0.1:38948_ef_deq",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 419883 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: control_collection
   [junit4]   2> 419885 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 419887 INFO  
(zkCallback-238-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ZkStateReader A cluster state change: 
WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, 
has occurred - updating... (live nodes size: 1)
   [junit4]   2> 420883 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 420884 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 420884 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 420884 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 420885 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 420885 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores/collection1/'
   [junit4]   2> 420901 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] o.a.s.c.Config 
loaded config solrconfig.xml with version 0 
   [junit4]   2> 420912 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 420919 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 420932 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] o.a.s.c.Config 
Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 420934 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 420946 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 421099 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Initialized with 
rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 421107 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 421108 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 421119 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 421132 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 421135 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 421135 WARN  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 421136 WARN  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 421136 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 421137 WARN  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 421137 WARN  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 421138 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 421138 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 421139 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 421139 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:57930/solr_hdfs_home
   [junit4]   2> 421139 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication 
disabled
   [junit4]   2> 421139 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 421139 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@301ca3b4
   [junit4]   2> 421140 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 421199 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data for 
locality metrics.
   [junit4]   2> 421199 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 421201 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 421217 WARN  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:57930/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 421217 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 421240 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 421240 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 421241 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS 
BlockCache
   [junit4]   2> 421271 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 421271 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.h.HdfsLocalityReporter Registering direcotry 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data/index 
for locality metrics.
   [junit4]   2> 421271 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 421271 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=31, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=37.9775390625, 
floorSegmentMB=0.49609375, forceMergeDeletesPctAllowed=21.14187365088694, 
segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.29227217884130485
   [junit4]   2> 421314 INFO  (IPC Server handler 5 on 57930) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47501 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-865ae032-ecb7-46de-b3c2-53cfb995547e:NORMAL:127.0.0.1:44115|RBW],
 
ReplicaUnderConstruction[[DISK]DS-2c343400-8c25-4ed6-b18d-32f894a75b5c:NORMAL:127.0.0.1:47501|FINALIZED]]}
 size 0
   [junit4]   2> 421319 INFO  (IPC Server handler 6 on 57930) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44115 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-2c343400-8c25-4ed6-b18d-32f894a75b5c:NORMAL:127.0.0.1:47501|FINALIZED],
 
ReplicaUnderConstruction[[DISK]DS-21581e0c-84f6-4c2c-ba6c-217e1972f2ef:NORMAL:127.0.0.1:44115|FINALIZED]]}
 size 0
   [junit4]   2> 421335 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b8e41dd8 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3ffd3a0d); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 421335 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 421337 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:57930/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 421341 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 421341 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 421342 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 421342 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 421342 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 421342 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 421342 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 421342 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 421343 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as 
default, creating implicit default
   [junit4]   2> 421345 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 421346 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 421347 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 421359 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 421362 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 421363 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 421368 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 421369 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
tlogDfsReplication=2
   [junit4]   2> 421383 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 421384 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 421387 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.5959589124048754]
   [junit4]   2> 421408 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b8e41dd8 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3ffd3a0d); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 421409 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 421409 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@24dbc202[collection1] 
main
   [junit4]   2> 421409 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 421410 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection 
configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 421411 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based 
storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 421411 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 421411 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 421411 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 421412 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 421412 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 421412 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 421413 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 421413 INFO  
(searcherExecutor-686-thread-1-processing-n:127.0.0.1:38948_ef_deq 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@24dbc202[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 421418 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed 
version buckets
   [junit4]   2> 421418 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 
256 version buckets from index
   [junit4]   2> 421418 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed 
version bucket highest value from index
   [junit4]   2> 421419 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent 
updates, using new clock 1512851411837124608
   [junit4]   2> 421419 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with 
highest version 1512851411837124608
   [junit4]   2> 421419 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 421419 INFO  
(coreLoadExecutor-685-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 421420 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:http://127.0.0.1:38948/ef_deq 
collection:control_collection shard:shard1
   [junit4]   2> 421421 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 421424 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/94553213096886276-core_node1-n_0000000000
   [junit4]   2> 421425 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 421427 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 421428 INFO  
(zkCallback-238-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ZkStateReader A cluster state change: 
WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, 
has occurred - updating... (live nodes size: 1)
   [junit4]   2> 421432 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 421432 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 421433 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:38948/ef_deq/collection1/
   [junit4]   2> 421433 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 421433 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:38948/ef_deq/collection1/ has no replicas
   [junit4]   2> 421433 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:38948/ef_deq/collection1/ shard1
   [junit4]   2> 421436 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 421442 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 421448 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:38948/ef_deq";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 421552 INFO  
(zkCallback-238-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ZkStateReader A cluster state change: 
WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, 
has occurred - updating... (live nodes size: 1)
   [junit4]   2> 421599 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are http://127.0.0.1:38948/ef_deq/collection1/ and leader is 
http://127.0.0.1:38948/ef_deq/collection1/
   [junit4]   2> 421599 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38948/ef_deq
   [junit4]   2> 421599 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 421599 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 421599 INFO  
(coreZkRegister-679-thread-1-processing-n:127.0.0.1:38948_ef_deq x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:38948_ef_deq 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 421601 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38948/ef_deq";,
   [junit4]   2>   "node_name":"127.0.0.1:38948_ef_deq",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 421602 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:38948/ef_deq";,
   [junit4]   2>   "node_name":"127.0.0.1:38948_ef_deq",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 421704 INFO  
(zkCallback-238-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ZkStateReader A cluster state change: 
WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, 
has occurred - updating... (live nodes size: 1)
   [junit4]   2> 421884 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 421885 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 421889 INFO  (zkCallback-240-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@46c15254 
name:ZooKeeperConnection Watcher:127.0.0.1:47992/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 421889 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 421889 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 421889 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 421895 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 422240 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores/collection1
   [junit4]   2> 422241 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001
   [junit4]   2> 422242 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 422245 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4269310b{/ef_deq,null,AVAILABLE}
   [junit4]   2> 422245 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@5ed14052{HTTP/1.1}{127.0.0.1:35903}
   [junit4]   2> 422245 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.e.j.s.Server Started @425710ms
   [junit4]   2> 422246 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:57930/hdfs__localhost_57930__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/ef_deq, hostPort=35903, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores}
   [junit4]   2> 422246 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 422246 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/'
   [junit4]   2> 422265 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 422267 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 422270 INFO  (zkCallback-241-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@17116deb 
name:ZooKeeperConnection Watcher:127.0.0.1:47992/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 422271 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 422271 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 422273 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 422273 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/solr.xml
   [junit4]   2> 422278 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores
   [junit4]   2> 422278 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 1005358643
   [junit4]   2> 422279 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/]
   [junit4]   2> 422279 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/lib
   [junit4]   2> 422279 WARN  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/lib).
   [junit4]   2> 422287 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 422288 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 422289 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 422289 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 422289 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47992/solr
   [junit4]   2> 422289 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 422289 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 422290 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 422291 INFO  (zkCallback-243-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6d080bfb 
name:ZooKeeperConnection Watcher:127.0.0.1:47992 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 422292 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 422293 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 422294 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.c.ConnectionManager Waiting for client 
to connect to ZooKeeper
   [junit4]   2> 422297 INFO  
(zkCallback-244-thread-1-processing-n:127.0.0.1:35903_ef_deq) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@69161b15 
name:ZooKeeperConnection Watcher:127.0.0.1:47992/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 422297 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.c.ConnectionManager Client is connected 
to ZooKeeper
   [junit4]   2> 422305 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.c.ZkStateReader Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 423312 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:35903_ef_deq
   [junit4]   2> 423313 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:35903_ef_deq
   [junit4]   2> 423314 INFO  
(zkCallback-238-thread-1-processing-n:127.0.0.1:38948_ef_deq) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.c.ZkStateReader A live node change: 
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has 
occurred - updating... (live nodes size: 1)
   [junit4]   2> 423314 INFO  (zkCallback-240-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live 
nodes size: 1)
   [junit4]   2> 423315 INFO  
(zkCallback-244-thread-1-processing-n:127.0.0.1:35903_ef_deq) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.c.ZkStateReader A live node change: 
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has 
occurred - updating... (live nodes size: 1)
   [junit4]   2> 423317 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 423317 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/94553213096886280-127.0.0.1:35903_ef_deq-n_0000000001
   [junit4]   2> 423318 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000 
to know if I could be the leader
   [junit4]   2> 423326 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.CoreContainer Security conf doesn't 
exist. Skipping setup for authorization module.
   [junit4]   2> 423326 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.CoreContainer No authentication plugin 
used.
   [junit4]   2> 423329 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.CoresLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores
   [junit4]   2> 423330 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores/collection1,
 collection=collection1, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 423331 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.CoresLocator Found core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_E3BD047C4F547756-001/shard-1-001/cores/collection1/
   [junit4]   2> 423331 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 423334 INFO  
(coreLoadExecutor-696-thread-1-processing-n:127.0.0.1:35903_ef_deq) 
[n:127.0.0.1:35903_ef_deq c:collection1   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 423335 INFO  
(coreLoadExecutor-696-thread-1-processing-n:127.0.0.1:35903_ef_deq) 
[n:127.0.0.1:35903_ef_deq c:collection1   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 423336 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 423336 INFO  
(TEST-HdfsBasicDistributedZk2Test.test-seed#[E3BD047C4F547756]) 
[n:127.0.0.1:35903_ef_deq    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 423338 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:35903/ef_deq";,
   [junit4]   2>   "node_name":"127.0.0.1:35903_ef_deq",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 423338 INFO  
(coreLoadExecutor-696-thread-1-processing-n:127.0.0.1:35903_ef_deq) 
[n:127.0.0.1:35903_ef_deq c:collection1   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 423339 INFO  
(OverseerStateUpdate-94553213096886276-127.0.0.1:38948_ef_deq-n_0000000000) 
[n:127.0.0.1:38948_ef_deq    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=2 message={
   [junit

[...truncated too long message...]

:34374
   [junit4]   2> 2200227 WARN  
(org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@22733ddd)
 [    ] o.a.h.h.s.b.DecommissionManager Monitor interrupted: 
java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 2200234 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[E3BD047C4F547756]-worker) 
[n:127.0.0.1:38212_lf%2Fs c:addReplicaColl s:shard1 r:core_node7 
x:propertyDotName] o.m.log Stopped 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 2200337 ERROR 
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[E3BD047C4F547756]-worker) 
[n:127.0.0.1:38212_lf%2Fs c:addReplicaColl s:shard1 r:core_node7 
x:propertyDotName] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2>        at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown 
Source)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2>        at 
com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2>        ... 53 more
   [junit4]   2> 2200341 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[E3BD047C4F547756]-worker) 
[n:127.0.0.1:38212_lf%2Fs c:addReplicaColl s:shard1 r:core_node7 
x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Sep 20, 2015 1:15:15 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: 
codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION,
 chunkSize=24886, maxDocsPerChunk=385, blockSize=3), 
termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION,
 chunkSize=24886, blockSize=3)), 
sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=bg_BG, 
timezone=SystemV/AST4
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=4,threads=2,free=158135520,total=533725184
   [junit4]   2> NOTE: All tests run in this JVM: [TestZkChroot, 
TestManagedStopFilterFactory, OverseerTaskQueueTest, TestExtendedDismaxParser, 
TestReRankQParserPlugin, TestDefaultSearchFieldResource, SearchHandlerTest, 
HdfsDirectoryTest, SharedFSAutoReplicaFailoverTest, AddBlockUpdateTest, 
UUIDFieldTest, SuggesterTSTTest, SegmentsInfoRequestHandlerTest, 
TestHighFrequencyDictionaryFactory, BasicDistributedZk2Test, 
HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [185/546] on J2 in 1849.26s, 1 test, 1 error <<< FAILURES!

[...truncated 1147 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:514:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
 There were test failures: 546 suites, 2145 tests, 2 errors, 29 ignored (17 
assumptions)

Total time: 187 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to