Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.6/27/

1 tests failed.
FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at 
__randomizedtesting.SeedInfo.seed([176FDA4AD2D42496:9F3BE5907C28496E]:0)
        at java.util.Arrays.copyOfRange(Arrays.java:3664)
        at java.lang.String.<init>(String.java:207)
        at java.lang.StringBuilder.toString(StringBuilder.java:407)
        at 
org.apache.http.client.utils.URLEncodedUtils.urlEncode(URLEncodedUtils.java:547)
        at 
org.apache.http.client.utils.URLEncodedUtils.encodeFormFields(URLEncodedUtils.java:652)
        at 
org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:404)
        at 
org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:382)
        at 
org.apache.http.client.entity.UrlEncodedFormEntity.<init>(UrlEncodedFormEntity.java:75)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.fillContentStream(HttpSolrClient.java:513)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:420)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1015)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1031)
        at 
org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:228)
        at 
org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:167)
        at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:669)
        at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:153)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)




Build Log:
[...truncated 14974 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/init-core-data-001
   [junit4]   2> 1937617 INFO  
(SUITE-FullSolrCloudDistribCmdsTest-seed#[176FDA4AD2D42496]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1937619 INFO  
(SUITE-FullSolrCloudDistribCmdsTest-seed#[176FDA4AD2D42496]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1937619 INFO  
(SUITE-FullSolrCloudDistribCmdsTest-seed#[176FDA4AD2D42496]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1937620 INFO  
(SUITE-FullSolrCloudDistribCmdsTest-seed#[176FDA4AD2D42496]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1937624 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1937624 INFO  (Thread-4463) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1937624 INFO  (Thread-4463) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1937626 ERROR (Thread-4463) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1937724 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:33676
   [junit4]   2> 1937729 INFO  (zkConnectionManagerCallback-4390-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1937735 INFO  (zkConnectionManagerCallback-4392-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1937743 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1937745 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1937747 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1937749 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1937750 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1937752 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1937754 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1937755 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1937757 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1937758 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1937760 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1937762 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly 
asked otherwise
   [junit4]   2> 1937940 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: 
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
   [junit4]   2> 1937942 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1937942 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1937942 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1937942 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@510531d8{/,null,AVAILABLE}
   [junit4]   2> 1937943 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@19ef666e{HTTP/1.1,[http/1.1]}{127.0.0.1:33027}
   [junit4]   2> 1937943 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server Started @1938024ms
   [junit4]   2> 1937943 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/tempDir-001/control/data,
 hostContext=/, hostPort=33027, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/control-001/cores}
   [junit4]   2> 1937943 ERROR 
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1937943 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1937944 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.6.0
   [junit4]   2> 1937944 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1937944 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1937944 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-16T13:14:43.446Z
   [junit4]   2> 1937946 INFO  (zkConnectionManagerCallback-4394-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1937947 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1937947 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/control-001/solr.xml
   [junit4]   2> 1937952 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1937952 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1937954 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1938334 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33676/solr
   [junit4]   2> 1938336 INFO  (zkConnectionManagerCallback-4398-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1938338 INFO  (zkConnectionManagerCallback-4400-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1938447 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:33027_
   [junit4]   2> 1938448 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.c.Overseer Overseer 
(id=72457420044828676-127.0.0.1:33027_-n_0000000000) starting
   [junit4]   2> 1938456 INFO  (zkConnectionManagerCallback-4407-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1938459 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 
127.0.0.1:33676/solr ready
   [junit4]   2> 1938460 INFO  
(OverseerStateUpdate-72457420044828676-127.0.0.1:33027_-n_0000000000) 
[n:127.0.0.1:33027_    ] o.a.s.c.Overseer Starting to work on the main queue : 
127.0.0.1:33027_
   [junit4]   2> 1938460 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33027_
   [junit4]   2> 1938462 INFO  
(OverseerStateUpdate-72457420044828676-127.0.0.1:33027_-n_0000000000) 
[n:127.0.0.1:33027_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 1938470 INFO  (zkCallback-4406-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1938496 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, 
keeping metrics history in memory.
   [junit4]   2> 1938523 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1938539 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1938539 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1938541 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) 
[n:127.0.0.1:33027_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions 
underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/control-001/cores
   [junit4]   2> 1938586 INFO  (zkConnectionManagerCallback-4413-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1938588 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1938589 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33676/solr ready
   [junit4]   2> 1938591 INFO  (qtp684498873-17804) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33027_&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1938600 INFO  
(OverseerThreadFactory-6826-thread-1-processing-n:127.0.0.1:33027_) 
[n:127.0.0.1:33027_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
control_collection
   [junit4]   2> 1938717 INFO  (qtp684498873-17799) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1938722 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_    
x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core 
create command 
qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1938723 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_    
x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault 
Allocating transient cache for 4 transient cores
   [junit4]   2> 1939748 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 1939764 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1939876 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1939896 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' 
using configuration from collection control_collection, trusted=true
   [junit4]   2> 1939896 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.control_collection.shard1.replica_n1' (registry 
'solr.core.control_collection.shard1.replica_n1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1939897 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore 
at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/control-001/cores/control_collection_shard1_replica_n1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 1939900 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=69.8916015625, 
floorSegmentMB=1.7724609375, forceMergeDeletesPctAllowed=22.931552185170048, 
segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, 
deletesPctAllowed=38.42396947966222
   [junit4]   2> 1939902 WARN  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1939976 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1939976 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1939978 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1939978 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1939980 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=1077938811, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 1939981 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1023191f[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1939983 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1939984 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1939985 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1939986 INFO  
(searcherExecutor-6831-thread-1-processing-n:127.0.0.1:33027_ 
x:control_collection_shard1_replica_n1 c:control_collection s:shard1) 
[n:127.0.0.1:33027_ c:control_collection s:shard1  
x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore 
[control_collection_shard1_replica_n1] Registered new searcher 
Searcher@1023191f[control_collection_shard1_replica_n1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1939986 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1620014358056665088
   [junit4]   2> 1940003 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ZkShardTerms Successful update of terms at 
/collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, 
version=0}
   [junit4]   2> 1940007 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1940007 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1940007 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:33027/control_collection_shard1_replica_n1/
   [junit4]   2> 1940007 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1940007 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SyncStrategy 
http://127.0.0.1:33027/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1940007 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 1940012 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:33027/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1940015 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1940022 INFO  (qtp684498873-17800) [n:127.0.0.1:33027_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT}
 status=0 QTime=1300
   [junit4]   2> 1940026 INFO  (qtp684498873-17804) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 1940122 INFO  (zkCallback-4399-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged 
path:/collections/control_collection/state.json] for collection 
[control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1940603 INFO  
(OverseerCollectionConfigSetProcessor-72457420044828676-127.0.0.1:33027_-n_0000000000)
 [n:127.0.0.1:33027_    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1941026 INFO  (qtp684498873-17804) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33027_&wt=javabin&version=2}
 status=0 QTime=2435
   [junit4]   2> 1941032 INFO  (zkConnectionManagerCallback-4418-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1941033 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1941034 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33676/solr ready
   [junit4]   2> 1941034 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 1941035 INFO  (qtp684498873-17803) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1941038 INFO  
(OverseerThreadFactory-6826-thread-2-processing-n:127.0.0.1:33027_) 
[n:127.0.0.1:33027_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
collection1
   [junit4]   2> 1941242 WARN  
(OverseerThreadFactory-6826-thread-2-processing-n:127.0.0.1:33027_) 
[n:127.0.0.1:33027_    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to 
create a collection (collection1) without cores.
   [junit4]   2> 1941256 INFO  (qtp684498873-17803) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 1941256 INFO  (qtp684498873-17803) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2}
 status=0 QTime=220
   [junit4]   2> 1941472 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-1-001
 of type NRT
   [junit4]   2> 1941473 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: 
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
   [junit4]   2> 1941474 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1941474 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1941474 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1941475 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@16f32569{/,null,AVAILABLE}
   [junit4]   2> 1941475 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@432ca69c{HTTP/1.1,[http/1.1]}{127.0.0.1:38074}
   [junit4]   2> 1941475 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server Started @1941556ms
   [junit4]   2> 1941475 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/tempDir-001/jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=38074, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-1-001/cores}
   [junit4]   2> 1941475 ERROR 
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1941475 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1941476 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.6.0
   [junit4]   2> 1941476 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1941476 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1941476 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-16T13:14:46.978Z
   [junit4]   2> 1941499 INFO  (zkConnectionManagerCallback-4420-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1941504 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x1016ba3bf1d0008, likely client has closed socket
   [junit4]   2> 1941504 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1941504 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-1-001/solr.xml
   [junit4]   2> 1941509 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1941509 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1941510 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1941715 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33676/solr
   [junit4]   2> 1941719 INFO  (zkConnectionManagerCallback-4424-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1941748 INFO  (zkConnectionManagerCallback-4426-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1941758 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1941771 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 
transient cores
   [junit4]   2> 1941771 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:38074_
   [junit4]   2> 1941776 INFO  (zkCallback-4417-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1941776 INFO  (zkCallback-4399-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1941776 INFO  (zkCallback-4406-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1941780 INFO  (zkCallback-4425-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1941806 INFO  (zkConnectionManagerCallback-4433-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1941808 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1941813 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33676/solr ready
   [junit4]   2> 1941813 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 1941850 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1941872 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1941872 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1941875 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-1-001/cores
   [junit4]   2> 1941944 INFO  (qtp684498873-17805) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params 
node=127.0.0.1:38074_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1941950 INFO  
(OverseerCollectionConfigSetProcessor-72457420044828676-127.0.0.1:33027_-n_0000000000)
 [n:127.0.0.1:33027_    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1941959 INFO  (qtp684498873-17803) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1941960 INFO  (qtp684498873-17799) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=1
   [junit4]   2> 1941962 INFO  (qtp1326092651-17858) [n:127.0.0.1:38074_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1941963 INFO  
(OverseerThreadFactory-6826-thread-3-processing-n:127.0.0.1:33027_) 
[n:127.0.0.1:33027_ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd Node 
Identified 127.0.0.1:38074_ for creating new replica of shard shard2 for 
collection collection1
   [junit4]   2> 1941967 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_    
x:collection1_shard2_replica_n61] o.a.s.h.a.CoreAdminOperation core create 
command 
qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1942997 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 1943016 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema 
[collection1_shard2_replica_n61] Schema name=test
   [junit4]   2> 1943143 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1943168 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard2_replica_n61' using configuration from 
collection collection1, trusted=true
   [junit4]   2> 1943169 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.collection1.shard2.replica_n61' (registry 
'solr.core.collection1.shard2.replica_n61') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1943169 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.SolrCore 
[[collection1_shard2_replica_n61] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-1-001/cores/collection1_shard2_replica_n61],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-1-001/cores/collection1_shard2_replica_n61/data/]
   [junit4]   2> 1943173 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=69.8916015625, 
floorSegmentMB=1.7724609375, forceMergeDeletesPctAllowed=22.931552185170048, 
segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, 
deletesPctAllowed=38.42396947966222
   [junit4]   2> 1943179 WARN  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1943280 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler 
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1943280 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1943282 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker 
Hard AutoCommit: disabled
   [junit4]   2> 1943282 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker 
Soft AutoCommit: disabled
   [junit4]   2> 1943285 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=1077938811, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 1943286 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@248b178f[collection1_shard2_replica_n61] main]
   [junit4]   2> 1943288 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1943289 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1943290 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1943290 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog 
Could not find max version in index or recent updates, using new clock 
1620014361521160192
   [junit4]   2> 1943293 INFO  
(searcherExecutor-6845-thread-1-processing-n:127.0.0.1:38074_ 
x:collection1_shard2_replica_n61 c:collection1 s:shard2) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.SolrCore 
[collection1_shard2_replica_n61] Registered new searcher 
Searcher@248b178f[collection1_shard2_replica_n61] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1943301 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.ZkShardTerms 
Successful update of terms at /collections/collection1/terms/shard2 to 
Terms{values={core_node62=0}, version=0}
   [junit4]   2> 1943305 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1943305 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1943305 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:38074/collection1_shard2_replica_n61/
   [junit4]   2> 1943305 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 1943308 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy 
http://127.0.0.1:38074/collection1_shard2_replica_n61/ has no replicas
   [junit4]   2> 1943308 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 1943321 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:38074/collection1_shard2_replica_n61/ shard2
   [junit4]   2> 1943324 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 1943329 INFO  (qtp1326092651-17853) [n:127.0.0.1:38074_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT}
 status=0 QTime=1362
   [junit4]   2> 1943332 INFO  (qtp684498873-17805) [n:127.0.0.1:33027_ 
c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null 
path=/admin/collections 
params={node=127.0.0.1:38074_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2}
 status=0 QTime=1387
   [junit4]   2> 1943558 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-2-001
 of type NRT
   [junit4]   2> 1943559 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: 
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
   [junit4]   2> 1943562 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1943562 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1943562 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1943562 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@503e9c54{/,null,AVAILABLE}
   [junit4]   2> 1943563 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@24d1a788{HTTP/1.1,[http/1.1]}{127.0.0.1:34292}
   [junit4]   2> 1943563 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server Started @1943644ms
   [junit4]   2> 1943563 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/tempDir-001/jetty2,
 replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=34292, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-2-001/cores}
   [junit4]   2> 1943563 ERROR 
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1943565 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1943565 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.6.0
   [junit4]   2> 1943565 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1943565 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1943565 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-16T13:14:49.067Z
   [junit4]   2> 1943567 INFO  (zkConnectionManagerCallback-4436-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1943569 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1943569 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-2-001/solr.xml
   [junit4]   2> 1943574 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1943574 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1943576 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1943848 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33676/solr
   [junit4]   2> 1943850 INFO  (zkConnectionManagerCallback-4440-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1943852 INFO  (zkConnectionManagerCallback-4442-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1943862 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1943868 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 
transient cores
   [junit4]   2> 1943868 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:34292_
   [junit4]   2> 1943870 INFO  (zkCallback-4406-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1943870 INFO  (zkCallback-4441-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1943871 INFO  (zkCallback-4417-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1943871 INFO  (zkCallback-4399-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1943873 INFO  (zkCallback-4425-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1943874 INFO  (zkCallback-4432-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1943899 INFO  (zkConnectionManagerCallback-4449-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1943900 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1943901 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33676/solr ready
   [junit4]   2> 1943902 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 1943929 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1943947 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1943947 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1943950 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-2-001/cores
   [junit4]   2> 1943953 INFO  
(OverseerCollectionConfigSetProcessor-72457420044828676-127.0.0.1:33027_-n_0000000000)
 [n:127.0.0.1:33027_    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1944009 INFO  (qtp684498873-17799) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params 
node=127.0.0.1:34292_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1944015 INFO  (qtp684498873-17805) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1944016 INFO  (qtp684498873-17803) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1944018 INFO  (qtp1234287692-17896) [n:127.0.0.1:34292_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1944021 INFO  (qtp1326092651-17854) [n:127.0.0.1:38074_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.collection1.shard2.replica_n61:INDEX.sizeInBytes}
 status=0 QTime=2
   [junit4]   2> 1944023 INFO  (qtp1326092651-17860) [n:127.0.0.1:38074_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1944023 INFO  
(OverseerThreadFactory-6826-thread-4-processing-n:127.0.0.1:33027_) 
[n:127.0.0.1:33027_ c:collection1 s:shard3  ] o.a.s.c.a.c.AddReplicaCmd Node 
Identified 127.0.0.1:34292_ for creating new replica of shard shard3 for 
collection collection1
   [junit4]   2> 1944026 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_    
x:collection1_shard3_replica_n63] o.a.s.h.a.CoreAdminOperation core create 
command 
qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1945060 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 1945081 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema 
[collection1_shard3_replica_n63] Schema name=test
   [junit4]   2> 1945204 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1945226 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard3_replica_n63' using configuration from 
collection collection1, trusted=true
   [junit4]   2> 1945226 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.collection1.shard3.replica_n63' (registry 
'solr.core.collection1.shard3.replica_n63') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1945227 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.SolrCore 
[[collection1_shard3_replica_n63] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-2-001/cores/collection1_shard3_replica_n63],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-2-001/cores/collection1_shard3_replica_n63/data/]
   [junit4]   2> 1945230 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=69.8916015625, 
floorSegmentMB=1.7724609375, forceMergeDeletesPctAllowed=22.931552185170048, 
segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, 
deletesPctAllowed=38.42396947966222
   [junit4]   2> 1945232 WARN  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1945300 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.u.UpdateHandler 
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1945300 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1945302 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker 
Hard AutoCommit: disabled
   [junit4]   2> 1945302 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker 
Soft AutoCommit: disabled
   [junit4]   2> 1945304 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=1077938811, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 1945304 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@4b025f0d[collection1_shard3_replica_n63] main]
   [junit4]   2> 1945305 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1945306 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1945307 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1945308 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog 
Could not find max version in index or recent updates, using new clock 
1620014363637186560
   [junit4]   2> 1945311 INFO  
(searcherExecutor-6859-thread-1-processing-n:127.0.0.1:34292_ 
x:collection1_shard3_replica_n63 c:collection1 s:shard3) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.SolrCore 
[collection1_shard3_replica_n63] Registered new searcher 
Searcher@4b025f0d[collection1_shard3_replica_n63] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1945315 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.ZkShardTerms 
Successful update of terms at /collections/collection1/terms/shard3 to 
Terms{values={core_node64=0}, version=0}
   [junit4]   2> 1945318 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1945318 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1945318 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:34292/collection1_shard3_replica_n63/
   [junit4]   2> 1945319 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 1945319 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy 
http://127.0.0.1:34292/collection1_shard3_replica_n63/ has no replicas
   [junit4]   2> 1945319 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 1945323 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:34292/collection1_shard3_replica_n63/ shard3
   [junit4]   2> 1945381 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 1945384 INFO  (qtp1234287692-17891) [n:127.0.0.1:34292_ 
c:collection1 s:shard3  x:collection1_shard3_replica_n63] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT}
 status=0 QTime=1358
   [junit4]   2> 1945387 INFO  (qtp684498873-17799) [n:127.0.0.1:33027_ 
c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null 
path=/admin/collections 
params={node=127.0.0.1:34292_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2}
 status=0 QTime=1378
   [junit4]   2> 1945578 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-3-001
 of type NRT
   [junit4]   2> 1945579 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: 
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
   [junit4]   2> 1945586 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1945587 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1945587 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1945587 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@76caa47a{/,null,AVAILABLE}
   [junit4]   2> 1945588 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@4d387304{HTTP/1.1,[http/1.1]}{127.0.0.1:44971}
   [junit4]   2> 1945588 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server Started @1945669ms
   [junit4]   2> 1945588 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/tempDir-001/jetty3,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=44971, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-3-001/cores}
   [junit4]   2> 1945588 ERROR 
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1945593 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1945593 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.6.0
   [junit4]   2> 1945593 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1945593 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1945594 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-16T13:14:51.096Z
   [junit4]   2> 1945604 INFO  (zkConnectionManagerCallback-4452-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1945606 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1945606 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-3-001/solr.xml
   [junit4]   2> 1945619 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1945619 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1945621 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1945833 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33676/solr
   [junit4]   2> 1945835 INFO  (zkConnectionManagerCallback-4456-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1945838 INFO  (zkConnectionManagerCallback-4458-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1945846 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1945852 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 
transient cores
   [junit4]   2> 1945852 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:44971_
   [junit4]   2> 1945854 INFO  (zkCallback-4406-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945854 INFO  (zkCallback-4441-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945854 INFO  (zkCallback-4417-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945854 INFO  (zkCallback-4448-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945854 INFO  (zkCallback-4425-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945855 INFO  (zkCallback-4432-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945855 INFO  (zkCallback-4399-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945864 INFO  (zkCallback-4457-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1945885 INFO  (zkConnectionManagerCallback-4465-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1945886 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1945887 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33676/solr ready
   [junit4]   2> 1945888 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history 
in memory.
   [junit4]   2> 1945919 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1945938 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1945938 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1945941 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-3-001/cores
   [junit4]   2> 1946037 INFO  
(OverseerCollectionConfigSetProcessor-72457420044828676-127.0.0.1:33027_-n_0000000000)
 [n:127.0.0.1:33027_    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1946039 INFO  (qtp684498873-17805) [n:127.0.0.1:33027_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params 
node=127.0.0.1:44971_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1946044 INFO  (qtp684498873-17799) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1946046 INFO  (qtp684498873-17803) [n:127.0.0.1:33027_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1946046 INFO  (qtp1234287692-17892) [n:127.0.0.1:34292_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.collection1.shard3.replica_n63:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1946050 INFO  (qtp1234287692-17892) [n:127.0.0.1:34292_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1946051 INFO  (qtp1326092651-17856) [n:127.0.0.1:38074_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.collection1.shard2.replica_n61:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1946053 INFO  (qtp1326092651-17858) [n:127.0.0.1:38074_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1946055 INFO  (qtp764383167-17933) [n:127.0.0.1:44971_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1946056 INFO  
(OverseerThreadFactory-6826-thread-5-processing-n:127.0.0.1:33027_) 
[n:127.0.0.1:33027_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node 
Identified 127.0.0.1:44971_ for creating new replica of shard shard1 for 
collection collection1
   [junit4]   2> 1946059 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_    
x:collection1_shard1_replica_n65] o.a.s.h.a.CoreAdminOperation core create 
command 
qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1947082 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 1947098 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema 
[collection1_shard1_replica_n65] Schema name=test
   [junit4]   2> 1947211 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1947229 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard1_replica_n65' using configuration from 
collection collection1, trusted=true
   [junit4]   2> 1947229 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.collection1.shard1.replica_n65' (registry 
'solr.core.collection1.shard1.replica_n65') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@47567a5d
   [junit4]   2> 1947230 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.SolrCore 
[[collection1_shard1_replica_n65] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-3-001/cores/collection1_shard1_replica_n65],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-3-001/cores/collection1_shard1_replica_n65/data/]
   [junit4]   2> 1947233 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=69.8916015625, 
floorSegmentMB=1.7724609375, forceMergeDeletesPctAllowed=22.931552185170048, 
segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, 
deletesPctAllowed=38.42396947966222
   [junit4]   2> 1947235 WARN  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1947302 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.u.UpdateHandler 
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1947302 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1947304 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker 
Hard AutoCommit: disabled
   [junit4]   2> 1947304 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker 
Soft AutoCommit: disabled
   [junit4]   2> 1947306 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=1077938811, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 1947306 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@3a865027[collection1_shard1_replica_n65] main]
   [junit4]   2> 1947308 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1947308 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1947309 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1947310 INFO  
(searcherExecutor-6873-thread-1-processing-n:127.0.0.1:44971_ 
x:collection1_shard1_replica_n65 c:collection1 s:shard1) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.SolrCore 
[collection1_shard1_replica_n65] Registered new searcher 
Searcher@3a865027[collection1_shard1_replica_n65] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1947310 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog 
Could not find max version in index or recent updates, using new clock 
1620014365736435712
   [junit4]   2> 1947316 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.ZkShardTerms 
Successful update of terms at /collections/collection1/terms/shard1 to 
Terms{values={core_node66=0}, version=0}
   [junit4]   2> 1947319 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1947319 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1947319 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:44971/collection1_shard1_replica_n65/
   [junit4]   2> 1947319 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 1947319 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy 
http://127.0.0.1:44971/collection1_shard1_replica_n65/ has no replicas
   [junit4]   2> 1947319 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 1947324 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:44971/collection1_shard1_replica_n65/ shard1
   [junit4]   2> 1947328 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 1947331 INFO  (qtp764383167-17928) [n:127.0.0.1:44971_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n65] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT}
 status=0 QTime=1271
   [junit4]   2> 1947333 INFO  (qtp684498873-17805) [n:127.0.0.1:33027_ 
c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null 
path=/admin/collections 
params={node=127.0.0.1:44971_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2}
 status=0 QTime=1293
   [junit4]   2> 1947511 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_176FDA4AD2D42496-001/shard-4-001
 of type NRT
   [junit4]   2> 1947512 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: 
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
   [junit4]   2> 1947514 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1947514 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1947514 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1947515 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@26ca6b3a{/,null,AVAILABLE}
   [junit4]   2> 1947515 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@1d69f5ff{HTTP/1.1,[http/1.1]}{127.0.0.1:33521}
   [junit4]   2> 1947515 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.e.j.s.Server Started @1947596ms
   [junit4]   2> 1947515 INFO  
(TEST-FullSolrCloudDistribCmdsTest.test-seed#[176FDA4AD2D42496]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyT

[...truncated too long message...]

128), 29parent_f1_s=PostingsFormat(name=Memory), 
33parent_f1_s=PostingsFormat(name=Memory), 39parent_f1_s=FSTOrd50, 
3child0_s=Lucene50(blocksize=128), 4child3_s=Lucene50(blocksize=128), 
12parent_f1_s=Lucene50(blocksize=128), 
22parent_f1_s=PostingsFormat(name=Memory), 
1grand0_s=PostingsFormat(name=Memory), 
45parent_f2_s=PostingsFormat(name=Memory), 
7parent_f1_s=PostingsFormat(name=Memory), 2child1_s=Lucene50(blocksize=128), 
32parent_f1_s=PostingsFormat(name=Memory), 
3parent_f2_s=PostingsFormat(name=Memory), 42parent_f1_s=FSTOrd50, 
6grand3_s=PostingsFormat(name=Memory), 31parent_f2_s=Lucene50(blocksize=128), 
41parent_f2_s=PostingsFormat(name=Memory), 5grand2_s=Lucene50(blocksize=128), 
11parent_f2_s=PostingsFormat(name=Memory), 21parent_f2_s=FSTOrd50, 
4grand1_s=PostingsFormat(name=Memory), 
51parent_f2_s=PostingsFormat(name=Memory), 7grand4_s=Lucene50(blocksize=128), 
3grand0_s=Lucene50(blocksize=128), 40parent_f1_s=PostingsFormat(name=Memory), 
50parent_f1_s=PostingsFormat(name=Memory), 
9parent_f1_s=Lucene50(blocksize=128), 30parent_f1_s=Lucene50(blocksize=128), 
14parent_f2_s=FSTOrd50, 20parent_f1_s=FSTOrd50, 1parent_f2_s=FSTOrd50, 
7grand0_s=Lucene50(blocksize=128), text_t=PostingsFormat(name=Memory), 
24parent_f2_s=Lucene50(blocksize=128), 
44parent_f2_s=PostingsFormat(name=Memory), 
34parent_f2_s=PostingsFormat(name=Memory), 
13parent_f2_s=Lucene50(blocksize=128), type_s=PostingsFormat(name=Memory), 
23parent_f2_s=PostingsFormat(name=Memory), 43parent_f2_s=FSTOrd50, 
10parent_f1_s=PostingsFormat(name=Memory), 
33parent_f2_s=PostingsFormat(name=Memory), 
22parent_f2_s=PostingsFormat(name=Memory), 4child0_s=FSTOrd50, 
12parent_f2_s=PostingsFormat(name=Memory), 
42parent_f2_s=Lucene50(blocksize=128), 32parent_f2_s=FSTOrd50, 
31parent_f1_s=FSTOrd50, 5grand1_s=PostingsFormat(name=Memory), 
6grand4_s=FSTOrd50, 11parent_f1_s=PostingsFormat(name=Memory), 
51parent_f1_s=PostingsFormat(name=Memory), 
21parent_f1_s=PostingsFormat(name=Memory), 4grand2_s=FSTOrd50, 
7grand3_s=PostingsFormat(name=Memory), 41parent_f1_s=Lucene50(blocksize=128), 
37parent_f1_s=PostingsFormat(name=Memory), 
40parent_f2_s=PostingsFormat(name=Memory), 50parent_f2_s=FSTOrd50, 
47parent_f1_s=PostingsFormat(name=Memory), 
14parent_f1_s=PostingsFormat(name=Memory), 
20parent_f2_s=Lucene50(blocksize=128), 34parent_f1_s=Lucene50(blocksize=128), 
24parent_f1_s=FSTOrd50, 44parent_f1_s=PostingsFormat(name=Memory), 
30parent_f2_s=PostingsFormat(name=Memory), 
5parent_f1_s=Lucene50(blocksize=128), 6grand0_s=FSTOrd50, 
46parent_f2_s=Lucene50(blocksize=128), 10parent_f2_s=FSTOrd50, 
8parent_f1_s=FSTOrd50, 27parent_f2_s=PostingsFormat(name=Memory), 
2parent_f2_s=Lucene50(blocksize=128), 4child1_s=PostingsFormat(name=Memory), 
2grand1_s=Lucene50(blocksize=128), 17parent_f2_s=Lucene50(blocksize=128), 
3child2_s=PostingsFormat(name=Memory), 16parent_f2_s=FSTOrd50, 
26parent_f2_s=PostingsFormat(name=Memory), 
1child0_s=PostingsFormat(name=Memory), 36parent_f2_s=FSTOrd50, 
7grand6_s=PostingsFormat(name=Memory), 
18parent_f1_s=PostingsFormat(name=Memory), 
28parent_f1_s=PostingsFormat(name=Memory), 
5grand0_s=PostingsFormat(name=Memory), 6grand5_s=Lucene50(blocksize=128), 
7grand2_s=PostingsFormat(name=Memory), 
48parent_f1_s=PostingsFormat(name=Memory), 
15parent_f1_s=PostingsFormat(name=Memory), 4grand3_s=Lucene50(blocksize=128), 
6parent_f1_s=PostingsFormat(name=Memory), 
25parent_f1_s=PostingsFormat(name=Memory), 0parent_f1_s=FSTOrd50, 
3grand2_s=PostingsFormat(name=Memory), 35parent_f1_s=FSTOrd50, 
19parent_f2_s=PostingsFormat(name=Memory), 
4parent_f2_s=PostingsFormat(name=Memory), 
37parent_f2_s=PostingsFormat(name=Memory), 47parent_f2_s=FSTOrd50, 
38parent_f1_s=Lucene50(blocksize=128), 5grand4_s=PostingsFormat(name=Memory), 
6grand1_s=Lucene50(blocksize=128), 5parent_f2_s=FSTOrd50, 
46parent_f1_s=FSTOrd50, 16parent_f1_s=Lucene50(blocksize=128), 
8parent_f2_s=PostingsFormat(name=Memory), 
39parent_f2_s=Lucene50(blocksize=128), 
29parent_f2_s=PostingsFormat(name=Memory), 
2parent_f1_s=PostingsFormat(name=Memory), 
4child2_s=PostingsFormat(name=Memory), rnd_b=Lucene50(blocksize=128), 
27parent_f1_s=Lucene50(blocksize=128), 45parent_f1_s=Lucene50(blocksize=128), 
17parent_f1_s=PostingsFormat(name=Memory), 
2grand0_s=PostingsFormat(name=Memory), 
49parent_f2_s=PostingsFormat(name=Memory), 3child1_s=FSTOrd50, 
7parent_f2_s=PostingsFormat(name=Memory), 
26parent_f1_s=PostingsFormat(name=Memory), 2child0_s=FSTOrd50, 
3parent_f1_s=PostingsFormat(name=Memory), 
36parent_f1_s=PostingsFormat(name=Memory)}, 
docValues:{_version_=DocValuesFormat(name=Asserting), 
id=DocValuesFormat(name=Lucene70), text_t=DocValuesFormat(name=Asserting)}, 
maxPointsInLeafNode=1190, maxMBSortInHeap=6.437865357597358, 
sim=RandomSimilarity(queryNorm=false): {}, locale=el, timezone=Asia/Kabul
   [junit4]   2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 
1.8.0_191 (64-bit)/cpus=4,threads=1,free=127359600,total=512229376
   [junit4]   2> NOTE: All tests run in this JVM: [DataDrivenBlockJoinTest, 
TestCollapseQParserPlugin, TestMissingGroups, ZkStateWriterTest, TestLRUCache, 
TestSystemIdResolver, TestNumericTerms32, ShufflingReplicaListTransformerTest, 
TestCloudSearcherWarming, DocumentAnalysisRequestHandlerTest, 
NotRequiredUniqueKeyTest, DistributedFacetPivotWhiteBoxTest, UpdateLogTest, 
TestHighlightDedupGrouping, TestFieldTypeResource, DocValuesMissingTest, 
TestPHPSerializedResponseWriter, TestQuerySenderListener, 
TestDownShardTolerantSearch, TestInPlaceUpdatesDistrib, TestCloudManagedSchema, 
DocValuesNotIndexedTest, TestCollationFieldDocValues, 
HdfsBasicDistributedZkTest, TestRawTransformer, 
DocExpirationUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, 
DistributedQueryComponentOptimizationTest, ReplaceNodeNoTargetTest, 
VMParamsZkACLAndCredentialsProvidersTest, SynonymTokenizerTest, TestMacros, 
TestFunctionQuery, SolrMetricReporterTest, TestFastOutputStream, 
TestCloudPivotFacet, TestXmlQParser, TestConfigSets, TestDynamicLoading, 
TestScoreJoinQPNoScore, PropertiesRequestHandlerTest, TestBinaryResponseWriter, 
BufferStoreTest, OverseerTaskQueueTest, BooleanFieldTest, HighlighterTest, 
MBeansHandlerTest, BlockJoinFacetDistribTest, TestPointFields, CacheHeaderTest, 
TestFieldCacheReopen, NoCacheHeaderTest, TestChildDocTransformer, 
TestExpandComponent, ActionThrottleTest, TimeZoneUtilsTest, 
TestNonDefinedSimilarityFactory, ConcurrentDeleteAndCreateCollectionTest, 
TestIndexSearcher, IndexSizeTriggerTest, BasicFunctionalityTest, 
TestJsonFacetsStatsParsing, NodeLostTriggerTest, 
TestMultiValuedNumericRangeQuery, TestPseudoReturnFields, 
SuggestComponentContextFilterQueryTest, LIRRollingUpdatesTest, 
SpellPossibilityIteratorTest, TestPrepRecovery, TestSimComputePlanAction, 
XmlUpdateRequestHandlerTest, TestSolr4Spatial2, 
DistributedFacetPivotLongTailTest, RestartWhileUpdatingTest, PluginInfoTest, 
AutoAddReplicasPlanActionTest, HdfsRestartWhileUpdatingTest, 
RequestHandlersTest, BasicDistributedZkTest, TestUnInvertedFieldException, 
MoreLikeThisHandlerTest, ExitableDirectoryReaderTest, TestSolrDeletionPolicy1, 
DistribJoinFromCollectionTest, TestSubQueryTransformerCrossCore, 
CheckHdfsIndexTest, TestUtilizeNode, TestImpersonationWithHadoopAuth, 
TestSweetSpotSimilarityFactory, TestRecovery, 
TestOverriddenPrefixQueryForCustomFieldType, TestGeoJSONResponseWriter, 
TaggerTest, TestStressCloudBlindAtomicUpdates, 
TestRuleBasedAuthorizationPlugin, TestLegacyFieldReuse, TestManagedSchemaAPI, 
TestIBSimilarityFactory, TestDocTermOrds, MaxSizeAutoCommitTest, 
TestSlowCompositeReaderWrapper, IndexSchemaTest, TestReloadDeadlock, 
MetricsConfigTest, PrimitiveFieldTypeTest, TestPhraseSuggestions, 
TestTrackingShardHandlerFactory, TestPolicyCloud, SolrInfoBeanTest, 
SaslZkACLProviderTest, TestCoreAdminApis, TestNamedUpdateProcessors, 
FileUtilsTest, TestCSVLoader, MultiSolrCloudTestCaseTest, 
CurrencyFieldTypeTest, HLLSerializationTest, WordBreakSolrSpellCheckerTest, 
TestReqParamsAPI, TestTlogReplica, TestCursorMarkWithoutUniqueKey, 
TermVectorComponentTest, AddSchemaFieldsUpdateProcessorFactoryTest, 
TestRTimerTree, MinimalSchemaTest, SolrCloudExampleTest, TestStressReorder, 
TestManagedResourceStorage, TestEmbeddedSolrServerSchemaAPI, RankQueryTest, 
TestTolerantSearch, TestInfoStreamLogging, TestReversedWildcardFilterFactory, 
NodeAddedTriggerIntegrationTest, RegexBoostProcessorTest, OutputWriterTest, 
TestDistributedGrouping, TestRandomFaceting, TestSimpleTrackingShardHandler, 
TestCharFilters, TokenizerChainTest, TestEmbeddedSolrServerAdminHandler, 
TestJettySolrRunner, AliasIntegrationTest, 
ChaosMonkeySafeLeaderWithPullReplicasTest, CreateCollectionCleanupTest, 
DeleteInactiveReplicaTest, DeleteNodeTest, DeleteShardTest, DeleteStatusTest, 
DistributedQueueTest, ForceLeaderTest, FullSolrCloudDistribCmdsTest]
   [junit4] Completed [506/836 (1!)] on J1 in 762.48s, 1 test, 1 error <<< 
FAILURES!

[...truncated 1154 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/solr/build/solr-core/test/temp/junit4-J1-20181216_124225_3483766793904151649004.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/heapdumps/java_pid10312.hprof
 ...
   [junit4] Heap dump file created [569788600 bytes in 3.816 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 9008 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/build.xml:651:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.6/checkout/build.xml:585:
 Some of the tests produced a heap dump, but did not fail. Maybe a suppressed 
OutOfMemoryError? Dumps created:
* java_pid10312.hprof

Total time: 256 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to