Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/601/
1 tests failed.
FAILED: org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=3086,
name=SocketProxy-Response-58341:44485, state=RUNNABLE,
group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=3086, name=SocketProxy-Response-58341:44485,
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
at
__randomizedtesting.SeedInfo.seed([978136A83E311B2D:1FD5097290CD76D5]:0)
Caused by: java.lang.RuntimeException: java.net.SocketException: Socket is
closed
at __randomizedtesting.SeedInfo.seed([978136A83E311B2D]:0)
at
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:347)
Caused by: java.net.SocketException: Socket is closed
at java.net.Socket.setSoTimeout(Socket.java:1137)
at
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:344)
Build Log:
[...truncated 10995 lines...]
[junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest
[junit4] 2> Creating dataDir:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/init-core-data-001
[junit4] 2> 275131 INFO
(SUITE-LeaderInitiatedRecoveryOnCommitTest-seed#[978136A83E311B2D]-worker) [
] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via:
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 275131 INFO
(SUITE-LeaderInitiatedRecoveryOnCommitTest-seed#[978136A83E311B2D]-worker) [
] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 275150 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 275158 INFO (Thread-643) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 275158 INFO (Thread-643) [ ] o.a.s.c.ZkTestServer Starting
server
[junit4] 2> 275257 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkTestServer start zk server on port:46421
[junit4] 2> 275531 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 275534 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml
to /configs/conf1/schema.xml
[junit4] 2> 275537 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 275538 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 275540 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 275548 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 275562 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 275564 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 275565 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 275566 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 275568 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 275996 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/control-001/cores/collection1
[junit4] 2> 276006 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 276023 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@6aeedd9e{/,null,AVAILABLE}
[junit4] 2> 276023 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@5f26d6ec{HTTP/1.1,[http/1.1]}{127.0.0.1:45614}
[junit4] 2> 276023 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server Started @280519ms
[junit4] 2> 276023 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/tempDir-001/control/data,
hostContext=/, hostPort=60734,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/control-001/cores}
[junit4] 2> 276032 ERROR
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 276032 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
6.4.0
[junit4] 2> 276032 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 276032 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 276032 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2016-12-15T14:39:55.167Z
[junit4] 2> 276050 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 276050 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/control-001/solr.xml
[junit4] 2> 276062 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 276070 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46421/solr
[junit4] 2> 276137 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:60734_ ] o.a.s.c.OverseerElectionContext I am going to be the
leader 127.0.0.1:60734_
[junit4] 2> 276138 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:60734_ ] o.a.s.c.Overseer Overseer
(id=97112083287441412-127.0.0.1:60734_-n_0000000000) starting
[junit4] 2> 276173 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:60734_ ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:60734_
[junit4] 2> 276176 INFO
(zkCallback-540-thread-1-processing-n:127.0.0.1:60734_) [n:127.0.0.1:60734_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 276287 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:60734_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/control-001/cores
[junit4] 2> 276287 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:60734_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 276327 INFO
(OverseerStateUpdate-97112083287441412-127.0.0.1:60734_-n_0000000000)
[n:127.0.0.1:60734_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard
shard=shard1
[junit4] 2> 277511 WARN
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.c.Config
Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory>
instead.
[junit4] 2> 277512 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.c.SolrConfig
Using Lucene MatchVersion: 6.4.0
[junit4] 2> 277535 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.s.IndexSchema
[collection1] Schema name=test
[junit4] 2> 277757 WARN
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.s.IndexSchema
[collection1] default search field in schema is text. WARNING: Deprecated,
please use 'df' on request instead.
[junit4] 2> 277793 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.s.IndexSchema
Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 277830 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.c.CoreContainer
Creating SolrCore 'collection1' using configuration from collection
control_collection
[junit4] 2> 277831 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.c.SolrCore
[[collection1] ] Opening new SolrCore at
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/control-001/cores/collection1],
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/control-001/cores/collection1/data/]
[junit4] 2> 277831 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e9aa36b
[junit4] 2> 277842 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy:
minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7311198731128292]
[junit4] 2> 278114 WARN
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type =
requestHandler,name = /dump,class = DumpRequestHandler,args =
{defaults={a=A,b=B}}}
[junit4] 2> 278135 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.u.UpdateHandler
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 278135 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 278136 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.u.CommitTracker
Hard AutoCommit: disabled
[junit4] 2> 278136 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.u.CommitTracker
Soft AutoCommit: disabled
[junit4] 2> 278136 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=22, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=72.625,
floorSegmentMB=1.6845703125, forceMergeDeletesPctAllowed=4.546658267128728,
segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 278137 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.s.SolrIndexSearcher Opening [Searcher@370b816b[collection1] main]
[junit4] 2> 278138 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 278138 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 278138 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 278139 INFO
(coreLoadExecutor-1572-thread-1-processing-n:127.0.0.1:60734_)
[n:127.0.0.1:60734_ c:control_collection x:collection1] o.a.s.u.UpdateLog
Could not find max version in index or recent updates, using new clock
1553793335714381824
[junit4] 2> 278141 INFO
(searcherExecutor-1573-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection x:collection1]
o.a.s.c.SolrCore [collection1] Registered new searcher
Searcher@370b816b[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 278162 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas
found to continue.
[junit4] 2> 278162 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new
leader - try and sync
[junit4] 2> 278162 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to
http://127.0.0.1:60734/collection1/
[junit4] 2> 278162 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync
replicas to me
[junit4] 2> 278162 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.SyncStrategy
http://127.0.0.1:60734/collection1/ has no replicas
[junit4] 2> 278167 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new
leader: http://127.0.0.1:60734/collection1/ shard1
[junit4] 2> 278318 INFO
(coreZkRegister-1565-thread-1-processing-n:127.0.0.1:60734_ x:collection1
c:control_collection) [n:127.0.0.1:60734_ c:control_collection s:shard1
r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery
necessary
[junit4] 2> 278370 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 278371 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46421/solr ready
[junit4] 2> 278371 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection
loss:false
[junit4] 2> 278372 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 278529 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001/cores/collection1
[junit4] 2> 278530 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001
[junit4] 2> 278535 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 278599 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@2b24d256{/,null,AVAILABLE}
[junit4] 2> 278599 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@37e8084d{HTTP/1.1,[http/1.1]}{127.0.0.1:58341}
[junit4] 2> 278599 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server Started @283094ms
[junit4] 2> 278599 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/tempDir-001/jetty1,
solrconfig=solrconfig.xml, hostContext=/, hostPort=40876,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001/cores}
[junit4] 2> 278600 ERROR
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 278600 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
6.4.0
[junit4] 2> 278600 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 278600 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 278600 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2016-12-15T14:39:57.735Z
[junit4] 2> 278665 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 278665 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001/solr.xml
[junit4] 2> 278691 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 278695 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46421/solr
[junit4] 2> 278744 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:40876_ ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (1)
[junit4] 2> 278763 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:40876_ ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:40876_
[junit4] 2> 278765 INFO (zkCallback-544-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 278765 INFO
(zkCallback-540-thread-2-processing-n:127.0.0.1:60734_) [n:127.0.0.1:60734_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 278765 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 279196 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:40876_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001/cores
[junit4] 2> 279196 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:40876_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 279210 INFO
(OverseerStateUpdate-97112083287441412-127.0.0.1:60734_-n_0000000000)
[n:127.0.0.1:60734_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard
shard=shard1
[junit4] 2> 279313 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 280225 WARN
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.c.Config Beginning
with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 280226 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 6.4.0
[junit4] 2> 280270 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] Schema name=test
[junit4] 2> 280556 WARN
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] default search field in schema is text. WARNING: Deprecated,
please use 'df' on request instead.
[junit4] 2> 280559 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded
schema test/1.0 with uniqueid field id
[junit4] 2> 280586 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.c.CoreContainer
Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 280587 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.c.SolrCore
[[collection1] ] Opening new SolrCore at
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001/cores/collection1],
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 280588 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX
monitoring is enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@5e9aa36b
[junit4] 2> 280589 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=46,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7311198731128292]
[junit4] 2> 280594 WARN
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 280646 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using
UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 280646 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 280647 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard
AutoCommit: disabled
[junit4] 2> 280647 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft
AutoCommit: disabled
[junit4] 2> 280650 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=37,
maxMergedSegmentMB=72.625, floorSegmentMB=1.6845703125,
forceMergeDeletesPctAllowed=4.546658267128728, segmentsPerTier=28.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 280651 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher
Opening [Searcher@59cca3dc[collection1] main]
[junit4] 2> 280653 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 280653 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 280653 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler
Commits will be reserved for 10000
[junit4] 2> 280656 INFO
(coreLoadExecutor-1583-thread-1-processing-n:127.0.0.1:40876_)
[n:127.0.0.1:40876_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not
find max version in index or recent updates, using new clock 1553793338353647616
[junit4] 2> 280679 INFO
(searcherExecutor-1584-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 x:collection1]
o.a.s.c.SolrCore [collection1] Registered new searcher
Searcher@59cca3dc[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 280706 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to
continue.
[junit4] 2> 280706 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try
and sync
[junit4] 2> 280706 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.SyncStrategy Sync replicas to
http://127.0.0.1:40876/collection1/
[junit4] 2> 280706 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 280706 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40876/collection1/ has no
replicas
[junit4] 2> 280711 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader:
http://127.0.0.1:40876/collection1/ shard1
[junit4] 2> 280814 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 280862 INFO
(coreZkRegister-1578-thread-1-processing-n:127.0.0.1:40876_ x:collection1
c:collection1) [n:127.0.0.1:40876_ c:collection1 s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 280980 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 281060 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001/cores/collection1
[junit4] 2> 281062 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001
[junit4] 2> 281063 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 281066 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@5e427f71{/,null,AVAILABLE}
[junit4] 2> 281066 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@22a7d824{HTTP/1.1,[http/1.1]}{127.0.0.1:50096}
[junit4] 2> 281066 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server Started @285561ms
[junit4] 2> 281066 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/tempDir-001/jetty2,
solrconfig=solrconfig.xml, hostContext=/, hostPort=37195,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001/cores}
[junit4] 2> 281066 ERROR
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 281067 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
6.4.0
[junit4] 2> 281067 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 281067 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 281067 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2016-12-15T14:40:00.202Z
[junit4] 2> 281095 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 281095 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001/solr.xml
[junit4] 2> 281113 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 281122 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46421/solr
[junit4] 2> 281228 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:37195_ ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (2)
[junit4] 2> 281233 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:37195_ ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:37195_
[junit4] 2> 281234 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 281234 INFO (zkCallback-544-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 281235 INFO
(zkCallback-540-thread-5-processing-n:127.0.0.1:60734_) [n:127.0.0.1:60734_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 281243 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 281351 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 281464 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:37195_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001/cores
[junit4] 2> 281464 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:37195_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 281471 INFO
(OverseerStateUpdate-97112083287441412-127.0.0.1:60734_-n_0000000000)
[n:127.0.0.1:60734_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard
shard=shard1
[junit4] 2> 281579 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 281579 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 282535 WARN
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.c.Config Beginning
with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 282536 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 6.4.0
[junit4] 2> 282567 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] Schema name=test
[junit4] 2> 282712 WARN
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] default search field in schema is text. WARNING: Deprecated,
please use 'df' on request instead.
[junit4] 2> 282726 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded
schema test/1.0 with uniqueid field id
[junit4] 2> 282760 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.c.CoreContainer
Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 282760 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.c.SolrCore
[[collection1] ] Opening new SolrCore at
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001/cores/collection1],
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 282762 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX
monitoring is enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@5e9aa36b
[junit4] 2> 282778 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=46,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7311198731128292]
[junit4] 2> 282806 WARN
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 282854 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using
UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 282854 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 282855 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard
AutoCommit: disabled
[junit4] 2> 282855 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft
AutoCommit: disabled
[junit4] 2> 282858 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=37,
maxMergedSegmentMB=72.625, floorSegmentMB=1.6845703125,
forceMergeDeletesPctAllowed=4.546658267128728, segmentsPerTier=28.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 282858 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher
Opening [Searcher@1b7baf3b[collection1] main]
[junit4] 2> 282860 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 282861 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 282861 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler
Commits will be reserved for 10000
[junit4] 2> 282861 INFO
(coreLoadExecutor-1594-thread-1-processing-n:127.0.0.1:37195_)
[n:127.0.0.1:37195_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not
find max version in index or recent updates, using new clock 1553793340665757696
[junit4] 2> 282864 INFO
(searcherExecutor-1595-thread-1-processing-n:127.0.0.1:37195_ x:collection1
c:collection1) [n:127.0.0.1:37195_ c:collection1 x:collection1]
o.a.s.c.SolrCore [collection1] Registered new searcher
Searcher@1b7baf3b[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 282876 INFO
(coreZkRegister-1589-thread-1-processing-n:127.0.0.1:37195_ x:collection1
c:collection1) [n:127.0.0.1:37195_ c:collection1 s:shard1 r:core_node2
x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 282890 INFO
(updateExecutor-553-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 282973 INFO
(recoveryExecutor-554-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process.
recoveringAfterStartup=true
[junit4] 2> 282974 INFO
(recoveryExecutor-554-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 282974 INFO
(recoveryExecutor-554-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates.
core=[collection1]
[junit4] 2> 282974 INFO
(recoveryExecutor-554-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates.
FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 282974 INFO
(recoveryExecutor-554-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core
[collection1] as recovering, leader is [http://127.0.0.1:40876/collection1/]
and I am [http://127.0.0.1:37195/collection1/]
[junit4] 2> 282985 INFO
(recoveryExecutor-554-thread-1-processing-n:127.0.0.1:37195_ x:collection1
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37195_ c:collection1 s:shard1
r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery
command to [http://127.0.0.1:40876]; [WaitForState:
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:37195_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 282988 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy accepted
Socket[addr=/127.0.0.1,port=44432,localport=40876], receiveBufferSize:531000
[junit4] 2> 283014 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy proxy connection
Socket[addr=/127.0.0.1,port=58341,localport=36777], receiveBufferSize=530904
[junit4] 2> 283026 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state:
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 283026 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1
(shard1 of collection1) have state: recovering
[junit4] 2> 283026 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1,
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader?
true, live=true, checkLive=true, currentState=down, localState=active,
nodeName=127.0.0.1:37195_, coreNodeName=core_node2,
onlyIfActiveCheckResult=false, nodeProps:
core_node2:{"core":"collection1","base_url":"http://127.0.0.1:37195","node_name":"127.0.0.1:37195_","state":"down"}
[junit4] 2> 283080 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 283080 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 283314 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001/cores/collection1
[junit4] 2> 283316 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001
[junit4] 2> 283317 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 283543 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@1ee6edf0{/,null,AVAILABLE}
[junit4] 2> 283544 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@5d988255{HTTP/1.1,[http/1.1]}{127.0.0.1:39512}
[junit4] 2> 283544 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server Started @288039ms
[junit4] 2> 283544 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/tempDir-001/jetty3,
solrconfig=solrconfig.xml, hostContext=/, hostPort=56938,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001/cores}
[junit4] 2> 283544 ERROR
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 283544 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
6.4.0
[junit4] 2> 283544 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 283544 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 283545 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2016-12-15T14:40:02.680Z
[junit4] 2> 283575 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 283575 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001/solr.xml
[junit4] 2> 283588 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 283589 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46421/solr
[junit4] 2> 283628 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:56938_ ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (3)
[junit4] 2> 283632 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:56938_ ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:56938_
[junit4] 2> 283634 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 283634 INFO
(zkCallback-540-thread-5-processing-n:127.0.0.1:60734_) [n:127.0.0.1:60734_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 283638 INFO (zkCallback-544-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 283638 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 283658 INFO
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 283678 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:56938_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001/cores
[junit4] 2> 283678 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:56938_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 283688 INFO
(OverseerStateUpdate-97112083287441412-127.0.0.1:60734_-n_0000000000)
[n:127.0.0.1:60734_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard
shard=shard1
[junit4] 2> 283792 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 283792 INFO
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 283792 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 284027 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1,
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader?
true, live=true, checkLive=true, currentState=recovering, localState=active,
nodeName=127.0.0.1:37195_, coreNodeName=core_node2,
onlyIfActiveCheckResult=false, nodeProps:
core_node2:{"core":"collection1","base_url":"http://127.0.0.1:37195","node_name":"127.0.0.1:37195_","state":"recovering"}
[junit4] 2> 284027 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering,
checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 284027 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={nodeName=127.0.0.1:37195_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
status=0 QTime=1001
[junit4] 2> 284715 WARN
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.c.Config Beginning
with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 284716 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 6.4.0
[junit4] 2> 284755 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] Schema name=test
[junit4] 2> 285047 WARN
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] default search field in schema is text. WARNING: Deprecated,
please use 'df' on request instead.
[junit4] 2> 285050 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded
schema test/1.0 with uniqueid field id
[junit4] 2> 285064 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.c.CoreContainer
Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 285065 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.c.SolrCore
[[collection1] ] Opening new SolrCore at
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001/cores/collection1],
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 285065 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX
monitoring is enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@5e9aa36b
[junit4] 2> 285066 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=46,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7311198731128292]
[junit4] 2> 285106 WARN
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 285123 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using
UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 285123 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 285123 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard
AutoCommit: disabled
[junit4] 2> 285123 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft
AutoCommit: disabled
[junit4] 2> 285128 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=37,
maxMergedSegmentMB=72.625, floorSegmentMB=1.6845703125,
forceMergeDeletesPctAllowed=4.546658267128728, segmentsPerTier=28.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 285138 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher
Opening [Searcher@253bf7b7[collection1] main]
[junit4] 2> 285141 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 285142 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 285142 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler
Commits will be reserved for 10000
[junit4] 2> 285143 INFO
(searcherExecutor-1606-thread-1-processing-n:127.0.0.1:56938_ x:collection1
c:collection1) [n:127.0.0.1:56938_ c:collection1 x:collection1]
o.a.s.c.SolrCore [collection1] Registered new searcher
Searcher@253bf7b7[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 285143 INFO
(coreLoadExecutor-1605-thread-1-processing-n:127.0.0.1:56938_)
[n:127.0.0.1:56938_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not
find max version in index or recent updates, using new clock 1553793343058608128
[junit4] 2> 285159 INFO
(coreZkRegister-1600-thread-1-processing-n:127.0.0.1:56938_ x:collection1
c:collection1) [n:127.0.0.1:56938_ c:collection1 s:shard1 r:core_node3
x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 285162 INFO
(updateExecutor-560-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 285162 INFO
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process.
recoveringAfterStartup=true
[junit4] 2> 285163 INFO
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 285163 INFO
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates.
core=[collection1]
[junit4] 2> 285163 INFO
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates.
FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 285163 INFO
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core
[collection1] as recovering, leader is [http://127.0.0.1:40876/collection1/]
and I am [http://127.0.0.1:56938/collection1/]
[junit4] 2> 285166 INFO
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery
command to [http://127.0.0.1:40876]; [WaitForState:
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:56938_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 285168 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy accepted
Socket[addr=/127.0.0.1,port=44444,localport=40876], receiveBufferSize:531000
[junit4] 2> 285169 INFO (qtp75889157-2921) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state:
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 285174 INFO (qtp75889157-2921) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1
(shard1 of collection1) have state: recovering
[junit4] 2> 285174 INFO (qtp75889157-2921) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1,
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader?
true, live=true, checkLive=true, currentState=down, localState=active,
nodeName=127.0.0.1:56938_, coreNodeName=core_node3,
onlyIfActiveCheckResult=false, nodeProps:
core_node3:{"core":"collection1","base_url":"http://127.0.0.1:56938","node_name":"127.0.0.1:56938_","state":"down"}
[junit4] 2> 285185 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy proxy connection
Socket[addr=/127.0.0.1,port=58341,localport=36789], receiveBufferSize=530904
[junit4] 2> 285268 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 285268 INFO
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 285269 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 285367 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001/cores/collection1
[junit4] 2> 285368 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001
[junit4] 2> 285368 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 285370 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@64ca511a{/,null,AVAILABLE}
[junit4] 2> 285370 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@4a8d233e{HTTP/1.1,[http/1.1]}{127.0.0.1:51149}
[junit4] 2> 285370 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.Server Started @289866ms
[junit4] 2> 285370 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/tempDir-001/jetty4,
solrconfig=solrconfig.xml, hostContext=/, hostPort=46533,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001/cores}
[junit4] 2> 285371 ERROR
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 285371 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
6.4.0
[junit4] 2> 285371 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 285371 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 285371 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2016-12-15T14:40:04.506Z
[junit4] 2> 285387 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 285387 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001/solr.xml
[junit4] 2> 285402 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 285403 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46421/solr
[junit4] 2> 285464 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:46533_ ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (4)
[junit4] 2> 285470 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:46533_ ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:46533_
[junit4] 2> 285472 INFO (zkCallback-544-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 285473 INFO
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 285473 INFO
(zkCallback-570-thread-1-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 285473 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 285473 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 285474 INFO
(zkCallback-540-thread-2-processing-n:127.0.0.1:60734_) [n:127.0.0.1:60734_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 285558 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:46533_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001/cores
[junit4] 2> 285559 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D])
[n:127.0.0.1:46533_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 285564 INFO
(OverseerStateUpdate-97112083287441412-127.0.0.1:60734_-n_0000000000)
[n:127.0.0.1:60734_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard
shard=shard1
[junit4] 2> 285667 INFO
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 285667 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 285667 INFO
(zkCallback-570-thread-1-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 285667 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 286174 INFO (qtp75889157-2921) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1,
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader?
true, live=true, checkLive=true, currentState=recovering, localState=active,
nodeName=127.0.0.1:56938_, coreNodeName=core_node3,
onlyIfActiveCheckResult=false, nodeProps:
core_node3:{"core":"collection1","base_url":"http://127.0.0.1:56938","node_name":"127.0.0.1:56938_","state":"recovering"}
[junit4] 2> 286174 INFO (qtp75889157-2921) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering,
checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 286174 INFO (qtp75889157-2921) [n:127.0.0.1:40876_ ]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={nodeName=127.0.0.1:56938_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
status=0 QTime=1005
[junit4] 2> 286606 WARN
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.c.Config Beginning
with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 286607 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 6.4.0
[junit4] 2> 286623 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] Schema name=test
[junit4] 2> 286717 WARN
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.s.IndexSchema
[collection1] default search field in schema is text. WARNING: Deprecated,
please use 'df' on request instead.
[junit4] 2> 286719 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded
schema test/1.0 with uniqueid field id
[junit4] 2> 286749 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.c.CoreContainer
Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 286750 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.c.SolrCore
[[collection1] ] Opening new SolrCore at
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001/cores/collection1],
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnCommitTest_978136A83E311B2D-001/shard-4-001/cores/collection1/data/]
[junit4] 2> 286750 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX
monitoring is enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@5e9aa36b
[junit4] 2> 286751 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=46,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7311198731128292]
[junit4] 2> 286759 WARN
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 286777 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using
UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 286777 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 286778 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard
AutoCommit: disabled
[junit4] 2> 286778 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft
AutoCommit: disabled
[junit4] 2> 286779 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=37,
maxMergedSegmentMB=72.625, floorSegmentMB=1.6845703125,
forceMergeDeletesPctAllowed=4.546658267128728, segmentsPerTier=28.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 286779 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher
Opening [Searcher@c9e718d[collection1] main]
[junit4] 2> 286780 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 286780 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 286781 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler
Commits will be reserved for 10000
[junit4] 2> 286781 INFO
(coreLoadExecutor-1616-thread-1-processing-n:127.0.0.1:46533_)
[n:127.0.0.1:46533_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not
find max version in index or recent updates, using new clock 1553793344776175616
[junit4] 2> 286785 INFO
(searcherExecutor-1617-thread-1-processing-n:127.0.0.1:46533_ x:collection1
c:collection1) [n:127.0.0.1:46533_ c:collection1 x:collection1]
o.a.s.c.SolrCore [collection1] Registered new searcher
Searcher@c9e718d[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 286787 INFO
(coreZkRegister-1611-thread-1-processing-n:127.0.0.1:46533_ x:collection1
c:collection1) [n:127.0.0.1:46533_ c:collection1 s:shard1 r:core_node4
x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 286788 INFO
(updateExecutor-567-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 286788 INFO
(recoveryExecutor-568-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process.
recoveringAfterStartup=true
[junit4] 2> 286789 INFO
(recoveryExecutor-568-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 286789 INFO
(recoveryExecutor-568-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates.
core=[collection1]
[junit4] 2> 286789 INFO
(recoveryExecutor-568-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates.
FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 286789 INFO
(recoveryExecutor-568-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core
[collection1] as recovering, leader is [http://127.0.0.1:40876/collection1/]
and I am [http://127.0.0.1:46533/collection1/]
[junit4] 2> 286793 INFO
(recoveryExecutor-568-thread-1-processing-n:127.0.0.1:46533_ x:collection1
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46533_ c:collection1 s:shard1
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery
command to [http://127.0.0.1:40876]; [WaitForState:
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:46533_&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 286795 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy accepted
Socket[addr=/127.0.0.1,port=44452,localport=40876], receiveBufferSize:531000
[junit4] 2> 286802 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy proxy connection
Socket[addr=/127.0.0.1,port=58341,localport=36797], receiveBufferSize=530904
[junit4] 2> 286803 INFO (qtp75889157-2923) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state:
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 286803 INFO (qtp75889157-2923) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1
(shard1 of collection1) have state: recovering
[junit4] 2> 286803 INFO (qtp75889157-2923) [n:127.0.0.1:40876_ ]
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1,
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader?
true, live=true, checkLive=true, currentState=down, localState=active,
nodeName=127.0.0.1:46533_, coreNodeName=core_node4,
onlyIfActiveCheckResult=false, nodeProps:
core_node4:{"core":"collection1","base_url":"http://127.0.0.1:46533","node_name":"127.0.0.1:46533_","state":"down"}
[junit4] 2> 286894 INFO
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 286894 INFO
(zkCallback-556-thread-1-processing-n:127.0.0.1:37195_) [n:127.0.0.1:37195_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 286894 INFO
(zkCallback-550-thread-1-processing-n:127.0.0.1:40876_) [n:127.0.0.1:40876_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 286894 INFO
(zkCallback-570-thread-1-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 287082 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 287082 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.LeaderInitiatedRecoveryOnCommitTest Running oneShardTest
[junit4] 2> 287084 INFO (SocketProxy-Acceptor-40876) [ ]
o.a.s.c.SocketProxy accepted
Socket[addr=/127.0.0.1,port=44454,localport=40876], receiveBufferSize:531000
[junit4] 2> 287090 INFO (qtp75889157-2922) [n:127.0.0.1:40876_ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
replicationFactor=3&maxShardsPerNode=1&name=c8n_1x3_commits
[...truncated too long message...]
olrCore org.apache.solr.core.SolrCore@27314396
[junit4] 2> 310215 WARN
(recoveryExecutor-561-thread-1-processing-n:127.0.0.1:56938_ x:collection1
s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:56938_ c:collection1 s:shard1
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Stopping recovery for
core=[collection1] coreNodeName=[core_node3]
[junit4] 2> 310221 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.Overseer Overseer (id=97112083287441423-127.0.0.1:56938_-n_0000000003)
closing
[junit4] 2> 310221 INFO
(OverseerStateUpdate-97112083287441423-127.0.0.1:56938_-n_0000000003)
[n:127.0.0.1:56938_ ] o.a.s.c.Overseer Overseer Loop exiting :
127.0.0.1:56938_
[junit4] 2> 310228 INFO
(zkCallback-570-thread-2-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 310228 INFO
(zkCallback-570-thread-1-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46533_
[junit4] 2> 310228 INFO (zkCallback-544-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 310229 INFO
(zkCallback-570-thread-1-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.Overseer Overseer
(id=97112083287441426-127.0.0.1:46533_-n_0000000004) starting
[junit4] 2> 311722 WARN
(zkCallback-563-thread-2-processing-n:127.0.0.1:56938_) [n:127.0.0.1:56938_
] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to
ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 311725 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Stopped
ServerConnector@5d988255{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 311725 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Stopped
o.e.j.s.ServletContextHandler@1ee6edf0{/,null,UNAVAILABLE}
[junit4] 2> 311728 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ChaosMonkey monkey: stop shard! 46533
[junit4] 2> 311728 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.CoreContainer Shutting down CoreContainer instance=1606792420
[junit4] 2> 311737 INFO (coreCloseExecutor-1673-thread-1)
[n:127.0.0.1:46533_ c:collection1 s:shard1 r:core_node4 x:collection1]
o.a.s.c.SolrCore [collection1] CLOSING SolrCore
org.apache.solr.core.SolrCore@6e369bf4
[junit4] 2> 311770 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.Overseer Overseer (id=97112083287441426-127.0.0.1:46533_-n_0000000004)
closing
[junit4] 2> 311770 INFO
(OverseerStateUpdate-97112083287441426-127.0.0.1:46533_-n_0000000004)
[n:127.0.0.1:46533_ ] o.a.s.c.Overseer Overseer Loop exiting :
127.0.0.1:46533_
[junit4] 2> 311773 INFO (zkCallback-544-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 313273 WARN
(zkCallback-570-thread-2-processing-n:127.0.0.1:46533_) [n:127.0.0.1:46533_
] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to
ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 313280 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.AbstractConnector Stopped
ServerConnector@4a8d233e{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 313280 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.e.j.s.h.ContextHandler Stopped
o.e.j.s.ServletContextHandler@64ca511a{/,null,UNAVAILABLE}
[junit4] 2> 313286 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ChaosMonkey monkey: stop shard! 60734
[junit4] 2> 313287 INFO
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.ZkTestServer connecting to 127.0.0.1:46421 46421
[junit4] 2> 313390 INFO (Thread-643) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:46421 46421
[junit4] 2> 313398 WARN (Thread-643) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/c8n_2x2_commits/state.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2> 3 /solr/collections/c8n_1x3_commits/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 56 /solr/overseer/queue
[junit4] 2> 31 /solr/overseer/collection-queue-work
[junit4] 2> 21 /solr/overseer/queue-work
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2>
[junit4] 2> 313398 WARN
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SocketProxy Closing 1 connections to: http://127.0.0.1:56938/, target:
http://127.0.0.1:39512/
[junit4] 2> 313398 WARN
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SocketProxy Closing 3 connections to: http://127.0.0.1:46533/, target:
http://127.0.0.1:51149/
[junit4] 2> 313398 WARN
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SocketProxy Closing 1 connections to: http://127.0.0.1:60734/, target:
http://127.0.0.1:45614/
[junit4] 2> 313398 WARN
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SocketProxy Closing 3 connections to: http://127.0.0.1:37195/, target:
http://127.0.0.1:50096/
[junit4] 2> 313398 WARN
(TEST-LeaderInitiatedRecoveryOnCommitTest.test-seed#[978136A83E311B2D]) [ ]
o.a.s.c.SocketProxy Closing 4 connections to: http://127.0.0.1:40876/, target:
http://127.0.0.1:58341/
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=LeaderInitiatedRecoveryOnCommitTest -Dtests.method=test
-Dtests.seed=978136A83E311B2D -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=sq -Dtests.timezone=America/Indianapolis -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] ERROR 38.3s J1 | LeaderInitiatedRecoveryOnCommitTest.test <<<
[junit4] > Throwable #1:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=3086, name=SocketProxy-Response-58341:44485,
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
[junit4] > at
__randomizedtesting.SeedInfo.seed([978136A83E311B2D:1FD5097290CD76D5]:0)
[junit4] > Caused by: java.lang.RuntimeException:
java.net.SocketException: Socket is closed
[junit4] > at
__randomizedtesting.SeedInfo.seed([978136A83E311B2D]:0)
[junit4] > at
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:347)
[junit4] > Caused by: java.net.SocketException: Socket is closed
[junit4] > at java.net.Socket.setSoTimeout(Socket.java:1137)
[junit4] > at
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:344)
[junit4] 2> 313402 INFO
(SUITE-LeaderInitiatedRecoveryOnCommitTest-seed#[978136A83E311B2D]-worker) [
] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> Dec 15, 2016 2:40:32 PM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked
thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {},
docValues:{}, maxPointsInLeafNode=1064, maxMBSortInHeap=5.690530284068884,
sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=sq,
timezone=America/Indianapolis
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation
1.8.0_102 (64-bit)/cpus=4,threads=1,free=89609384,total=435159040
[junit4] 2> NOTE: All tests run in this JVM: [TestBackupRepositoryFactory,
TestJettySolrRunner, TestWordDelimiterFilterFactory, JsonLoaderTest,
TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory,
BaseCdcrDistributedZkTest, TestSystemIdResolver, StatsReloadRaceTest,
CollectionReloadTest, TestTrie, TermsComponentTest, DocValuesTest,
TestElisionMultitermQuery, TestFieldResource, TestComplexPhraseQParserPlugin,
HdfsRestartWhileUpdatingTest, AddSchemaFieldsUpdateProcessorFactoryTest,
VMParamsZkACLAndCredentialsProvidersTest, TestNumericTerms32,
DirectSolrConnectionTest, ReplicationFactorTest, WordBreakSolrSpellCheckerTest,
TestTrackingShardHandlerFactory, FacetPivotSmallTest, SparseHLLTest,
TestClusterProperties, TestBM25SimilarityFactory,
OverseerCollectionConfigSetProcessorTest, ExitableDirectoryReaderTest,
TestAuthenticationFramework, TestRequestForwarding, TestCharFilters,
OpenCloseCoreStressTest, TestMacros, TestSegmentSorting, TestManagedSchema,
TolerantUpdateProcessorTest, SystemInfoHandlerTest,
TestExclusionRuleCollectionAccess, SampleTest, TestQueryTypes,
TestFuzzyAnalyzedSuggestions, BigEndianAscendingWordSerializerTest,
TestSQLHandlerNonCloud, TestTolerantUpdateProcessorRandomCloud,
SolrCLIZkUtilsTest, TestSimpleQParserPlugin, FileUtilsTest, TestLRUCache,
CachingDirectoryFactoryTest, DateMathParserTest, URLClassifyProcessorTest,
TestSubQueryTransformerCrossCore, DistributedQueryComponentCustomSortTest,
SolrIndexSplitterTest, TestJsonFacetRefinement, SortByFunctionTest,
TestNoOpRegenerator, DocValuesNotIndexedTest, DeleteNodeTest,
TestSolrQueryResponse, LeaderInitiatedRecoveryOnCommitTest]
[junit4] Completed [113/662 (1!)] on J1 in 39.03s, 1 test, 1 error <<<
FAILURES!
[...truncated 64209 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
