Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22557/
Java: 64bit/jdk-10 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir
Error Message:
Captured an uncaught exception in thread: Thread[id=33769,
name=cdcr-replicator-10553-thread-1, state=RUNNABLE,
group=TGRP-CdcrBidirectionalTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=33769, name=cdcr-replicator-10553-thread-1,
state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Caused by: java.lang.AssertionError: 1607369260134301696 != 1607369260133253120
at __randomizedtesting.SeedInfo.seed([5F99FCF1DFE0BC02]:0)
at
org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611)
at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125)
at
org.apache.solr.handler.CdcrReplicatorScheduler.lambda$start$0(CdcrReplicatorScheduler.java:81)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 14040 lines...]
[junit4] Suite: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
[junit4] 2> 1964240 INFO
(SUITE-CdcrBidirectionalTest-seed#[5F99FCF1DFE0BC02]-worker) [ ]
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/init-core-data-001
[junit4] 2> 1964241 INFO
(SUITE-CdcrBidirectionalTest-seed#[5F99FCF1DFE0BC02]-worker) [ ]
o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false)
w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1964241 INFO
(SUITE-CdcrBidirectionalTest-seed#[5F99FCF1DFE0BC02]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via:
@org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0,
clientAuth=0.0/0.0)
[junit4] 2> 1964242 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testBiDir
[junit4] 2> 1964242 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster2-001
[junit4] 2> 1964242 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1964242 INFO (Thread-6071) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1964242 INFO (Thread-6071) [ ] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 1964243 ERROR (Thread-6071) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 1964342 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.ZkTestServer start zk server on port:43833
[junit4] 2> 1964344 INFO (zkConnectionManagerCallback-10893-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964347 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git:
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10+46
[junit4] 2> 1964348 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1964348 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1964348 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1964348 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@41538b9f{/solr,null,AVAILABLE}
[junit4] 2> 1964349 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.AbstractConnector Started ServerConnector@28a42211{SSL,[ssl,
http/1.1]}{127.0.0.1:46357}
[junit4] 2> 1964349 INFO (jetty-launcher-10890-thread-1) [ ]
o.e.j.s.Server Started @1964376ms
[junit4] 2> 1964349 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
hostPort=46357}
[junit4] 2> 1964350 ERROR (jetty-launcher-10890-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 1964350 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.s.SolrDispatchFilter Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1964350 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
8.0.0
[junit4] 2> 1964350 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1964350 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1964350 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2018-07-29T23:26:01.692151Z
[junit4] 2> 1964351 INFO (zkConnectionManagerCallback-10895-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964351 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1964409 INFO (jetty-launcher-10890-thread-1) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43833/solr
[junit4] 2> 1964409 INFO (zkConnectionManagerCallback-10899-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964411 INFO (zkConnectionManagerCallback-10901-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964443 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.c.OverseerElectionContext I am going to be
the leader 127.0.0.1:46357_solr
[junit4] 2> 1964443 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.c.Overseer Overseer
(id=72071294106206211-127.0.0.1:46357_solr-n_0000000000) starting
[junit4] 2> 1964446 INFO (zkConnectionManagerCallback-10908-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964447 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster
at 127.0.0.1:43833/solr ready
[junit4] 2> 1964447 INFO
(OverseerStateUpdate-72071294106206211-127.0.0.1:46357_solr-n_0000000000) [
] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46357_solr
[junit4] 2> 1964447 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:46357_solr
[junit4] 2> 1964447 INFO (zkCallback-10907-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1964447 INFO (zkCallback-10900-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1964457 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.h.a.MetricsHistoryHandler No .system
collection, keeping metrics history in memory.
[junit4] 2> 1964469 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_46357.solr.node' (registry 'solr.node') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1964475 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_46357.solr.jvm' (registry 'solr.jvm') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1964475 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_46357.solr.jetty' (registry 'solr.jetty') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1964475 INFO (jetty-launcher-10890-thread-1)
[n:127.0.0.1:46357_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster2-001/node1/.
[junit4] 2> 1964483 INFO (zkConnectionManagerCallback-10911-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964485 INFO (zkConnectionManagerCallback-10914-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964485 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster1-001
[junit4] 2> 1964486 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1964486 INFO (Thread-6081) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1964486 INFO (Thread-6081) [ ] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 1964487 ERROR (Thread-6081) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 1964586 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.ZkTestServer start zk server on port:38141
[junit4] 2> 1964588 INFO (zkConnectionManagerCallback-10918-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964591 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid
0x1000c75cca80000, likely client has closed socket
[junit4] 2> 1964592 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git:
d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10+46
[junit4] 2> 1964592 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1964592 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1964592 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1964592 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@73f831f6{/solr,null,AVAILABLE}
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.AbstractConnector Started ServerConnector@532ce48b{SSL,[ssl,
http/1.1]}{127.0.0.1:37179}
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.e.j.s.Server Started @1964619ms
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
hostPort=37179}
[junit4] 2> 1964593 ERROR (jetty-launcher-10915-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.s.SolrDispatchFilter Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version
8.0.0
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1964593 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2018-07-29T23:26:01.935353Z
[junit4] 2> 1964594 INFO (zkConnectionManagerCallback-10920-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964594 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1964672 INFO (jetty-launcher-10915-thread-1) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38141/solr
[junit4] 2> 1964679 INFO (zkConnectionManagerCallback-10924-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964680 INFO (zkConnectionManagerCallback-10926-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964710 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.c.OverseerElectionContext I am going to be
the leader 127.0.0.1:37179_solr
[junit4] 2> 1964710 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.c.Overseer Overseer
(id=72071294122196995-127.0.0.1:37179_solr-n_0000000000) starting
[junit4] 2> 1964713 INFO (zkConnectionManagerCallback-10933-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964714 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster
at 127.0.0.1:38141/solr ready
[junit4] 2> 1964714 INFO
(OverseerStateUpdate-72071294122196995-127.0.0.1:37179_solr-n_0000000000) [
] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37179_solr
[junit4] 2> 1964714 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:37179_solr
[junit4] 2> 1964715 INFO
(OverseerStateUpdate-72071294122196995-127.0.0.1:37179_solr-n_0000000000) [
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1964715 INFO (zkCallback-10932-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1964722 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.h.a.MetricsHistoryHandler No .system
collection, keeping metrics history in memory.
[junit4] 2> 1964732 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_37179.solr.node' (registry 'solr.node') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1964736 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_37179.solr.jvm' (registry 'solr.jvm') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1964736 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_37179.solr.jetty' (registry 'solr.jetty') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1964737 INFO (jetty-launcher-10915-thread-1)
[n:127.0.0.1:37179_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster1-001/node1/.
[junit4] 2> 1964749 INFO (zkConnectionManagerCallback-10936-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964752 INFO (zkConnectionManagerCallback-10939-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964753 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.c.CdcrBidirectionalTest cluster2 zkHost = 127.0.0.1:43833/solr
[junit4] 2> 1964753 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.c.CdcrBidirectionalTest cluster1 zkHost = 127.0.0.1:38141/solr
[junit4] 2> 1964754 INFO (zkConnectionManagerCallback-10941-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964756 INFO (zkConnectionManagerCallback-10945-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1964756 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1964757 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38141/solr ready
[junit4] 2> 1964765 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 1964766 INFO (OverseerThreadFactory-10514-thread-1) [ ]
o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster1
[junit4] 2> 1964872 INFO
(OverseerStateUpdate-72071294122196995-127.0.0.1:37179_solr-n_0000000000) [
] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-cluster1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37179/solr";,
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1964872 INFO
(OverseerStateUpdate-72071294122196995-127.0.0.1:37179_solr-n_0000000000) [
] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"cdcr-cluster1_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37179/solr";,
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1965076 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create
command
qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1965076 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create
command
qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1965076 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.TransientSolrCoreCacheDefault
Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1966085 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1966085 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1966098 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.IndexSchema [cdcr-cluster1_shard2_replica_n2] Schema name=minimal
[junit4] 2> 1966098 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.IndexSchema [cdcr-cluster1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1966099 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1966099 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1966099 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard1_replica_n1' using
configuration from collection cdcr-cluster1, trusted=true
[junit4] 2> 1966099 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard2_replica_n2' using
configuration from collection cdcr-cluster1, trusted=true
[junit4] 2> 1966099 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_37179.solr.core.cdcr-cluster1.shard1.replica_n1' (registry
'solr.core.cdcr-cluster1.shard1.replica_n1') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1966099 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_37179.solr.core.cdcr-cluster1.shard2.replica_n2' (registry
'solr.core.cdcr-cluster1.shard2.replica_n2') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1966099 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1966099 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1966099 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.SolrCore [[cdcr-cluster1_shard1_replica_n1] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster1-001/node1/cdcr-cluster1_shard1_replica_n1],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster1-001/node1/./cdcr-cluster1_shard1_replica_n1/data/]
[junit4] 2> 1966099 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.SolrCore [[cdcr-cluster1_shard2_replica_n2] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster1-001/node1/cdcr-cluster1_shard2_replica_n2],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster1-001/node1/./cdcr-cluster1_shard2_replica_n2/data/]
[junit4] 2> 1966195 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.UpdateHandler Using UpdateLog implementation:
org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1966195 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1966195 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.UpdateHandler Using UpdateLog implementation:
org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1966195 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1966195 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1966195 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1966196 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1966196 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1966197 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@2ad5e12b[cdcr-cluster1_shard2_replica_n2] main]
[junit4] 2> 1966197 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@4cd84ad5[cdcr-cluster1_shard1_replica_n1] main]
[junit4] 2> 1966198 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/cdcr-cluster1
[junit4] 2> 1966198 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/cdcr-cluster1
[junit4] 2> 1966198 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/cdcr-cluster1
[junit4] 2> 1966198 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/cdcr-cluster1
[junit4] 2> 1966198 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at
/configs/cdcr-cluster1/managed-schema
[junit4] 2> 1966198 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at
/configs/cdcr-cluster1/managed-schema
[junit4] 2> 1966198 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1966198 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1966198 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1966198 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1966201 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.h.CdcrBufferStateManager Created znode
/collections/cdcr-cluster1/cdcr/state/buffer
[junit4] 2> 1966203 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.h.CdcrProcessStateManager Created znode
/collections/cdcr-cluster1/cdcr/state/process
[junit4] 2> 1966213 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using
new clock 1607369242501447680
[junit4] 2> 1966214 INFO
(searcherExecutor-10519-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore
[cdcr-cluster1_shard1_replica_n1] Registered new searcher
Searcher@4cd84ad5[cdcr-cluster1_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1966214 INFO
(searcherExecutor-10520-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard2 r:core_node4
x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore
[cdcr-cluster1_shard2_replica_n2] Registered new searcher
Searcher@2ad5e12b[cdcr-cluster1_shard2_replica_n2]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1966214 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using
new clock 1607369242502496256
[junit4] 2> 1966217 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ZkShardTerms Successful update of terms at
/collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=0},
version=0}
[junit4] 2> 1966218 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ZkShardTerms Successful update of terms at
/collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=0},
version=0}
[junit4] 2> 1966218 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1966218 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1966218 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.SyncStrategy Sync replicas to
https://127.0.0.1:37179/solr/cdcr-cluster1_shard2_replica_n2/
[junit4] 2> 1966219 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1966219 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.SyncStrategy
https://127.0.0.1:37179/solr/cdcr-cluster1_shard2_replica_n2/ has no replicas
[junit4] 2> 1966219 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 1966219 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1966219 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1966219 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.SyncStrategy Sync replicas to
https://127.0.0.1:37179/solr/cdcr-cluster1_shard1_replica_n1/
[junit4] 2> 1966219 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1966219 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.SyncStrategy
https://127.0.0.1:37179/solr/cdcr-cluster1_shard1_replica_n1/ has no replicas
[junit4] 2> 1966219 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 1966220 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard2
[junit4] 2> 1966220 INFO (zkCallback-10925-thread-2) [ ]
o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard1
[junit4] 2> 1966221 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
https://127.0.0.1:37179/solr/cdcr-cluster1_shard2_replica_n2/ shard2
[junit4] 2> 1966222 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
https://127.0.0.1:37179/solr/cdcr-cluster1_shard1_replica_n1/ shard1
[junit4] 2> 1966323 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1966327 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin}
status=0 QTime=1251
[junit4] 2> 1966372 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1966379 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin}
status=0 QTime=1302
[junit4] 2> 1966381 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most
30 seconds. Check all shard replicas
[junit4] 2> 1966475 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/cdcr-cluster1/state.json] for collection [cdcr-cluster1] has
occurred - updating... (live nodes size: [1])
[junit4] 2> 1966768 INFO
(OverseerCollectionConfigSetProcessor-72071294122196995-127.0.0.1:37179_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 1967381 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
params={collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2}
status=0 QTime=2616
[junit4] 2> 1967383 INFO (zkConnectionManagerCallback-10949-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1967390 INFO (zkConnectionManagerCallback-10953-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1967391 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1967392 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43833/solr ready
[junit4] 2> 1967401 INFO (qtp2041494791-33574) [n:127.0.0.1:46357_solr
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 1967402 INFO (OverseerThreadFactory-10501-thread-1) [ ]
o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster2
[junit4] 2> 1967506 INFO
(OverseerStateUpdate-72071294106206211-127.0.0.1:46357_solr-n_0000000000) [
] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-cluster2_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46357/solr";,
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1967507 INFO
(OverseerStateUpdate-72071294106206211-127.0.0.1:46357_solr-n_0000000000) [
] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"cdcr-cluster2_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46357/solr";,
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1967725 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create
command
qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1967725 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create
command
qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1967726 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault
Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1968733 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1968733 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1968737 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.s.IndexSchema [cdcr-cluster2_shard2_replica_n2] Schema name=minimal
[junit4] 2> 1968737 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.s.IndexSchema [cdcr-cluster2_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1968738 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1968738 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1968738 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard2_replica_n2' using
configuration from collection cdcr-cluster2, trusted=true
[junit4] 2> 1968738 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard1_replica_n1' using
configuration from collection cdcr-cluster2, trusted=true
[junit4] 2> 1968738 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_46357.solr.core.cdcr-cluster2.shard2.replica_n2' (registry
'solr.core.cdcr-cluster2.shard2.replica_n2') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1968738 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.m.r.SolrJmxReporter JMX monitoring for
'solr_46357.solr.core.cdcr-cluster2.shard1.replica_n1' (registry
'solr.core.cdcr-cluster2.shard1.replica_n1') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@227394fc
[junit4] 2> 1968738 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1968738 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1968738 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.SolrCore [[cdcr-cluster2_shard2_replica_n2] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster2-001/node1/cdcr-cluster2_shard2_replica_n2],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster2-001/node1/./cdcr-cluster2_shard2_replica_n2/data/]
[junit4] 2> 1968738 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.SolrCore [[cdcr-cluster2_shard1_replica_n1] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster2-001/node1/cdcr-cluster2_shard1_replica_n1],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.cdcr.CdcrBidirectionalTest_5F99FCF1DFE0BC02-001/cdcr-cluster2-001/node1/./cdcr-cluster2_shard1_replica_n1/data/]
[junit4] 2> 1968827 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.u.UpdateHandler Using UpdateLog implementation:
org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1968827 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1968828 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1968828 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1968829 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@45c1a138[cdcr-cluster2_shard1_replica_n1] main]
[junit4] 2> 1968830 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/cdcr-cluster2
[junit4] 2> 1968831 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/cdcr-cluster2
[junit4] 2> 1968831 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at
/configs/cdcr-cluster2/managed-schema
[junit4] 2> 1968831 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1968832 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1968832 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.CdcrBufferStateManager Created znode
/collections/cdcr-cluster2/cdcr/state/buffer
[junit4] 2> 1968833 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.CdcrProcessStateManager Created znode
/collections/cdcr-cluster2/cdcr/state/process
[junit4] 2> 1968834 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using
new clock 1607369245249765376
[junit4] 2> 1968834 INFO
(searcherExecutor-10532-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore
[cdcr-cluster2_shard1_replica_n1] Registered new searcher
Searcher@45c1a138[cdcr-cluster2_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1968836 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.ZkShardTerms Successful update of terms at
/collections/cdcr-cluster2/terms/shard1 to Terms{values={core_node3=0},
version=0}
[junit4] 2> 1968838 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1968838 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1968838 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.SyncStrategy Sync replicas to
https://127.0.0.1:46357/solr/cdcr-cluster2_shard1_replica_n1/
[junit4] 2> 1968838 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1968838 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.SyncStrategy
https://127.0.0.1:46357/solr/cdcr-cluster2_shard1_replica_n1/ has no replicas
[junit4] 2> 1968838 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 1968839 INFO (zkCallback-10900-thread-1) [ ]
o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard1
[junit4] 2> 1968840 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
https://127.0.0.1:46357/solr/cdcr-cluster2_shard1_replica_n1/ shard1
[junit4] 2> 1968842 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.u.UpdateHandler Using UpdateLog implementation:
org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1968842 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1968843 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1968843 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1968845 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@1cf5d124[cdcr-cluster2_shard2_replica_n2] main]
[junit4] 2> 1968846 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/cdcr-cluster2
[junit4] 2> 1968847 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/cdcr-cluster2
[junit4] 2> 1968847 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at
/configs/cdcr-cluster2/managed-schema
[junit4] 2> 1968847 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1968848 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1968850 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using
new clock 1607369245266542592
[junit4] 2> 1968850 INFO
(searcherExecutor-10531-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore
[cdcr-cluster2_shard2_replica_n2] Registered new searcher
Searcher@1cf5d124[cdcr-cluster2_shard2_replica_n2]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1968853 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.ZkShardTerms Successful update of terms at
/collections/cdcr-cluster2/terms/shard2 to Terms{values={core_node4=0},
version=0}
[junit4] 2> 1968854 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1968854 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1968854 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.SyncStrategy Sync replicas to
https://127.0.0.1:46357/solr/cdcr-cluster2_shard2_replica_n2/
[junit4] 2> 1968854 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1968854 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.SyncStrategy
https://127.0.0.1:46357/solr/cdcr-cluster2_shard2_replica_n2/ has no replicas
[junit4] 2> 1968854 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 1968855 INFO (zkCallback-10900-thread-1) [ ]
o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard2
[junit4] 2> 1968856 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
https://127.0.0.1:46357/solr/cdcr-cluster2_shard2_replica_n2/ shard2
[junit4] 2> 1968992 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1968993 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin}
status=0 QTime=1267
[junit4] 2> 1969007 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1969009 INFO (qtp2041494791-33572) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin}
status=0 QTime=1283
[junit4] 2> 1969010 INFO (qtp2041494791-33574) [n:127.0.0.1:46357_solr
] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most
30 seconds. Check all shard replicas
[junit4] 2> 1969109 INFO (zkCallback-10900-thread-2) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/cdcr-cluster2/state.json] for collection [cdcr-cluster2] has
occurred - updating... (live nodes size: [1])
[junit4] 2> 1969404 INFO
(OverseerCollectionConfigSetProcessor-72071294106206211-127.0.0.1:46357_solr-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 1970011 INFO (qtp2041494791-33574) [n:127.0.0.1:46357_solr
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
params={collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2}
status=0 QTime=2609
[junit4] 2> 1970014 INFO (zkConnectionManagerCallback-10960-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1970016 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1970016 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43833/solr ready
[junit4] 2> 1970021 INFO (qtp2041494791-33576) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1970021 INFO (qtp2041494791-33705) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1970021 INFO (qtp2041494791-33576) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1970021 INFO (qtp2041494791-33705) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1970022 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2}
status=0 QTime=4
[junit4] 2> 1970022 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.h.CdcrReplicatorManager Create new update log reader for target
cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard1
[junit4] 2> 1970022 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection:
cdcr-cluster2, shard: shard1
[junit4] 2> 1970023 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 1970024 INFO
(cdcr-bootstrap-status-10956-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to
bootstrap target collection: cdcr-cluster2 shard: shard1 leader:
https://127.0.0.1:46357/solr/cdcr-cluster2_shard1_replica_n1/
[junit4] 2> 1970025 INFO (zkCallback-10925-thread-2) [ ]
o.a.s.h.CdcrProcessStateManager The CDCR process state has changed:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard1
[junit4] 2> 1970025 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.h.CdcrProcessStateManager The CDCR process state has changed:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard2
[junit4] 2> 1970025 INFO (zkCallback-10925-thread-2) [ ]
o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher:
STARTED @ cdcr-cluster1:shard1
[junit4] 2> 1970025 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher:
STARTED @ cdcr-cluster1:shard2
[junit4] 2> 1970026 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/cdcr
params={qt=/cdcr&_stateVer_=cdcr-cluster1:4&action=start&wt=javabin&version=2}
status=0 QTime=13
[junit4] 2> 1970026 INFO (qtp2041494791-33574) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={qt=/cdcr&masterUrl=https://127.0.0.1:37179/solr/cdcr-cluster1_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2}
status=0 QTime=1
[junit4] 2> 1970027 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1970028 INFO
(cdcr-bootstrap-status-10956-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap
running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 1970030 INFO (zkConnectionManagerCallback-10965-thread-1) [
] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1970031 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Starting to buffer
updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1970031 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1970032 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43833/solr ready
[junit4] 2> 1970036 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1607369246510153728,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1970038 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1970038 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1970038 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update
params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}
status=0 QTime=2
[junit4] 2> 1970040 INFO (qtp2041494791-33569) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1970040 INFO (qtp2041494791-33569) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1970041 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr
path=/replication
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0
QTime=0
[junit4] 2> 1970041 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 1970041 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 1970041 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1970041 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 1970041 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher New index in Master.
Deleting mine...
[junit4] 2> 1970043 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening
[Searcher@5cfbd222[cdcr-cluster2_shard1_replica_n1] main]
[junit4] 2> 1970044 INFO (qtp2041494791-33574) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1970044 INFO (qtp2041494791-33574) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1970044 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2}
status=0 QTime=10
[junit4] 2> 1970044 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager Create new update log reader for target
cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard2
[junit4] 2> 1970044 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection:
cdcr-cluster2, shard: shard2
[junit4] 2> 1970044 INFO (zkCallback-10925-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 1970044 INFO
(searcherExecutor-10532-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore
[cdcr-cluster2_shard1_replica_n1] Registered new searcher
Searcher@5cfbd222[cdcr-cluster2_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1970044 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard1 r:core_node3
x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler No replay needed.
[junit4] 2> 1970045 INFO (cdcr-bootstrap-status-10961-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection:
cdcr-cluster2 shard: shard2 leader:
https://127.0.0.1:46357/solr/cdcr-cluster2_shard2_replica_n2/
[junit4] 2> 1970047 INFO (qtp2041494791-33570) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={qt=/cdcr&masterUrl=https://127.0.0.1:37179/solr/cdcr-cluster1_shard2_replica_n2/&action=BOOTSTRAP&wt=javabin&version=2}
status=0 QTime=0
[junit4] 2> 1970047 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Starting to buffer
updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1970048 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1970048 INFO (cdcr-bootstrap-status-10961-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping
for 2000 ms
[junit4] 2> 1970052 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1607369246526930944,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1970052 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1970052 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1970052 INFO (qtp1190622519-33680) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update
params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}
status=0 QTime=0
[junit4] 2> 1970053 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr
path=/replication
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0
QTime=0
[junit4] 2> 1970053 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 1970053 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 1970053 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1970053 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 1970053 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher New index in Master.
Deleting mine...
[junit4] 2> 1970054 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening
[Searcher@24fb8f8a[cdcr-cluster2_shard2_replica_n2] main]
[junit4] 2> 1970055 INFO
(searcherExecutor-10531-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore
[cdcr-cluster2_shard2_replica_n2] Registered new searcher
Searcher@24fb8f8a[cdcr-cluster2_shard2_replica_n2]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1970055 INFO
(recoveryExecutor-10897-thread-1-processing-n:127.0.0.1:46357_solr
x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4)
[n:127.0.0.1:46357_solr c:cdcr-cluster2 s:shard2 r:core_node4
x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler No replay needed.
[junit4] 2> 1972026 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.c.CdcrBidirectionalTest Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 1972028 INFO (qtp2041494791-33569) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1972028 INFO
(cdcr-bootstrap-status-10956-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap
successful in 3 seconds
[junit4] 2> 1972031 INFO (qtp1190622519-33624) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.ZkShardTerms Successful update of terms at
/collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=1},
version=1}
[junit4] 2> 1972031 INFO (qtp1190622519-33624) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update
params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 1972032 INFO (qtp2041494791-33575) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1972032 INFO (qtp2041494791-33575) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1972032 INFO (qtp2041494791-33576) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1972032 INFO (qtp2041494791-33576) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1972032 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2}
status=0 QTime=3
[junit4] 2> 1972032 INFO
(cdcr-bootstrap-status-10956-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new
update log reader for target cdcr-cluster2 with checkpoint -1 @
cdcr-cluster1:shard1
[junit4] 2> 1972033 INFO
(cdcr-bootstrap-status-10956-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Bootstrap
successful, giving the go-ahead to replicator
[junit4] 2> 1972038 INFO (qtp1190622519-33681) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.ZkShardTerms Successful update of terms at
/collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=1},
version=1}
[junit4] 2> 1972038 INFO (qtp1190622519-33681) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update
params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 1972049 INFO (qtp2041494791-33576) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1972049 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1607369248620937216,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1972049 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1607369248620937216,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1972049 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@10e6c837
commitCommandVersion:1607369248620937216
[junit4] 2> 1972049 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@cb8637c
commitCommandVersion:1607369248620937216
[junit4] 2> 1972049 INFO (cdcr-bootstrap-status-10961-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager CDCR bootstrap successful in 3 seconds
[junit4] 2> 1972059 INFO (qtp2041494791-33570) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1972059 INFO (qtp2041494791-33570) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1972060 INFO (qtp2041494791-33575) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1972060 INFO (qtp2041494791-33575) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1972060 INFO (qtp2041494791-33704) [n:127.0.0.1:46357_solr
c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr
params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2}
status=0 QTime=11
[junit4] 2> 1972060 INFO (cdcr-bootstrap-status-10961-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager Create new update log reader for target
cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard2
[junit4] 2> 1972061 INFO (cdcr-bootstrap-status-10961-thread-1) [ ]
o.a.s.h.CdcrReplicatorManager Bootstrap successful, giving the go-ahead to
replicator
[junit4] 2> 1972163 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@21a9d9ef[cdcr-cluster1_shard1_replica_n1] main]
[junit4] 2> 1972164 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1972164 INFO
(searcherExecutor-10519-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard1 r:core_node3
x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore
[cdcr-cluster1_shard1_replica_n1] Registered new searcher
Searcher@21a9d9ef[cdcr-cluster1_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C43)))}
[junit4] 2> 1972164 INFO (qtp1190622519-33625) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update
params={update.distrib=FROMLEADER&update.chain=cdcr-processor-chain&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:37179/solr/cdcr-cluster1_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
status=0 QTime=116
[junit4] 2> 1972168 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@7cef242d[cdcr-cluster1_shard2_replica_n2] main]
[junit4] 2> 1972168 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1972168 INFO
(searcherExecutor-10520-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4)
[n:127.0.0.1:37179_solr c:cdcr-cluster1 s:shard2 r:core_node4
x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore
[cdcr-cluster1_shard2_replica_n2] Registered new searcher
Searcher@7cef242d[cdcr-cluster1_shard2_replica_n2]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C57)))}
[junit4] 2> 1972169 INFO (qtp1190622519-33629) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update
params={update.distrib=FROMLEADER&update.chain=cdcr-processor-chain&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:37179/solr/cdcr-cluster1_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
status=0 QTime=120
[junit4] 2> 1972169 INFO (qtp1190622519-33630) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update
params={_stateVer_=cdcr-cluster1:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
status=0 QTime=129
[junit4] 2> 1972169 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[5F99FCF1DFE0BC02]) [ ]
o.a.s.c.c.CdcrBidirectionalTest Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 1972172 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update
params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 1972172 INFO (qtp1190622519-33681) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update
params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 1972174 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1607369248752009216,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1972174 INFO (qtp1190622519-33624) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1607369248752009216,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1972174 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@cb8637c
commitCommandVersion:1607369248752009216
[junit4] 2> 1972174 INFO (qtp1190622519-33624) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@10e6c837
commitCommandVersion:1607369248752009216
[junit4] 2> 1972325 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.s.SolrIndexSearcher Opening
[Searcher@50429059[cdcr-cluster1_shard2_replica_n2] main]
[junit4] 2> 1972325 INFO (qtp1190622519-33628) [n:127.0.0.1:37179_solr
c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1972326 INFO
(searcherExecutor-10520-thread-1-processing-n:127.0.0.1:37179_solr
x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2
[...truncated too long message...]
cheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/byte-buddy-1.6.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/commons-collections-3.2.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/commons-math3-3.6.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/mockito-core-2.6.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/netty-all-4.0.36.Final.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/objenesis-2.5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queryparser/classes/test:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/backward-codecs/classes/test:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/contrib/analysis-extras/lib/icu4j-62.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/home/jenkins/.ant/lib/ivy-2.4.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/home/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.6.0.jar
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J2-20180729_225317_2917825527145690212039.events
@/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J2-20180729_225317_2918260046903010531044.suites
-stdin
[junit4] ERROR: JVM J2 ended with an exception: Forked process returned with
error code: 134. Very likely a JVM crash. See process stdout at:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J2-20180729_225317_2912101277426885607835.sysout
[junit4] at
com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at
com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[junit4] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[junit4] at java.base/java.lang.Thread.run(Thread.java:844)
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:633: The following
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:577: The following
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:59: The following
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build.xml:267: The
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/common-build.xml:556: The
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1568:
The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092:
At least one slave process threw an exception, first: Forked process returned
with error code: 134. Very likely a JVM crash. See process stdout at:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J2-20180729_225317_2912101277426885607835.sysout
Total time: 65 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
