[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-10.0.1) - Build # 3095 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3095/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseParallelGC

11 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:38451_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/12)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",           
"base_url":"http://127.0.0.1:38451/solr";,           
"node_name":"127.0.0.1:38451_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265945388056525"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"hdfsbackuprestore_restored_shard1_1_replica_n63",           
"base_url":"http://127.0.0.1:38451/solr";,           
"node_name":"127.0.0.1:38451_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265945388093775"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"hdfsbackuprestore_restored_shard1_0_replica_n65",           
"base_url":"http://127.0.0.1:38451/solr";,           
"node_name":"127.0.0.1:38451_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265945388123048"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:38451_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/12)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",
          "base_url":"http://127.0.0.1:38451/solr";,
          "node_name":"127.0.0.1:38451_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265945388056525"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"hdfsbackuprestore_restored_shard1_1_replica_n63",
          "base_url":"http://127.0.0.1:38451/solr";,
          "node_name":"127.0.0.1:38451_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265945388093775"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"hdfsbackuprestore_restored_shard1_0_replica_n65",
          "base_url":"http://127.0.0.1:38451/solr";,
          "node_name":"127.0.0.1:38451_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265945388123048"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test(TestHdfsCloudBackupRestore.java:213)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:38105_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/12)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",           
"base_url":"http://127.0.0.1:38105/solr";,           
"node_name":"127.0.0.1:38105_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265956818476689"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"hdfsbackuprestore_restored_shard1_1_replica_n63",           
"base_url":"http://127.0.0.1:38105/solr";,           
"node_name":"127.0.0.1:38105_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265956818515377"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"hdfsbackuprestore_restored_shard1_0_replica_n65",           
"base_url":"http://127.0.0.1:38105/solr";,           
"node_name":"127.0.0.1:38105_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265956818545589"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:38105_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/12)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",
          "base_url":"http://127.0.0.1:38105/solr";,
          "node_name":"127.0.0.1:38105_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265956818476689"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"hdfsbackuprestore_restored_shard1_1_replica_n63",
          "base_url":"http://127.0.0.1:38105/solr";,
          "node_name":"127.0.0.1:38105_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265956818515377"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"hdfsbackuprestore_restored_shard1_0_replica_n65",
          "base_url":"http://127.0.0.1:38105/solr";,
          "node_name":"127.0.0.1:38105_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265956818545589"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test(TestHdfsCloudBackupRestore.java:213)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:41321_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/11)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",           
"base_url":"http://127.0.0.1:41321/solr";,           
"node_name":"127.0.0.1:41321_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265982814062346"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"hdfsbackuprestore_restored_shard1_1_replica_n63",           
"base_url":"http://127.0.0.1:41321/solr";,           
"node_name":"127.0.0.1:41321_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265982814085521"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"hdfsbackuprestore_restored_shard1_0_replica_n65",           
"base_url":"http://127.0.0.1:41321/solr";,           
"node_name":"127.0.0.1:41321_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265982814103467"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:41321_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/11)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",
          "base_url":"http://127.0.0.1:41321/solr";,
          "node_name":"127.0.0.1:41321_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265982814062346"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"hdfsbackuprestore_restored_shard1_1_replica_n63",
          "base_url":"http://127.0.0.1:41321/solr";,
          "node_name":"127.0.0.1:41321_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265982814085521"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"hdfsbackuprestore_restored_shard1_0_replica_n65",
          "base_url":"http://127.0.0.1:41321/solr";,
          "node_name":"127.0.0.1:41321_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265982814103467"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test(TestHdfsCloudBackupRestore.java:213)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:40289_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/12)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",           
"base_url":"http://127.0.0.1:40289/solr";,           
"node_name":"127.0.0.1:40289_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265983046439427"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"hdfsbackuprestore_restored_shard1_1_replica_n63",           
"base_url":"http://127.0.0.1:40289/solr";,           
"node_name":"127.0.0.1:40289_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265983046481012"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"hdfsbackuprestore_restored_shard1_0_replica_n65",           
"base_url":"http://127.0.0.1:40289/solr";,           
"node_name":"127.0.0.1:40289_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265983046506317"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:40289_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/12)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",
          "base_url":"http://127.0.0.1:40289/solr";,
          "node_name":"127.0.0.1:40289_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265983046439427"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"hdfsbackuprestore_restored_shard1_1_replica_n63",
          "base_url":"http://127.0.0.1:40289/solr";,
          "node_name":"127.0.0.1:40289_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265983046481012"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"hdfsbackuprestore_restored_shard1_0_replica_n65",
          "base_url":"http://127.0.0.1:40289/solr";,
          "node_name":"127.0.0.1:40289_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265983046506317"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test(TestHdfsCloudBackupRestore.java:213)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35683_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/11)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",           
"base_url":"http://127.0.0.1:35683/solr";,           
"node_name":"127.0.0.1:35683_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542264615786561495"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"hdfsbackuprestore_restored_shard1_1_replica_n63",           
"base_url":"http://127.0.0.1:35683/solr";,           
"node_name":"127.0.0.1:35683_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542264615786580102"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"hdfsbackuprestore_restored_shard1_0_replica_n65",           
"base_url":"http://127.0.0.1:35683/solr";,           
"node_name":"127.0.0.1:35683_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542264615786596125"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35683_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(hdfsbackuprestore_restored//collections/hdfsbackuprestore_restored/state.json/11)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"hdfsbackuprestore_restored_shard2_replica_n61",
          "base_url":"http://127.0.0.1:35683/solr";,
          "node_name":"127.0.0.1:35683_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542264615786561495"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"hdfsbackuprestore_restored_shard1_1_replica_n63",
          "base_url":"http://127.0.0.1:35683/solr";,
          "node_name":"127.0.0.1:35683_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542264615786580102"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"hdfsbackuprestore_restored_shard1_0_replica_n65",
          "base_url":"http://127.0.0.1:35683/solr";,
          "node_name":"127.0.0.1:35683_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542264615786596125"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test(TestHdfsCloudBackupRestore.java:213)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:40471_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"backuprestore_restored_shard2_replica_n61",           
"base_url":"https://127.0.0.1:40471/solr";,           
"node_name":"127.0.0.1:40471_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265967862948796"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"backuprestore_restored_shard1_1_replica_n63",           
"base_url":"https://127.0.0.1:40471/solr";,           
"node_name":"127.0.0.1:40471_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265967862987983"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"backuprestore_restored_shard1_0_replica_n65",           
"base_url":"https://127.0.0.1:40471/solr";,           
"node_name":"127.0.0.1:40471_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265967863018246"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:40471_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"backuprestore_restored_shard2_replica_n61",
          "base_url":"https://127.0.0.1:40471/solr";,
          "node_name":"127.0.0.1:40471_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265967862948796"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"backuprestore_restored_shard1_1_replica_n63",
          "base_url":"https://127.0.0.1:40471/solr";,
          "node_name":"127.0.0.1:40471_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265967862987983"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"backuprestore_restored_shard1_0_replica_n65",
          "base_url":"https://127.0.0.1:40471/solr";,
          "node_name":"127.0.0.1:40471_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265967863018246"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:38607_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"backuprestore_restored_shard2_replica_n61",           
"base_url":"https://127.0.0.1:38607/solr";,           
"node_name":"127.0.0.1:38607_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265968599149488"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"backuprestore_restored_shard1_1_replica_n63",           
"base_url":"https://127.0.0.1:38607/solr";,           
"node_name":"127.0.0.1:38607_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265968599191569"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"backuprestore_restored_shard1_0_replica_n65",           
"base_url":"https://127.0.0.1:38607/solr";,           
"node_name":"127.0.0.1:38607_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265968599223132"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:38607_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"backuprestore_restored_shard2_replica_n61",
          "base_url":"https://127.0.0.1:38607/solr";,
          "node_name":"127.0.0.1:38607_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265968599149488"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"backuprestore_restored_shard1_1_replica_n63",
          "base_url":"https://127.0.0.1:38607/solr";,
          "node_name":"127.0.0.1:38607_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265968599191569"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"backuprestore_restored_shard1_0_replica_n65",
          "base_url":"https://127.0.0.1:38607/solr";,
          "node_name":"127.0.0.1:38607_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265968599223132"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35503_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"backuprestore_restored_shard2_replica_n61",           
"base_url":"https://127.0.0.1:35503/solr";,           
"node_name":"127.0.0.1:35503_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265982798866060"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"backuprestore_restored_shard1_1_replica_n63",           
"base_url":"https://127.0.0.1:35503/solr";,           
"node_name":"127.0.0.1:35503_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265982798902665"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"backuprestore_restored_shard1_0_replica_n65",           
"base_url":"https://127.0.0.1:35503/solr";,           
"node_name":"127.0.0.1:35503_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265982798925757"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35503_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"backuprestore_restored_shard2_replica_n61",
          "base_url":"https://127.0.0.1:35503/solr";,
          "node_name":"127.0.0.1:35503_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265982798866060"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"backuprestore_restored_shard1_1_replica_n63",
          "base_url":"https://127.0.0.1:35503/solr";,
          "node_name":"127.0.0.1:35503_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265982798902665"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"backuprestore_restored_shard1_0_replica_n65",
          "base_url":"https://127.0.0.1:35503/solr";,
          "node_name":"127.0.0.1:35503_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265982798925757"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:42365_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"backuprestore_restored_shard2_replica_n61",           
"base_url":"https://127.0.0.1:42365/solr";,           
"node_name":"127.0.0.1:42365_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265996233552252"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"backuprestore_restored_shard1_1_replica_n63",           
"base_url":"https://127.0.0.1:42365/solr";,           
"node_name":"127.0.0.1:42365_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265996233583183"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"backuprestore_restored_shard1_0_replica_n65",           
"base_url":"https://127.0.0.1:42365/solr";,           
"node_name":"127.0.0.1:42365_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265996233612092"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:42365_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"backuprestore_restored_shard2_replica_n61",
          "base_url":"https://127.0.0.1:42365/solr";,
          "node_name":"127.0.0.1:42365_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265996233552252"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"backuprestore_restored_shard1_1_replica_n63",
          "base_url":"https://127.0.0.1:42365/solr";,
          "node_name":"127.0.0.1:42365_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265996233583183"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"backuprestore_restored_shard1_0_replica_n65",
          "base_url":"https://127.0.0.1:42365/solr";,
          "node_name":"127.0.0.1:42365_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265996233612092"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35363_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/11)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"backuprestore_restored_shard2_replica_n61",           
"base_url":"https://127.0.0.1:35363/solr";,           
"node_name":"127.0.0.1:35363_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265937934565781"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"backuprestore_restored_shard1_1_replica_n63",           
"base_url":"https://127.0.0.1:35363/solr";,           
"node_name":"127.0.0.1:35363_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265937934619654"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"backuprestore_restored_shard1_0_replica_n65",           
"base_url":"https://127.0.0.1:35363/solr";,           
"node_name":"127.0.0.1:35363_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542265937934660216"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35363_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/11)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"backuprestore_restored_shard2_replica_n61",
          "base_url":"https://127.0.0.1:35363/solr";,
          "node_name":"127.0.0.1:35363_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265937934565781"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"backuprestore_restored_shard1_1_replica_n63",
          "base_url":"https://127.0.0.1:35363/solr";,
          "node_name":"127.0.0.1:35363_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265937934619654"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"backuprestore_restored_shard1_0_replica_n65",
          "base_url":"https://127.0.0.1:35363/solr";,
          "node_name":"127.0.0.1:35363_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542265937934660216"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35465_solr has 3 replicas. Expected num replicas : 2. state:  
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
   "pullReplicas":0,   "replicationFactor":1,   "shards":{     "shard2":{       
"range":"0-7fffffff",       "state":"active",       "replicas":{"core_node62":{ 
          "core":"backuprestore_restored_shard2_replica_n61",           
"base_url":"https://127.0.0.1:35465/solr";,           
"node_name":"127.0.0.1:35465_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542264620086216682"},     "shard1_1":{       
"range":"c0000000-ffffffff",       "state":"active",       
"replicas":{"core_node64":{           
"core":"backuprestore_restored_shard1_1_replica_n63",           
"base_url":"https://127.0.0.1:35465/solr";,           
"node_name":"127.0.0.1:35465_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542264620086268170"},     "shard1_0":{       
"range":"80000000-bfffffff",       "state":"active",       
"replicas":{"core_node66":{           
"core":"backuprestore_restored_shard1_0_replica_n65",           
"base_url":"https://127.0.0.1:35465/solr";,           
"node_name":"127.0.0.1:35465_solr",           "state":"active",           
"type":"NRT",           "force_set_state":"false",           "leader":"true"}}, 
      "stateTimestamp":"1542264620086290083"}},   "router":{     
"name":"compositeId",     "field":"shard_s"},   "maxShardsPerNode":"-1",   
"autoAddReplicas":"true",   "nrtReplicas":1,   "tlogReplicas":0}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35465_solr has 3 replicas. Expected 
num replicas : 2. state: 
DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/10)={
  "pullReplicas":0,
  "replicationFactor":1,
  "shards":{
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node62":{
          "core":"backuprestore_restored_shard2_replica_n61",
          "base_url":"https://127.0.0.1:35465/solr";,
          "node_name":"127.0.0.1:35465_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542264620086216682"},
    "shard1_1":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{"core_node64":{
          "core":"backuprestore_restored_shard1_1_replica_n63",
          "base_url":"https://127.0.0.1:35465/solr";,
          "node_name":"127.0.0.1:35465_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542264620086268170"},
    "shard1_0":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{"core_node66":{
          "core":"backuprestore_restored_shard1_0_replica_n65",
          "base_url":"https://127.0.0.1:35465/solr";,
          "node_name":"127.0.0.1:35465_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false",
          "leader":"true"}},
      "stateTimestamp":"1542264620086290083"}},
  "router":{
    "name":"compositeId",
    "field":"shard_s"},
  "maxShardsPerNode":"-1",
  "autoAddReplicas":"true",
  "nrtReplicas":1,
  "tlogReplicas":0}
        at 
__randomizedtesting.SeedInfo.seed([864380771B02AB2:803007DDDF4C474A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
        at java.base/java.util.HashMap.forEach(HashMap.java:1341)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
        at 
org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
        at 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 14505 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore
   [junit4]   2> 1834081 INFO  
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.TestLocalFSCloudBackupRestore_864380771B02AB2-001/init-core-data-001
   [junit4]   2> 1834081 WARN  
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
   [junit4]   2> 1834082 INFO  
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1834082 INFO  
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, 
clientAuth=0.0/0.0)
   [junit4]   2> 1834082 INFO  
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cl

[...truncated too long message...]

ca_n61] o.a.s.m.SolrMetricManager Closing metric reporters for 
registry=solr.collection.backuprestore_restored.shard2.leader, tag=39077fc2
   [junit4]   2> 78836 INFO  (coreCloseExecutor-241-thread-3) 
[n:127.0.0.1:42365_solr c:backuprestore_restored s:shard1_1 r:core_node64 
x:backuprestore_restored_shard1_1_replica_n63] o.a.s.m.SolrMetricManager 
Closing metric reporters for 
registry=solr.core.backuprestore_restored.shard1_1.replica_n63, tag=5eb71aa4
   [junit4]   2> 78836 INFO  (coreCloseExecutor-241-thread-3) 
[n:127.0.0.1:42365_solr c:backuprestore_restored s:shard1_1 r:core_node64 
x:backuprestore_restored_shard1_1_replica_n63] o.a.s.m.r.SolrJmxReporter 
Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@485b3b8a: 
rootName = solr_42365, domain = 
solr.core.backuprestore_restored.shard1_1.replica_n63, service url = null, 
agent id = null] for registry 
solr.core.backuprestore_restored.shard1_1.replica_n63 / 
com.codahale.metrics.MetricRegistry@fd9a771
   [junit4]   2> 78842 INFO  (coreCloseExecutor-242-thread-3) 
[n:127.0.0.1:37787_solr c:backuprestore s:shard1_1 r:core_node8 
x:backuprestore_shard1_1_replica_n6] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.core.backuprestore.shard1_1.replica_n6, tag=7db53f3b
   [junit4]   2> 78842 INFO  (coreCloseExecutor-242-thread-3) 
[n:127.0.0.1:37787_solr c:backuprestore s:shard1_1 r:core_node8 
x:backuprestore_shard1_1_replica_n6] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@6e6354a9: rootName = 
solr_37787, domain = solr.core.backuprestore.shard1_1.replica_n6, service url = 
null, agent id = null] for registry solr.core.backuprestore.shard1_1.replica_n6 
/ com.codahale.metrics.MetricRegistry@47f38a6d
   [junit4]   2> 78844 INFO  (coreCloseExecutor-241-thread-4) 
[n:127.0.0.1:42365_solr c:backuprestore_restored s:shard1_0 r:core_node66 
x:backuprestore_restored_shard1_0_replica_n65] o.a.s.m.SolrMetricManager 
Closing metric reporters for 
registry=solr.core.backuprestore_restored.shard1_0.replica_n65, tag=15308c27
   [junit4]   2> 78844 INFO  (coreCloseExecutor-241-thread-4) 
[n:127.0.0.1:42365_solr c:backuprestore_restored s:shard1_0 r:core_node66 
x:backuprestore_restored_shard1_0_replica_n65] o.a.s.m.r.SolrJmxReporter 
Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74d35b9d: 
rootName = solr_42365, domain = 
solr.core.backuprestore_restored.shard1_0.replica_n65, service url = null, 
agent id = null] for registry 
solr.core.backuprestore_restored.shard1_0.replica_n65 / 
com.codahale.metrics.MetricRegistry@2c533f5d
   [junit4]   2> 78849 INFO  (coreCloseExecutor-241-thread-3) 
[n:127.0.0.1:42365_solr c:backuprestore_restored s:shard1_1 r:core_node64 
x:backuprestore_restored_shard1_1_replica_n63] o.a.s.m.SolrMetricManager 
Closing metric reporters for 
registry=solr.collection.backuprestore_restored.shard1_1.leader, tag=5eb71aa4
   [junit4]   2> 78849 INFO  (coreCloseExecutor-241-thread-4) 
[n:127.0.0.1:42365_solr c:backuprestore_restored s:shard1_0 r:core_node66 
x:backuprestore_restored_shard1_0_replica_n65] o.a.s.m.SolrMetricManager 
Closing metric reporters for 
registry=solr.collection.backuprestore_restored.shard1_0.leader, tag=15308c27
   [junit4]   2> 78851 INFO  (coreCloseExecutor-242-thread-1) 
[n:127.0.0.1:37787_solr c:backuprestore s:shard1 r:core_node2 
x:backuprestore_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.backuprestore.shard1.leader, tag=5257d458
   [junit4]   2> 78851 INFO  (coreCloseExecutor-242-thread-2) 
[n:127.0.0.1:37787_solr c:backuprestore s:shard1_0 r:core_node7 
x:backuprestore_shard1_0_replica_n5] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.backuprestore.shard1_0.leader, 
tag=2c76869
   [junit4]   2> 78851 INFO  (coreCloseExecutor-242-thread-3) 
[n:127.0.0.1:37787_solr c:backuprestore s:shard1_1 r:core_node8 
x:backuprestore_shard1_1_replica_n6] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.collection.backuprestore.shard1_1.leader, 
tag=7db53f3b
   [junit4]   2> 78854 INFO  (jetty-closer-131-thread-1) [    ] 
o.a.s.c.Overseer Overseer 
(id=72165088549994500-127.0.0.1:42365_solr-n_0000000000) closing
   [junit4]   2> 78854 INFO  
(OverseerStateUpdate-72165088549994500-127.0.0.1:42365_solr-n_0000000000) 
[n:127.0.0.1:42365_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
127.0.0.1:42365_solr
   [junit4]   2> 78854 WARN  
(OverseerAutoScalingTriggerThread-72165088549994500-127.0.0.1:42365_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we 
are closed, exiting.
   [junit4]   2> 78856 INFO  (jetty-closer-131-thread-2) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@2ac5383c{/solr,null,UNAVAILABLE}
   [junit4]   2> 78856 INFO  (jetty-closer-131-thread-2) [    ] o.e.j.s.session 
node0 Stopped scavenging
   [junit4]   2> 78857 INFO  (jetty-closer-131-thread-1) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@5f07aba6{/solr,null,UNAVAILABLE}
   [junit4]   2> 78857 INFO  (jetty-closer-131-thread-1) [    ] o.e.j.s.session 
node0 Stopped scavenging
   [junit4]   2> 78858 ERROR 
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper 
server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 78858 INFO  
(SUITE-TestLocalFSCloudBackupRestore-seed#[864380771B02AB2]-worker) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:46837 46837
   [junit4]   2> 78979 INFO  (Thread-321) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:46837 46837
   [junit4]   2> 78980 WARN  (Thread-321) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/aliases.json
   [junit4]   2>        3       /solr/collections/backuprestore/terms/shard1_0
   [junit4]   2>        3       
/solr/collections/backuprestore_restored/terms/shard1_0
   [junit4]   2>        3       
/solr/collections/backuprestore_restored/terms/shard1_1
   [junit4]   2>        3       /solr/collections/backuprestore/terms/shard1_1
   [junit4]   2>        3       /solr/collections/backuprestore/terms/shard2
   [junit4]   2>        3       
/solr/collections/backuprestore_restored/terms/shard2
   [junit4]   2>        2       /solr/security.json
   [junit4]   2>        2       /solr/configs/conf1
   [junit4]   2>        2       /solr/collections/backuprestore/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/clusterstate.json
   [junit4]   2>        5       /solr/clusterprops.json
   [junit4]   2>        2       
/solr/collections/backuprestore_restored/state.json
   [junit4]   2>        2       /solr/collections/backuprestore/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/live_nodes
   [junit4]   2>        5       /solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestLocalFSCloudBackupRestore_864380771B02AB2-003
   [junit4]   2> Nov 15, 2018 7:13:16 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), 
sim=RandomSimilarity(queryNorm=false): {}, locale=fur, timezone=Asia/Bahrain
   [junit4]   2> NOTE: Linux 4.15.0-38-generic amd64/Oracle Corporation 10.0.1 
(64-bit)/cpus=8,threads=1,free=277653072,total=457179136
   [junit4]   2> NOTE: All tests run in this JVM: [TestHdfsCloudBackupRestore, 
TestLocalFSCloudBackupRestore, TestLocalFSCloudBackupRestore, 
TestLocalFSCloudBackupRestore]
   [junit4] Completed [10/10 (10!)] on J1 in 13.10s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 20 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1567: The 
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1092: 
There were test failures: 10 suites, 10 tests, 10 failures [seed: 
864380771B02AB2]

Total time: 1 minute 21 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[repro]   5/5 failed: 
org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=c4961d48dfb66223977ce6678f6de94f93f841c9, 
workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #3094
Recording test results
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org