In case this helps somebody in the future, given how completely unhelpful
the Solr error message is - turns out the problem was occurring because in
solrconfig.xml the updateLog was disabled. I have enabled updateLog the
following way and "restore" operation started working:
<updateLog>
<str name="dir">${solr.ulog.dir:}</str>
<int
name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}</int>
</updateLog>
On Wed, Apr 8, 2020 at 5:17 PM Eugene Livis <eli...@basistech.com> wrote:
> Hello,
>
> I have been unsuccessfully trying to find a way to restore a collection
> using the Collections API for the last several days. *I would
> greatly appreciate any help as I am now stuck.* I am using Solr 8.2 in
> cloud mode. To simplify things, I have only a single Solr node on my local
> machine (though I have also tried multi-node setups). I am able to
> successfully create collections, index documents, and run searches against
> the index. General gist of what I am trying to do is to be able to backup
> an existing collection, delete that collection, and then restore the
> collection from backup when needed. Our use case requires creating
> thousands of collections, which I believe is highly not recommended in Solr
> Cloud (is that correct?), so this backup-delete-restore mechanism is our
> way of reducing the number of collections in Solr at any given time. Just
> FYI, in our use case it is ok for searches to take a couple of minutes to
> produce results.
>
> So far I have not been able to restore a single collection using the
> collection RESTORE API. I am able to create backups by running the
> following HTTP command in Solr admin console:
>
>
> http://localhost:8983/solr/admin/collections?action=BACKUP&name=test1_backup2&collection=test1_20200407_170438_20200407_170441&location=C:\TEST\DELETE\BACKUPS
>
>
> The backup appears to be successfully created. It contains a snapshot from
> "shard1", which is the only shard in the cluster. It also contains the
> "zk_backup" directory, which contains my Solr config and
> "collection_state.json" file. And it contains a "backup.properties" file.
> Everything looks good to me.
>
> However, when I attempt to restore the collection using the following HTTP
> command:
>
>
> http://localhost:8983/solr/admin/collections?action=RESTORE&name=test1_backup2&location=C:\TEST\DELETE\BACKUPS&collection=test1_backup_NEW
>
>
> I get the same extremely vague error messages in Solr logs:
>
>
> *RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing
> SolrCore 'test1_backup_NEW_shard1_replica_n1': Unable to create core
> [test1_backup_NEW_shard1_replica_n1] Caused by: nullERROR
> (qtp1099855928-24) [c:test1_backup_NEW ] o.a.s.s.HttpSolrCall
> null:org.apache.solr.common.SolrException: ADDREPLICA failed to create
> replica *
>
> Below is the full Solr log of the restore command:
>
> 2020-04-08 21:10:56.753 INFO (qtp1099855928-24) [ ]
> o.a.s.h.a.CollectionsHandler Invoked Collection Action :restore with params
> name=test1_backup2&action=RESTORE&location=C:\TEST\DELETE\BACKUPS&collection=test1_backup_NEW
> and sendToOCPQueue=true
> 2020-04-08 21:10:56.783 INFO
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.RestoreCmd Using existing config
> AutopsyConfig
> 2020-04-08 21:10:56.783 INFO
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.RestoreCmd Starting restore into
> collection=test1_backup_NEW with backup_name=test1_backup2 at
> location=file:///C:/TEST/DELETE/BACKUPS/
> 2020-04-08 21:10:56.784 INFO
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.CreateCollectionCmd Create collection
> test1_backup_NEW
> 2020-04-08 21:10:56.900 WARN
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to
> create a collection (test1_backup_NEW) without cores.
> 2020-04-08 21:10:56.908 INFO
> (OverseerStateUpdate-72057883336638464-localhost:8983_solr-n_0000000003) [
> ] o.a.s.c.o.SliceMutator Update shard state invoked for collection:
> test1_backup_NEW with message: {
> "shard1":"construction",
> "collection":"test1_backup_NEW",
> "operation":"updateshardstate"}
> 2020-04-08 21:10:56.908 INFO
> (OverseerStateUpdate-72057883336638464-localhost:8983_solr-n_0000000003) [
> ] o.a.s.c.o.SliceMutator Update shard state shard1 to construction
> 2020-04-08 21:10:56.925 INFO
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.RestoreCmd Adding replica for
> shard=shard1
> collection=DocCollection(test1_backup_NEW//collections/test1_backup_NEW/state.json/0)={
> "pullReplicas":0,
> "replicationFactor":1,
> "shards":{"shard1":{
> "range":"80000000-7fffffff",
> "state":"active",
> "replicas":{}}},
> "router":{"name":"compositeId"},
> "maxShardsPerNode":"1",
> "autoAddReplicas":"false",
> "nrtReplicas":1,
> "tlogReplicas":0}
> 2020-04-08 21:10:56.928 INFO
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.AddReplicaCmd Node Identified
> localhost:8983_solr for creating new replica of shard shard1 for collection
> test1_backup_NEW
> 2020-04-08 21:10:56.939 INFO
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica
> command.
> 2020-04-08 21:10:56.945 INFO
> (OverseerStateUpdate-72057883336638464-localhost:8983_solr-n_0000000003) [
> ] o.a.s.c.o.SliceMutator createReplica() {
> "operation":"addreplica",
> "collection":"test1_backup_NEW",
> "shard":"shard1",
> "core":"test1_backup_NEW_shard1_replica_n1",
> "state":"down",
> "base_url":"http://localhost:8983/solr";,
> "node_name":"localhost:8983_solr",
> "type":"NRT"}
> 2020-04-08 21:10:57.348 INFO (qtp1099855928-19) [
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core
> create command
> qt=/admin/cores&coreNodeName=core_node2&collection.configName=AutopsyConfig&name=test1_backup_NEW_shard1_replica_n1&action=CREATE&collection=test1_backup_NEW&shard=shard1&wt=javabin&version=2&replicaType=NRT
> 2020-04-08 21:10:57.373 WARN (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.c.SolrConfig Couldn't add files from C:\total\crap\dir\ignored
> filtered by null to classpath: C:\total\crap\dir\ignored
> 2020-04-08 21:10:57.374 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.c.SolrResourceLoader [test1_backup_NEW_shard1_replica_n1] Added 56
> libs to classloader, from paths:
> [/C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/clustering/lib,
> /C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/extraction/lib,
> /C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/langid/lib,
> /C:/Bitnami/solr-8.2.0-2/apache-solr/contrib/velocity/lib]
> 2020-04-08 21:10:57.395 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.2.0
> 2020-04-08 21:10:57.424 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.s.IndexSchema [test1_backup_NEW_shard1_replica_n1] Schema
> name=Autopsy Keyword Search
> 2020-04-08 21:10:57.451 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.s.IndexSchema Loaded schema Autopsy Keyword Search/2.2 with uniqueid
> field id
> 2020-04-08 21:10:57.473 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.c.CoreContainer Creating SolrCore
> 'test1_backup_NEW_shard1_replica_n1' using configuration from collection
> test1_backup_NEW, trusted=true
> 2020-04-08 21:10:57.473 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr.core.test1_backup_NEW.shard1.replica_n1' (registry
> 'solr.core.test1_backup_NEW.shard1.replica_n1') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@214b199c
> 2020-04-08 21:10:57.473 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.c.SolrCore [[test1_backup_NEW_shard1_replica_n1] ] Opening new
> SolrCore at
> [C:\Bitnami\solr-8.2.0-2\apache-solr\server\solr\test1_backup_NEW_shard1_replica_n1],
> dataDir=[C:\Bitnami\solr-8.2.0-2\apache-solr\server\solr\test1_backup_NEW_shard1_replica_n1\data\]
> 2020-04-08 21:10:57.491 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
> 2020-04-08 21:10:57.527 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.u.CommitTracker Hard AutoCommit: if 15000 uncommitted docs;
> 2020-04-08 21:10:57.527 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.u.CommitTracker Soft AutoCommit: disabled
> 2020-04-08 21:10:57.535 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.s.SolrIndexSearcher Opening
> [Searcher@59a2b753[test1_backup_NEW_shard1_replica_n1]
> main]
> 2020-04-08 21:10:57.536 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with
> znodeBase: /configs/AutopsyConfig
> 2020-04-08 21:10:57.537 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
> ZooKeeperStorageIO:path=/configs/AutopsyConfig
> 2020-04-08 21:10:57.537 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.h.c.SpellCheckComponent Initializing spell checkers
> 2020-04-08 21:10:57.537 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.s.DirectSolrSpellChecker init:
> {name=default,field=name,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
> 2020-04-08 21:10:57.540 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
> 2020-04-08 21:10:57.541 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.c.SolrCore [test1_backup_NEW_shard1_replica_n1] CLOSING SolrCore
> org.apache.solr.core.SolrCore@3a4eb909
> 2020-04-08 21:10:57.541 INFO
> (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
> x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
> r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.QuerySenderListener
> QuerySenderListener sending requests to
> Searcher@59a2b753[test1_backup_NEW_shard1_replica_n1]
> main{ExitableDirectoryReader(UninvertingDirectoryReader())}
> 2020-04-08 21:10:57.541 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.core.test1_backup_NEW.shard1.replica_n1, tag=3a4eb909
> 2020-04-08 21:10:57.542 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.m.r.SolrJmxReporter Closing reporter
> [org.apache.solr.metrics.reporters.SolrJmxReporter@54eb018f: rootName =
> null, domain = solr.core.test1_backup_NEW.shard1.replica_n1, service url =
> null, agent id = null] for registry
> solr.core.test1_backup_NEW.shard1.replica_n1 /
> com.codahale.metrics.MetricRegistry@55bc77fd
> 2020-04-08 21:10:57.542 INFO
> (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
> x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
> r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.S.Request
> [test1_backup_NEW_shard1_replica_n1] webapp=null path=null
> params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
> hits=0 status=0 QTime=1
> 2020-04-08 21:10:57.542 INFO
> (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
> x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
> r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.QuerySenderListener
> QuerySenderListener done.
> 2020-04-08 21:10:57.542 INFO
> (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
> x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
> r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading
> spell index for spellchecker: default
> 2020-04-08 21:10:57.542 INFO
> (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
> x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
> r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading
> spell index for spellchecker: wordbreak
> 2020-04-08 21:10:57.542 INFO
> (searcherExecutor-22-thread-1-processing-n:localhost:8983_solr
> x:test1_backup_NEW_shard1_replica_n1 c:test1_backup_NEW s:shard1
> r:core_node2) [c:test1_backup_NEW s:shard1 r:core_node2
> x:test1_backup_NEW_shard1_replica_n1] o.a.s.c.SolrCore
> [test1_backup_NEW_shard1_replica_n1] Registered new searcher
> Searcher@59a2b753[test1_backup_NEW_shard1_replica_n1]
> main{ExitableDirectoryReader(UninvertingDirectoryReader())}
> 2020-04-08 21:10:57.552 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.collection.test1_backup_NEW.shard1.leader, tag=3a4eb909
> 2020-04-08 21:10:57.560 ERROR (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error
> CREATEing SolrCore 'test1_backup_NEW_shard1_replica_n1': Unable to create
> core [test1_backup_NEW_shard1_replica_n1] Caused by: null
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1195)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
> at
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:505)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
> at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:311)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
> at
> org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
> at
> org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
> at
> org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Unable to create core
> [test1_backup_NEW_shard1_replica_n1]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1266)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1156)
> ... 50 more
> Caused by: org.apache.solr.common.SolrException
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1055)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:882)
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
> ... 51 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.solr.core.SolrCore.bufferUpdatesIfConstructing(SolrCore.java:1097)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1029)
> ... 53 more
>
> 2020-04-08 21:10:57.561 INFO (qtp1099855928-19) [c:test1_backup_NEW
> s:shard1 r:core_node2 x:test1_backup_NEW_shard1_replica_n1]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
> params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=AutopsyConfig&name=test1_backup_NEW_shard1_replica_n1&action=CREATE&collection=test1_backup_NEW&shard=shard1&wt=javabin&version=2&replicaType=NRT}
> status=400 QTime=213
> 2020-04-08 21:10:57.577 ERROR
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error
> from shard: http://localhost:8983/solr =>
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException:
> Error from server at null: Error CREATEing SolrCore
> 'test1_backup_NEW_shard1_replica_n1': Unable to create core
> [test1_backup_NEW_shard1_replica_n1] Caused by: null
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:727)
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException:
> Error from server at null: Error CREATEing SolrCore
> 'test1_backup_NEW_shard1_replica_n1': Unable to create core
> [test1_backup_NEW_shard1_replica_n1] Caused by: null
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:727)
> ~[?:?]
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:400)
> ~[?:?]
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:746)
> ~[?:?]
> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274)
> ~[?:?]
> at
> org.apache.solr.handler.component.HttpShardHandler.request(HttpShardHandler.java:238)
> ~[?:?]
> at
> org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:199)
> ~[?:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_222-1-ojdkbuild]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_222-1-ojdkbuild]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_222-1-ojdkbuild]
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
> ~[metrics-core-4.0.5.jar:4.0.5]
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_222-1-ojdkbuild]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_222-1-ojdkbuild]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222-1-ojdkbuild]
> 2020-04-08 21:10:57.578 ERROR
> (OverseerThreadFactory-9-thread-1-processing-n:localhost:8983_solr)
> [c:test1_backup_NEW ] o.a.s.c.a.c.OverseerCollectionMessageHandler
> Collection: test1_backup_NEW operation: restore
> failed:org.apache.solr.common.SolrException: ADDREPLICA failed to create
> replica
> at
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler$ShardRequestTracker.processResponses(OverseerCollectionMessageHandler.java:1030)
> at
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler$ShardRequestTracker.processResponses(OverseerCollectionMessageHandler.java:1013)
> at
> org.apache.solr.cloud.api.collections.AddReplicaCmd.lambda$addReplica$1(AddReplicaCmd.java:177)
> at
> org.apache.solr.cloud.api.collections.AddReplicaCmd.addReplica(AddReplicaCmd.java:199)
> at
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.addReplica(OverseerCollectionMessageHandler.java:708)
> at
> org.apache.solr.cloud.api.collections.RestoreCmd.call(RestoreCmd.java:286)
> at
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:264)
> at
> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:505)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by:
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException:
> Error from server at null: Error CREATEing SolrCore
> 'test1_backup_NEW_shard1_replica_n1': Unable to create core
> [test1_backup_NEW_shard1_replica_n1] Caused by: null
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:727)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:400)
> at
> org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:746)
> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274)
> at
> org.apache.solr.handler.component.HttpShardHandler.request(HttpShardHandler.java:238)
> at
> org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:199)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181)
> ... 4 more
>
> 2020-04-08 21:10:57.604 INFO (qtp1099855928-24) [c:test1_backup_NEW ]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
> params={name=test1_backup2&action=RESTORE&location=C:\TEST\DELETE\BACKUPS&collection=test1_backup_NEW}
> status=500 QTime=850
> 2020-04-08 21:10:57.605 ERROR (qtp1099855928-24) [c:test1_backup_NEW ]
> o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: ADDREPLICA
> failed to create replica
> at
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)
> at
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:280)
> at
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:252)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:505)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
> at java.lang.Thread.run(Thread.java:748)
>
> 2020-04-08 21:10:58.775 INFO
>
> (OverseerCollectionConfigSetProcessor-72057883336638464-localhost:8983_solr-n_0000000003)
> [ ] o.a.s.c.OverseerTaskQueue Response ZK path:
> /overseer/collection-queue-work/qnr-0000000014 doesn't exist. Requestor
> may have disconnected from ZooKeeper
> 2020-04-08 21:11:16.690 INFO (qtp1099855928-21) [ ]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={wt=javabin&version=2&key=solr.core.test1_backup_NEW2.shard1.replica_n1:QUERY./select.requests&key=solr.core.test1_20200407_170438_20200407_170441.shard1.replica_n1:UPDATE./update.requests&key=solr.core.test1_backup_NEW2.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test1_20200407_170438_20200407_170441.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test1_backup_NEW2.shard1.replica_n1:UPDATE./update.requests&key=solr.core.xp_20200408_162046_20200408_162051.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test1_backup_NEW.shard1.replica_n1:UPDATE./update.requests&key=solr.core.xp_20200408_162046_20200408_162051.shard1.replica_n1:UPDATE./update.requests&key=solr.core.test1_20200407_170438_20200407_170441.shard1.replica_n1:QUERY./select.requests&key=solr.core.xp_20200408_162046_20200408_162051.shard1.replica_n1:QUERY./select.requests&key=solr.core.test1_backup_NEW.shard1.replica_n1:QUERY./select.requests&key=solr.core.test1_backup_NEW.shard1.replica_n1:INDEX.sizeInBytes}
> status=0 QTime=51
> 2020-04-08 21:11:16.731 INFO (qtp1099855928-24) [ ]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used}
> status=0 QTime=27
>
>
>
