Hello, We just witnessed this too with 7.7. No no obvious messages in the logs, the replica status would not come out of 'down'.
Meanwhile we got another weird exception from a neighbouring collection sharing the same nodes: 2019-02-18 13:47:20.622 ERROR (updateExecutor-3-thread-1-processing-n:idx1:8983_solr x:search_20180717_shard1_replica_t81 c:search_20180717 s:shard1 r:core_node82 ) [c:search_20180717 s:shard1 r:core_node82 x:search_20180717_shard1_replica_t81] o.a.s.u.SolrCmdDistributor org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Er ror from server at http://idx5:8983/solr/search_20180717_shard1_replica_t91: invalid boolean value: replicas at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:491) at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260) at org.apache.solr.update.SolrCmdDistributor.doRequest(SolrCmdDistributor.java:326) at org.apache.solr.update.SolrCmdDistributor.lambda$submit$0(SolrCmdDistributor.java:315) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Although probably not related, is this a known thing? Or shall i open an issue. Thanks, Markus -----Original message----- > From:Jeff Courtade <courtadej...@gmail.com> > Sent: Friday 15th February 2019 21:54 > To: solr-user@lucene.apache.org > Subject: Re: solr cloud version upgrade 7.6 to 7.7 collection indexes all > marked as down > > Yes... nothing in the logs does mean that there was nothing of interest. I > have actual entries. > > This is a test environment so this isn't an emergency. Thanks for the > clarification about what I should be seeing. > > I was just so flabbergasted by this because it's so strange I had to tell > somebody and yell at the universe basically so I yelled at the solar > mailing list. > > This is an automated upgrading so the next step is to go through and > manually perform all the steps and see if I get the same behavior. > > I am fairly certain I just going to be some dumb thing that I'm doing and I > will be happy to update the mailing list when I figure this out for > everyone's Mutual entertainment. > -- > Jeff Courtade > M: 240.507.6116 > > On Fri, Feb 15, 2019, 12:33 PM Erick Erickson <erickerick...@gmail.com > wrote: > > > Hmmm. I'm assuming that "nothing in the logs" is node/logs/solr.log, and > > that > > you're not finding errors/exceptipons. Just sanity checking here. > > > > My guess: you're picking up the default SOLR_HOME which is in your new > > installation directory and all your > > replicas are under the old install directory. > > > > There should be some kind of message in the log files indicating that > > Solr is at least trying to load replicas, something similar to: > > > > Using system property solr.solr.home: > > /Users/Erick/apache/solrVersions/playspace/solr/example/cloud/node1/solr > > > > and/or: > > > > CorePropertiesLocator Found 3 core definitions underneath > > /Users/Erick/apache/solrVersions/playspace/solr/example/cloud/node1/solr > > > > A bit of background: When Solr starts up, it recursively descends from > > SOLR_HOME and whenever it finds a "core.properties" file > > it says "Aha, this must be a core, I'll try to load it". So if > > SOLR_HOME is doesn't point to an ancestor of your existing replicas, > > Solr won't find any replicas and everything will stay down. _If_ > > SOLR_HOME is defined in solr.in.sh, this should just be picked up. > > > > Best, > > Erick > > > > On Thu, Feb 14, 2019 at 7:43 PM Zheng Lin Edwin Yeo > > <edwinye...@gmail.com> wrote: > > > > > > Hi, > > > > > > Which version of zookeeper are you using? > > > > > > Also, if you tried to query the index, did you get any error message? > > > > > > Regards, > > > Edwin > > > > > > > > > On Fri, 15 Feb 2019 at 02:34, Jeff Courtade <courtadej...@gmail.com> > > wrote: > > > > > > > Hi, > > > > > > > > I am working n doing a simple point upgrade from solr 7.6 to 7.7 cloud. > > > > > > > > 6 servers > > > > 3 zookeepers > > > > one simple test collection using the prepackages _default config. > > > > > > > > i stop all solr servers leaving the zookeepers up. > > > > > > > > change out the binaries and put the solr.in.sh file back in place with > > > > memory and directory stuff. > > > > > > > > The index directory does not move the files dont change > > > > > > > > i start up the new binaries and it starts with no errors in the logs > > but > > > > all of the indexes are "down" > > > > > > > > I have no clue here. nothing in the logs > > > > > >
