I agree with Shawn that having instanceDir=/opt/solr/server/solr/configsets/articles as where your core is located is strange, how are you starting Solr? In particular, do those nodes use a "-s' parameter to redefine SOLR_HOME?
The "-d" option (assuming you created the collection with bin/solr) just tells Solr where the configset you're using is located on your local disk, it does _not_ put the instanceDir there, it just looks for the configset to upload to ZK. Check if there's a "core.properties" file in "/opt/solr/server/solr/configsets/articles". The presence of that file indicates that that's the home of the replica. Now to the very weird bit. This sounds an awful lot like https://issues.apache.org/jira/browse/SOLR-11503. Especially if your old system was Solr 6.6.1 or 6.6.2.. Short form, "core.properties" needs to contain a, you guessed it, "coreNodeName" property that corresponds to what's in ZooKeeper. However, I simply cannot reconcile this being the problem with what you're reporting. There's code in 7x that repairs this issue automatically. Is there any chance for the node in question that it's _not_ running 7.4 and still on 6.6.1/2? Maybe Shawn's latest comment would reconcile that? How to fix. Well, fixing should be automatic unless the fixer-upper code is no longer in 7.4, but on a quick check the code _is_ in 7.4. > Make very, very sure the Solr you're running on the nodes in question is 7.4, > or at least _NOT_ 6.6.1/2 > Make very, very sure that you don't specify some strange "-s" parameter or > have defined SOLR_HOME to point to /opt/solr/server/solr/configsets/articles, > although that shouldn't really matter. > When you DELETEREPLICA, go to the node and manually see if the directory > /opt/solr/server/solr/configsets/articles exists. It shouldn't (see the > deleteInstanceDir option in the DELETEREPLICA for why). If none of that makes any difference, please show us the full output of > ps aux | grep solr > export (looking for you redefining SOLR_HOME) > the "core.properties" file in the indicated directory. Best, Erick On Thu, Sep 20, 2018 at 7:36 AM Shawn Heisey <apa...@elyograg.org> wrote: > > On 9/20/2018 8:22 AM, Schaum Mallik wrote: > > Thanks for the response Shawn. > > > > My follow up question is how would the zookeeper ensemble know that the > > location of the indexes has changed? Also do I need to apply the same > > changes to the other 2 solr nodes which are working fine? > > This move is not to change the location, it's to stop Solr from trying > to load the indexes completely. Solr will no longer try to load them. > I think these are invalid indexes from when the Solr instance was NOT > running in cloud mode, and have nothing at all to do with your working > collections. But just in case I'm wrong, I'm having you move them > instead of delete them, so they can be put back if it turns out they > actually were needed. > > ZooKeeper doesn't know anything at all about Solr and the data it puts > in ZK. It is just a service that stores cluster data where all nodes > can read it and handles elections when it is asked to. > > Thanks, > Shawn >
