Jason Gerlowski created SOLR-18569:
--------------------------------------
Summary: Solr should clean up /node_roles entries, like /live_nodes
Key: SOLR-18569
URL: https://issues.apache.org/jira/browse/SOLR-18569
Project: Solr
Issue Type: Bug
Reporter: Jason Gerlowski
On Solr startup, ZkController.createEphemeralLiveNode creates
{{/live_nodes/<node>}} and one ephemeral {{/node_roles/<role>/<mode>/<node>}}
znode per node role in a single atomic ZK "multi" operation.
On graceful shutdown, ZkController.removeEphemeralLiveNode deletes
{{/live_nodes/<node>}}, but never does any cleanup for the corresponding
{{/node_roles}} entries.
These lingering entries are ephemeral and will get removed from ZK within
{{zkClientTimeout}} millis after Solr shuts down. But they can still cause
problems! For instance: we periodically see problems on a Solr restart because
the new Solr node tries to create its {{/node_role}} entries but fails because
the nodes already exist!
{code}
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists
at org.apache.zookeeper.KeeperException.create(KeeperException.java:125)
at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1778)
at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1650)
at
org.apache.solr.common.cloud.SolrZkClient.lambda$multi$12(SolrZkClient.java:792)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
at org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:792)
at
org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:1244)
at org.apache.solr.cloud.ZkController.init(ZkController.java:1059)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:408)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:139)
at org.apache.solr.core.CoreContainer.loadInternal(CoreContainer.java:874)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:808)
{code}
We should give ZkController.removeEphemeralLiveNode the same cleanup for
/node_roles that it already has for /live_nodes, so that restarts don't hit
this on startup.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
