cpoerschke commented on a change in pull request #288:
URL: https://github.com/apache/solr/pull/288#discussion_r742125759
##########
File path:
solr/core/src/java/org/apache/solr/update/processor/DistributedZkUpdateProcessor.java
##########
@@ -316,6 +317,41 @@ public void processDelete(DeleteUpdateCommand cmd) throws
IOException {
protected void doDeleteById(DeleteUpdateCommand cmd) throws IOException {
setupRequest(cmd);
+ if (broadcastDeleteById &&
DistribPhase.parseParam(req.getParams().get(DISTRIB_UPDATE_PARAM)) ==
DistribPhase.NONE ) {
+
+ log.debug("The deleteById command is missing the required route,
broadcasting to leaders of other shards");
+
+ ModifiableSolrParams outParams = new
ModifiableSolrParams(filterParams(req.getParams()));
+ outParams.set(DISTRIB_UPDATE_PARAM, DistribPhase.TOLEADER.toString());
+ outParams.set(DISTRIB_FROM, ZkCoreNodeProps.getCoreUrl(
+ zkController.getBaseUrl(), req.getCore().getName()));
+
+ SolrParams params = req.getParams();
+ String route = params.get(ShardParams._ROUTE_);
+ DocCollection coll = clusterState.getCollection(collection);
+ Collection<Slice> slices = coll.getRouter().getSearchSlices(route,
params, coll);
+
+ // if just one slice, we can skip this
Review comment:
2/4 - if the "just one slice" logic happened earlier so that
`broadcastDeleteById` is only true if there's more than one slice this:
* would remove the need for this `if` here
* could facilitate code sharing with `doDeleteByQuery`
##########
File path:
solr/core/src/java/org/apache/solr/update/processor/DistributedZkUpdateProcessor.java
##########
@@ -316,6 +317,41 @@ public void processDelete(DeleteUpdateCommand cmd) throws
IOException {
protected void doDeleteById(DeleteUpdateCommand cmd) throws IOException {
setupRequest(cmd);
+ if (broadcastDeleteById &&
DistribPhase.parseParam(req.getParams().get(DISTRIB_UPDATE_PARAM)) ==
DistribPhase.NONE ) {
+
+ log.debug("The deleteById command is missing the required route,
broadcasting to leaders of other shards");
Review comment:
1/4 - minor: maybe include `cmd.getId()` in the debug logging here
##########
File path:
solr/core/src/java/org/apache/solr/update/processor/DistributedZkUpdateProcessor.java
##########
@@ -383,47 +419,7 @@ protected void doDeleteByQuery(DeleteUpdateCommand cmd)
throws IOException {
if (rollupReplicationTracker == null) {
rollupReplicationTracker = new RollupRequestReplicationTracker();
}
- boolean leaderForAnyShard = false; // start off by assuming we are not
a leader for any shard
-
- ModifiableSolrParams outParams = new
ModifiableSolrParams(filterParams(req.getParams()));
- outParams.set(DISTRIB_UPDATE_PARAM, DistribPhase.TOLEADER.toString());
- outParams.set(DISTRIB_FROM, ZkCoreNodeProps.getCoreUrl(
- zkController.getBaseUrl(), req.getCore().getName()));
-
- SolrParams params = req.getParams();
- String route = params.get(ShardParams._ROUTE_);
- Collection<Slice> slices = coll.getRouter().getSearchSlices(route,
params, coll);
-
- List<SolrCmdDistributor.Node> leaders = new ArrayList<>(slices.size());
- for (Slice slice : slices) {
- String sliceName = slice.getName();
- Replica leader;
- try {
- leader = zkController.getZkStateReader().getLeaderRetry(collection,
sliceName);
- } catch (InterruptedException e) {
- throw new SolrException(SolrException.ErrorCode.SERVICE_UNAVAILABLE,
"Exception finding leader for shard " + sliceName, e);
- }
-
- // TODO: What if leaders changed in the meantime?
- // should we send out slice-at-a-time and if a node returns "hey, I'm
not a leader" (or we get an error because it went down) then look up the new
leader?
-
- // Am I the leader for this slice?
- ZkCoreNodeProps coreLeaderProps = new ZkCoreNodeProps(leader);
- String leaderCoreNodeName = leader.getName();
- String coreNodeName = cloudDesc.getCoreNodeName();
- isLeader = coreNodeName.equals(leaderCoreNodeName);
-
- if (isLeader) {
- // don't forward to ourself
- leaderForAnyShard = true;
- } else {
- leaders.add(new SolrCmdDistributor.ForwardNode(coreLeaderProps,
zkController.getZkStateReader(), collection, sliceName, maxRetriesOnForward));
- }
- }
-
- outParams.remove("commit"); // this will be distributed from the local
commit
-
- cmdDistrib.distribDelete(cmd, leaders, outParams, false,
rollupReplicationTracker, null);
+ boolean leaderForAnyShard = forwardDelete(coll, cmd);
Review comment:
3/4 - added a commit to the pull request branch -- feel free to revert
or amend -- that factors out `forwardDelete` method from `doDeleteByQuery`, for
potential code sharing with `doDeleteById`
##########
File path:
solr/core/src/java/org/apache/solr/update/processor/DistributedZkUpdateProcessor.java
##########
@@ -316,6 +317,41 @@ public void processDelete(DeleteUpdateCommand cmd) throws
IOException {
protected void doDeleteById(DeleteUpdateCommand cmd) throws IOException {
setupRequest(cmd);
+ if (broadcastDeleteById &&
DistribPhase.parseParam(req.getParams().get(DISTRIB_UPDATE_PARAM)) ==
DistribPhase.NONE ) {
+
+ log.debug("The deleteById command is missing the required route,
broadcasting to leaders of other shards");
+
+ ModifiableSolrParams outParams = new
ModifiableSolrParams(filterParams(req.getParams()));
+ outParams.set(DISTRIB_UPDATE_PARAM, DistribPhase.TOLEADER.toString());
+ outParams.set(DISTRIB_FROM, ZkCoreNodeProps.getCoreUrl(
+ zkController.getBaseUrl(), req.getCore().getName()));
+
+ SolrParams params = req.getParams();
+ String route = params.get(ShardParams._ROUTE_);
+ DocCollection coll = clusterState.getCollection(collection);
+ Collection<Slice> slices = coll.getRouter().getSearchSlices(route,
params, coll);
+
+ // if just one slice, we can skip this
+ if (slices.size() > 1) {
+ List<SolrCmdDistributor.Node> leaders = new ArrayList<>(slices.size()
- 1);
+ for (Slice slice : slices) {
+ String sliceName = slice.getName();
+ if (!sliceName.equals(cloudDesc.getShardId())) {
+ Replica leader;
+ try {
+ leader =
zkController.getZkStateReader().getLeaderRetry(collection, sliceName);
+ } catch (InterruptedException e) {
+ throw new
SolrException(SolrException.ErrorCode.SERVICE_UNAVAILABLE, "Exception finding
leader for shard " + sliceName, e);
+ }
+ ZkCoreNodeProps coreLeaderProps = new ZkCoreNodeProps(leader);
+ leaders.add(new SolrCmdDistributor.ForwardNode(coreLeaderProps,
zkController.getZkStateReader(), collection, sliceName, maxRetriesOnForward));
+ }
+ }
+ outParams.remove("commit"); // this will be distributed from the local
commit
+ cmdDistrib.distribDelete(cmd, leaders, outParams, false,
rollupReplicationTracker, null);
+ }
+ }
+
// check if client has requested minimum replication factor information.
will set replicationTracker to null if
// we aren't the leader or subShardLeader
checkReplicationTracker(cmd);
Review comment:
4/4 - `doDeleteByQuery` does rollup replication tracker logic before the
forward-delete logic, `doDeleteById` here does check replication tracker logic
after the forward-logic -- i've not yet considered this difference in detail,
just noticed it whilst factoring out the forward-delete logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
