[
https://issues.apache.org/jira/browse/SOLR-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669027#comment-16669027
]
Erick Erickson commented on SOLR-12940:
---------------------------------------
I want to raise one concern here. This will essentially do a "full sync" of the
index on the leader on all replicas in a collection at once, correct? That can
move massive amounts of data around the network, and I've seen cascading
failures result from that, endless recovery loops and the like.
Is there a way to throttle this?
I happen to be working on SOLR-12259 currently, which will add another
operation to the mix so I'll link that in. The thrust of that JIRA is to
rewrite all the segments to "do something", one case is to add docValues
without re-indexing. Seems to me that the mechanism in this JIRA is applicable
to SOLR-12259 too.
> Optimize and expunge deletes should execute only on the leader instead of all
> replicas of the collection
> --------------------------------------------------------------------------------------------------------
>
> Key: SOLR-12940
> URL: https://issues.apache.org/jira/browse/SOLR-12940
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Shalin Shekhar Mangar
> Priority: Major
> Fix For: 7.6, master (8.0)
>
>
> Optimize and expunge deletes are broadcasted to all replicas of the
> collection (even to replicas of inactive shards!) but they don't need to be.
> We can optimize by only executing such commands on the leader and have the
> replicas pull the index over the network when ready.
> Synchronizing replication recovery to happen after optimize completes was
> trickier in the past when all we had was HTTP requests but now we have the
> terms mechanism which goes via ZK and can be relied on.
> This gives us a nice way to index/update fully and then call optimize/expunge
> deletes only on the leader while replicas continue to serve query traffic
> without disruption. This use-case will also need the ability to route queries
> only to replicas and not to the leader.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
