[
https://issues.apache.org/jira/browse/SOLR-18190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18070920#comment-18070920
]
David Smiley commented on SOLR-18190:
-------------------------------------
nit: the word "write freeze" is a rather obscure way to more simply / commonly
say "read-only".
Since most/all segments will need to be rewritten, wouldn't it make sense for
NRT non-leaders to do full recovery (i.e. replicate the index)?
I suppose anyone reading this plan, in order to understand it, must already by
quite familiar with how the core level index upgrade works.
> Collection-Level Index Upgrade API in SolrCloud (UPRGRADECOLLECTIONINDEX)
> -------------------------------------------------------------------------
>
> Key: SOLR-18190
> URL: https://issues.apache.org/jira/browse/SOLR-18190
> Project: Solr
> Issue Type: Improvement
> Reporter: Rahul Goswami
> Assignee: Rahul Goswami
> Priority: Major
>
> *+Objective+*
> Expose index-upgrade functionality at collection scope in SolrCloud as a new
> "UPGRADECOLLECTIONINDEX" Collections API with async support and
> `REQUESTSTATUS` tracking.
> *+Approach+*
> _Write freeze_ *+* _hybrid local upgrade_ - Collection is set to `readOnly`
> for the duration. Each replica type is upgraded via its designed index-update
> mechanism. Which means
> * Leader is upgraded using the recently introduced CoreAdmin
> UPGRADECOREINDEX API
> * Each NRT replica gets individually upgraded using the same
> UPGRADECOREINDEX API
> * TLOG/PULL replicas converge via the usual replication mechanism.
> Coordinator waits for replicas to converge in a timebound manner before
> declaring success.
> Why not upgrade only the leader and rely on distributed forwarding to NRT
> replicas ? `DistributedZkUpdateProcessor` enforces the collection-level
> `readOnly` on every node, including replicas receiving forwarded updates.
> Forwarding is blocked by the same write freeze that protects against external
> writes.
> +*Operational Flow*+
> 1. Coordinator sets `readOnly=true` on the collection via `MODIFYCOLLECTION`
> (blocks all external writes at `DistributedZkUpdateProcessor`).
> 2. For each shard (sequentially):
> a. Identify the current leader
> b. Upgrade the leader via CoreAdmin `UPGRADECOREINDEX` action. The leader
> uses a stripped-down chain (`LogUpdateProcessor` → `RunUpdateProcessor`, no
> `DistributedUpdateProcessor`) to rewrite old segments locally. No version
> reassignment, no distributed forwarding. The original `{_}version{_}` is
> preserved both in the indexed document and on the `AddUpdateCommand` (for
> tlog consistency). After rewriting, the original merge policy is restored
> before commit.
> c. Upgrade NRT non-leader replicas in parallel using the same mechanism.
> Each NRT replica independently rewrites its own segments with zero network
> I/O.
> d. TLOG/PULL replicas converge via their normal background replication from
> the now-upgraded leader.
> e. Convergence polling: Poll all replicas with `checkOnly=true` until every
> replica reports no old-format segments remaining. ("checkOnly" is a new
> lightweight param introduced on the UPGRADECOREINDEX CoreAdmin API to check
> for presence of any old-format segments)
> 3. Clear `readOnly=false` only after all shards validate. On any failure, the
> collection remains read-only for operator intervention.
>
> *+Limitations+*
> - Nested documents: Not supported (existing limitation in
> `UpgradeCoreIndex`).
> - Write downtime: The collection is unavailable for external writes for the
> duration of the upgrade. {_}Note{_}: The CoreAdmin API doesn't have this
> limitation currently in standalone mode, but in SolrCloud mode with the
> possibility of leader election mid-upgrade and considering overall cluster
> stability/state correctness factors, reducing another variable by blocking
> writes makes the design simpler to reason about.
> - Leader election resilience: If a leader election occurs during the
> upgrade, progress may be lost and the command must be re-run. This should be
> fine since the operation is designed to be idempotent.
> - Co-located replica IO: NRT replicas on the same node are upgraded in
> parallel, which may cause IO contention. Node-aware throttling deferred to a
> future version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
