[
https://issues.apache.org/jira/browse/HDDS-16136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-16136:
------------------------------
Description:
SCM HA groups whose Ratis peers were recorded with IPv4 literal addresses
cannot move to IPv6-only operation by editing ozone-site.xml alone. SCM starts
its Ratis server with an empty group and recovers the peers, including their
addresses, from Raft storage, so the stored IPv4 addresses keep being used.
This is not a blocker for IPv6 support in general. It does not affect:
* New clusters deployed with IPv6 addresses.
* Peers configured with DNS names, which remain the preferred migration path
because the stored hostname resolves to IPv6.
* Dual-stack clusters that keep IPv4 connectivity.
* OM HA, which passes its configured peers to Ratis at startup, so its
connections follow ozone-site.xml. Only the stored and displayed addresses may
stay stale.
Recommended approach:
* Use a Ratis startup option that refreshes the stored peer addresses from the
peers supplied at startup, as proposed in the review of
[apache/ratis#1630|https://github.com/apache/ratis/pull/1630] (RATIS-2082 was
closed as Won't Do). SCM would pass its configured peers when starting its
Ratis server and enable the option, so ozone-site.xml becomes the single source
for both the addresses SCM connects to and the addresses Ratis stores. A wrong
address is fixed by correcting the configuration and restarting. This needs the
option in a Ratis release and an Ozone upgrade to that release.
Alternatives considered:
* Register the configured SCM peers with Ratis before the server starts,
without the startup option. This is an Ozone only change, but it relies on
Ratis keeping the first address registered for a peer, and the stored and
displayed addresses would stay stale and could be propagated by later SCM
membership changes.
* Replace each SCM through decommission and bootstrap with its new address.
This works with existing tools but takes many steps per SCM and reduces fault
tolerance while it runs.
* Run the offline ratis-shell {{local raftMetaConf}} command on each stopped
SCM. This works today but is manual and error prone, and a wrong address leaves
that SCM unable to rejoin. It can be documented as an interim procedure until
the startup option is available.
Requirements:
* Preserve the Ratis group ID, peer IDs, storage, and committed state.
* Support a rolling restart while both IPv4 and IPv6 addresses are reachable.
* Allow a wrong address to be corrected by fixing the configuration and
restarting.
* Validate leader election and failover after migration.
* Add SCM migration tests.
Parent epic: HDDS-15763.
was:
SCM HA groups whose Ratis peers were recorded with IPv4 literal addresses
cannot move to IPv6-only operation by editing ozone-site.xml alone. SCM starts
its Ratis server with an empty group and recovers the peers, including their
addresses, from Raft storage, so the stored IPv4 addresses keep being used.
This is not a blocker for IPv6 support in general. It does not affect:
* New clusters deployed with IPv6 addresses.
* Peers configured with DNS names, which remain the preferred migration path
because the stored hostname resolves to IPv6.
* Dual-stack clusters that keep IPv4 connectivity.
* OM HA, which passes its configured peers to Ratis at startup, so its
connections follow ozone-site.xml. Only the stored and displayed addresses may
stay stale.
Candidate approaches:
* Document the offline ratis-shell {{local raftMetaConf}} command as a manual
procedure run on each stopped SCM.
* Register the configured SCM peers with Ratis before the server starts, as OM
does, so connections follow ozone-site.xml.
* Use a Ratis startup option that refreshes the stored peer addresses from the
supplied peers, if Ratis adds one. This was proposed in the review of
[apache/ratis#1630|https://github.com/apache/ratis/pull/1630]; RATIS-2082 was
closed as Won't Do.
Replacing each SCM through decommission and bootstrap with its new address
remains a fallback with existing tools.
Requirements:
* Preserve the Ratis group ID, peer IDs, storage, and committed state.
* Support a rolling restart while both IPv4 and IPv6 addresses are reachable.
* Allow a wrong address to be corrected by fixing the configuration and
restarting.
* Validate leader election and failover after migration.
* Add SCM migration tests.
Parent epic: HDDS-15763.
> Support migrating SCM HA from IPv4 literal Ratis peer addresses to IPv6-only
> ----------------------------------------------------------------------------
>
> Key: HDDS-16136
> URL: https://issues.apache.org/jira/browse/HDDS-16136
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Siyao Meng
> Priority: Major
>
> SCM HA groups whose Ratis peers were recorded with IPv4 literal addresses
> cannot move to IPv6-only operation by editing ozone-site.xml alone. SCM
> starts its Ratis server with an empty group and recovers the peers, including
> their addresses, from Raft storage, so the stored IPv4 addresses keep being
> used.
> This is not a blocker for IPv6 support in general. It does not affect:
> * New clusters deployed with IPv6 addresses.
> * Peers configured with DNS names, which remain the preferred migration path
> because the stored hostname resolves to IPv6.
> * Dual-stack clusters that keep IPv4 connectivity.
> * OM HA, which passes its configured peers to Ratis at startup, so its
> connections follow ozone-site.xml. Only the stored and displayed addresses
> may stay stale.
> Recommended approach:
> * Use a Ratis startup option that refreshes the stored peer addresses from
> the peers supplied at startup, as proposed in the review of
> [apache/ratis#1630|https://github.com/apache/ratis/pull/1630] (RATIS-2082 was
> closed as Won't Do). SCM would pass its configured peers when starting its
> Ratis server and enable the option, so ozone-site.xml becomes the single
> source for both the addresses SCM connects to and the addresses Ratis stores.
> A wrong address is fixed by correcting the configuration and restarting. This
> needs the option in a Ratis release and an Ozone upgrade to that release.
> Alternatives considered:
> * Register the configured SCM peers with Ratis before the server starts,
> without the startup option. This is an Ozone only change, but it relies on
> Ratis keeping the first address registered for a peer, and the stored and
> displayed addresses would stay stale and could be propagated by later SCM
> membership changes.
> * Replace each SCM through decommission and bootstrap with its new address.
> This works with existing tools but takes many steps per SCM and reduces fault
> tolerance while it runs.
> * Run the offline ratis-shell {{local raftMetaConf}} command on each stopped
> SCM. This works today but is manual and error prone, and a wrong address
> leaves that SCM unable to rejoin. It can be documented as an interim
> procedure until the startup option is available.
> Requirements:
> * Preserve the Ratis group ID, peer IDs, storage, and committed state.
> * Support a rolling restart while both IPv4 and IPv6 addresses are reachable.
> * Allow a wrong address to be corrected by fixing the configuration and
> restarting.
> * Validate leader election and failover after migration.
> * Add SCM migration tests.
> Parent epic: HDDS-15763.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
