[
https://issues.apache.org/jira/browse/IGNITE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-28867:
--------------------------------------
Description:
Allow administrators to replace node TLS certificates and trust stores without
restarting nodes. When an external system delivers certificates, let Ignite
load valid replacements automatically before the current chains expire.
Existing connections remain usable; administrators prepare peer trust and
verify new connections.
h3. Parts and consumers
# IGNITE-29208 preserves the peer address of wrapped SSL engines. This
independently fixes existing addressed clients; [PR
#13750|https://github.com/apache/ignite/pull/13750] is merged.
# IGNITE-29209 provides a factory-owned context and certificate snapshot from
one store read. Explicit {{init()}} publishes replacements; repeated
{{create()}}/{{sslBundle()}} calls reuse accepted material. Reload/status and
renewal consume this API. [PR
#13751|https://github.com/apache/ignite/pull/13751] covers this part.
# IGNITE-29210 adds operator reload/status for discovery, communication, the
client connector and binary REST, with replacement checks, per-node results,
permissions and audit. It supplies the shared mechanism used by the remaining
tasks.
# IGNITE-29211 schedules opt-in certificate renewal, retries and monitoring.
The external delivery system remains responsible for issuing and supplying
replacements.
# IGNITE-29212 connects HTTP REST to reload/status. Together with IGNITE-29211,
it also supports automatic HTTPS certificate renewal.
h3. Merge and integration order
Merge IGNITE-29209, then open and merge IGNITE-29210. Open IGNITE-29211 and
IGNITE-29212 after that; they may merge in either order.
After IGNITE-29211 and IGNITE-29212 are both present, include the single [joint
integration
commit|https://github.com/anton-vinogradov/ignite/commit/e1a7c86c428cf775ef86750bfdca746d1cdeb88c]
in the PR that merges second. Apply it once, without replaying older
integration patches. It adds HTTPS automatic-renewal coverage and the
corresponding documentation/Javadoc. Run {{JettySslContextReloadTest}} and
{{SslRenewalNodeTest}} on that combined branch.
After a prerequisite is squash-merged, preserve the prepared branch and record
its exact prerequisite head. Create the dependent review branch from current
master and apply only its diff against that recorded head. If a PR is already
open, merge master normally. This keeps each review limited to its task without
force-pushing prepared branches.
h3. Completion
Each part must include its tests and user/developer documentation. Run targeted
checks on refreshed heads and verify the combined solution with mutual TLS,
trust changes, existing configurations and real automatic HTTPS renewal.
Preserve HTTP's configured TLS parameters when combining the branches. The API
has current local evidence for two startup tests followed by 16 API tests,
Checkstyle and doclint; this does not establish full CI for the child or
combined branches.
Final heads and combined validation: API {{f6b57b36}}, manual {{c4247675}},
renewal {{34b1f7a7}}, HTTP {{f313ae1e}}. The combined integration {{ed06380c}}
passed 86 targeted SSL tests with two existing JMX variants skipped and
Checkstyle passed. These are local checks, not full CI.
Follow [certificate rollout and
recovery|https://github.com/anton-vinogradov/ignite/blob/ignite-29210/docs/_docs/security/ssl-tls.adoc#upgrading-certificates]
and [factory
migration|https://github.com/anton-vinogradov/ignite/blob/ignite-29210/docs/_docs/security/ssl-tls.adoc#migrating-an-existing-factory].
Factory subclass compatibility changes belong to IGNITE-29209; transport and
Jetty migration limits are documented in their owning tasks. Close the umbrella
after all parts merge and combined validation passes.
The separate {{KillCommandsControlShTest}} stability fix is prepared on
{{ignite-consistency-cancellation-status}} for independent review. At
{{54b82830}}, both requested CLI tests passed; the two existing JMX variants
were skipped, and Checkstyle passed. It is not part of the TLS changes or a
dependency of this umbrella.
was:
Allow administrators to replace TLS certificates and trust stores without
restarting nodes. When an external system supplies new certificates, allow
Ignite to load them automatically before the current chain expires.
h3. What each part provides and what uses it
# IGNITE-29208 preserves the peer address when SSLContextWrapper creates an
engine. This independently fixes existing addressed client engines. [PR
#13750|https://github.com/apache/ignite/pull/13750] is merged.
# IGNITE-29209 makes the factory own an accepted SslBundle built from one
loaded KeyMaterial, containing the context and nonempty X.509 chains from key
entries. create() returns a stable forwarding context; init() loads and
publishes replacement material. Entries without such a chain are omitted.
IGNITE-29210 checks the dates of included chains, reports them in status and
rejects a known empty chain map; a mixed store is not rejected merely because
other entries were skipped. IGNITE-29211 uses the earliest expiry across all
reported chains to schedule renewal. SSLContextWrapper handles TLS parameters
only. [PR #13751|https://github.com/apache/ignite/pull/13751] is open.
# IGNITE-29210 registers factories and their transports and provides manual
reload and status for discovery, communication, the client connector and binary
REST. Its provider delegates to factory-owned state. Factory init() validates
and replaces the context and metadata together, keeping transports sharing a
factory on the same certificates. IGNITE-29211 reuses this replacement
mechanism for scheduled attempts; IGNITE-29212 connects HTTP REST to it.
# IGNITE-29211 adds a factory-owned renewal planner and unsuccessful-attempt
counter, using an executor owned and stopped by the node, plus the
ssl.ConsecutiveRenewalFailures metric. The factory installs an automatic
replacement only when every reported chain is valid and the minimum expiry
across all reported chains advances. A direct init() call also replans renewal.
The certificate source remains responsible for supplying replacements.
# IGNITE-29212 adds httpSslFactory so Jetty HTTP REST can use managed contexts.
This makes HTTPS certificates available to reload and status, and to automatic
renewal once IGNITE-29211 is present.
The factory API change is contained in IGNITE-29209: create() is final,
initializes on first use and then returns the same forwarding context without
rereading stores. init() loads a candidate through createSslBundle() and
publishes its context and metadata together; a failed initialization preserves
the accepted bundle. sslBundle() returns that accepted snapshot. The protected
sslCtx field and no-argument createKeyManagers() hook are removed; affected
subclasses must move store loading to loadKeyMaterial(), remove custom caching
and be recompiled.
Custom key managers require a createSslBundle() override that initializes a
fresh context, configures its trust and TLS settings, and returns matching
certificate metadata. To decorate an already built context, call
super.createSslBundle() and retain its certificate chains in the returned
bundle. Loaded stores must remain unchanged while their contexts are in use.
Sources providing ready-made contexts can use plain Factory<SSLContext> with
unknown metadata; IGNITE-29210 adapts them while retaining the original factory
identity for registration. Communication keeps its context between connections.
Ownership of the accepted bundle remains in the factory throughout the later
parts. Newly created engines and newly obtained socket factories use
replacement material; previously obtained engines, socket factories and sockets
retain their original material.
h3. Review and merge order
All five subtasks are created and their branches are prepared. IGNITE-29208 is
merged. Review and merge IGNITE-29209 next; certificate metadata is carried by
SslBundle and does not depend on SSLContextWrapper.
Open the PR for IGNITE-29210 after IGNITE-29209 merges. Open IGNITE-29211 and
IGNITE-29212 after IGNITE-29210 merges. Automatic renewal and HTTP REST may
then merge in either order.
After both IGNITE-29211 and IGNITE-29212 are present, include the single
[prepared integration
commit|https://github.com/anton-vinogradov/ignite/commit/e1a7c86c428cf775ef86750bfdca746d1cdeb88c]
in the PR that merges second. It replaces the old integration patch list. It
adds HTTP REST renewal documentation and Javadoc and an HTTPS test showing that
automatic renewal presents the replacement certificate on a new connection. Do
not apply earlier integration commits as well. Before merging that second PR,
run JettySslContextReloadTest and SslRenewalNodeTest.
h3. Updating branches after a squash merge
Before opening a dependent PR, create a new review branch from the current
master. Keep the prepared branch and record the exact head of its prerequisite
that was squash-merged. For example, prepare IGNITE-29210 after IGNITE-29209
has been squash-merged. Here upstream is apache/ignite and origin is the
contributor fork:
{code:bash}
git fetch upstream master
git fetch origin refs/heads/ignite-29210:refs/remotes/origin/ignite-29210
git switch -c ignite-29210-review upstream/master
git diff <recorded-ignite-29209-head> origin/ignite-29210 | git apply --index
git diff --cached --check
git diff --cached
{code}
Resolve any rejected hunks, run the relevant tests, then commit and push the
new review branch. This preserves the prepared branches and requires no
force-push. If a PR is already open, merge master into its branch normally
instead. When integrating the API change with master, retain the SSL wrapper
regression and new SSL test registrations in SecurityTestSuite.
When combining automatic renewal and HTTP REST, retain
CurrentSslContext.engineGetDefaultSSLParameters(). It must forward to the
active context so Jetty retains the configured protocols and cipher suites.
JettySslContextReloadTest checks the protocols and cipher suites that Jetty
uses.
h3. Completion criteria
Each PR includes its tests and documentation and passes CI on its current head.
Update dependent branches after their prerequisites merge so reviewers see only
the changes belonging to that task.
The initialization API is published in PR #13751 at 22e3ec58. The combined
implementation passed 86 targeted SSL tests locally, with two existing JMX
variants skipped. These local checks and CI results from earlier revisions do
not establish a full CI result for the current rework.
Reload checks available certificate metadata and supported TLS settings.
Administrators remain responsible for peer trust and testing new connections.
Close this umbrella after all five parts merge and the combined solution passes
SSL tests, including mutual TLS and compatibility with existing configurations.
[PR #13335|https://github.com/apache/ignite/pull/13335] is the original
prototype from which these changes were split. Its factory API is superseded by
IGNITE-29209.
> [Umbrella] TLS certificate reload and automatic renewal
> -------------------------------------------------------
>
> Key: IGNITE-28867
> URL: https://issues.apache.org/jira/browse/IGNITE-28867
> Project: Ignite
> Issue Type: Task
> Reporter: Mikhail Petrov
> Assignee: Anton Vinogradov
> Priority: Major
> Labels: ise
> Fix For: 2.19
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Allow administrators to replace node TLS certificates and trust stores
> without restarting nodes. When an external system delivers certificates, let
> Ignite load valid replacements automatically before the current chains
> expire. Existing connections remain usable; administrators prepare peer trust
> and verify new connections.
> h3. Parts and consumers
> # IGNITE-29208 preserves the peer address of wrapped SSL engines. This
> independently fixes existing addressed clients; [PR
> #13750|https://github.com/apache/ignite/pull/13750] is merged.
> # IGNITE-29209 provides a factory-owned context and certificate snapshot from
> one store read. Explicit {{init()}} publishes replacements; repeated
> {{create()}}/{{sslBundle()}} calls reuse accepted material. Reload/status and
> renewal consume this API. [PR
> #13751|https://github.com/apache/ignite/pull/13751] covers this part.
> # IGNITE-29210 adds operator reload/status for discovery, communication, the
> client connector and binary REST, with replacement checks, per-node results,
> permissions and audit. It supplies the shared mechanism used by the remaining
> tasks.
> # IGNITE-29211 schedules opt-in certificate renewal, retries and monitoring.
> The external delivery system remains responsible for issuing and supplying
> replacements.
> # IGNITE-29212 connects HTTP REST to reload/status. Together with
> IGNITE-29211, it also supports automatic HTTPS certificate renewal.
> h3. Merge and integration order
> Merge IGNITE-29209, then open and merge IGNITE-29210. Open IGNITE-29211 and
> IGNITE-29212 after that; they may merge in either order.
> After IGNITE-29211 and IGNITE-29212 are both present, include the single
> [joint integration
> commit|https://github.com/anton-vinogradov/ignite/commit/e1a7c86c428cf775ef86750bfdca746d1cdeb88c]
> in the PR that merges second. Apply it once, without replaying older
> integration patches. It adds HTTPS automatic-renewal coverage and the
> corresponding documentation/Javadoc. Run {{JettySslContextReloadTest}} and
> {{SslRenewalNodeTest}} on that combined branch.
> After a prerequisite is squash-merged, preserve the prepared branch and
> record its exact prerequisite head. Create the dependent review branch from
> current master and apply only its diff against that recorded head. If a PR is
> already open, merge master normally. This keeps each review limited to its
> task without force-pushing prepared branches.
> h3. Completion
> Each part must include its tests and user/developer documentation. Run
> targeted checks on refreshed heads and verify the combined solution with
> mutual TLS, trust changes, existing configurations and real automatic HTTPS
> renewal. Preserve HTTP's configured TLS parameters when combining the
> branches. The API has current local evidence for two startup tests followed
> by 16 API tests, Checkstyle and doclint; this does not establish full CI for
> the child or combined branches.
> Final heads and combined validation: API {{f6b57b36}}, manual {{c4247675}},
> renewal {{34b1f7a7}}, HTTP {{f313ae1e}}. The combined integration
> {{ed06380c}} passed 86 targeted SSL tests with two existing JMX variants
> skipped and Checkstyle passed. These are local checks, not full CI.
> Follow [certificate rollout and
> recovery|https://github.com/anton-vinogradov/ignite/blob/ignite-29210/docs/_docs/security/ssl-tls.adoc#upgrading-certificates]
> and [factory
> migration|https://github.com/anton-vinogradov/ignite/blob/ignite-29210/docs/_docs/security/ssl-tls.adoc#migrating-an-existing-factory].
> Factory subclass compatibility changes belong to IGNITE-29209; transport and
> Jetty migration limits are documented in their owning tasks. Close the
> umbrella after all parts merge and combined validation passes.
> The separate {{KillCommandsControlShTest}} stability fix is prepared on
> {{ignite-consistency-cancellation-status}} for independent review. At
> {{54b82830}}, both requested CLI tests passed; the two existing JMX variants
> were skipped, and Checkstyle passed. It is not part of the TLS changes or a
> dependency of this umbrella.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
