[
https://issues.apache.org/jira/browse/FLINK-41435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LuChengxiang updated FLINK-41435:
---------------------------------
Summary: Hadoop delegation token refresh can target a different UGI than
Hadoop clients (was: SecurityUtils.install should be idempotent for equivalent
security configurations)
> Hadoop delegation token refresh can target a different UGI than Hadoop clients
> ------------------------------------------------------------------------------
>
> Key: FLINK-41435
> URL: https://issues.apache.org/jira/browse/FLINK-41435
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Configuration
> Affects Versions: 2.0.0, 1.20.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0
> Environment: Flink 1.20.x and 2.0 through 2.4, plus master
> (2.5-SNAPSHOT); Kubernetes and standalone application mode with Kerberos and
> HDFS delegation tokens.
> Reporter: LuChengxiang
> Priority: Major
> Labels: pull-request-available
>
> *Background*
> In Kubernetes and standalone application mode, Flink installs the security
> context twice:
> * KubernetesApplicationClusterEntrypoint and
> StandaloneApplicationClusterEntryPoint call
> ClusterEntrypoint.installSecurityContext() before fetching user artifacts.
> * ClusterEntrypoint.startCluster() calls installSecurityContext() again.
> Both calls are part of the normal startup path. Each call creates a new
> SecurityConfiguration and runs the security modules and context factory
> again. With Kerberos this performs another login and can replace the process
> login UGI.
> Components created under the first UGI, for example Hadoop clients used while
> fetching artifacts, then remain bound to a UGI that no longer receives
> refreshed delegation tokens. The refresh path can update a different UGI than
> the one used by those clients, so token refresh can silently stop; the job
> later fails with errors such as SecretManager$InvalidToken or an expired HDFS
> delegation token.
> *How it is triggered*
> This happens in Kubernetes and standalone application mode when Kerberos is
> enabled. The first install is required before fetching user artifacts; the
> second install is part of ClusterEntrypoint.startCluster(). The second call
> cannot simply be removed, because startCluster() still needs a security
> context.
> A naive comparison of the two Configuration objects would also not help:
> ApplicationClusterEntryPoint.configureExecution() adds pipeline.jars,
> pipeline.classpaths and execution.target between the two calls. The
> security-relevant options are unchanged, so the fix must compare only the
> options consumed by the security modules.
> *Root cause*
> SecurityUtils.install() unconditionally runs the security modules and creates
> a new security context. It does not remember the last successfully installed
> configuration, so an equivalent configuration is treated as a real
> re-installation.
> *Proposed change*
> * Give SecurityConfiguration value semantics for the options consumed by the
> bundled security modules: security.*, zookeeper.sasl.*, flink.hadoop.*,
> fs.hdfs.* and io.tmp.dirs. Include the configured security context and module
> factories.
> * Remember the last successfully installed SecurityConfiguration in
> SecurityUtils.
> * Make SecurityUtils.install() synchronized and return early when the new
> configuration equals the installed one.
> * Clear the remembered configuration in SecurityUtils.uninstall().
> * Do not reinstall because of unrelated runtime options such as
> pipeline.jars, pipeline.classpaths or execution.target.
> *Scope*
> This issue only covers idempotent installation. Delegation token
> receiver/provider target-UGI handling, partial-install rollback and
> file-system cleanup during token acquisition are out of scope. The fix keeps
> a single UGI for the two Flink install calls; it does not make the process
> resilient to arbitrary later changes of the global Hadoop login user by user
> code or connectors.
> *Testing*
> * SecurityUtilsInstallTest: an equal configuration after adding pipeline
> options does not reinstall; a different security-relevant configuration does
> reinstall; install after uninstall installs again.
> * SecurityConfigurationTest: security-relevant prefixes affect equality;
> unrelated options do not.
> * Existing SecurityUtilsTest still passes.
> * Verified on master (2.5-SNAPSHOT, Java 17, Hadoop 3.4.3): 16 tests pass.
> *Acceptance criteria*
> * Repeated install() with an equivalent security configuration keeps the
> existing context and modules.
> * A different security-relevant configuration still installs a new context
> and modules.
> * Unrelated runtime options do not trigger reinstallation.
> * uninstall() clears the stored installation state.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
