Jason O'Sullivan created HDDS-17726:
---------------------------------------
Summary: Ozone CLI fails when hadoop.security.token.service.use_ip
is false
Key: HDDS-17726
URL: https://issues.apache.org/jira/browse/HDDS-17726
Project: Apache Ozone
Issue Type: Bug
Components: Ozone CLI
Affects Versions: 2.1.0
Reporter: Jason O'Sullivan
Assignee: Jason O'Sullivan
h3. Problem
Setting hadoop.security.token.service.use_ip=false in core-site.xml crashes
Ozone CLI commands that contact the cluster with ExceptionInInitializerError
caused by NoClassDefFoundError: org/xbill/DNS/ResolverConfig. Local commands
that do not bootstrap a client (such as ozone sh --help) are unaffected.
h3. Root Cause
1. CLI commands that communicate with the cluster construct an RpcClient, which
invokes UserGroupInformation.getCurrentUser() during client bootstrap.
2. Initializing UserGroupInformation calls
SecurityUtil.getAuthenticationMethod(), triggering the static initializer of
SecurityUtil.
3. With hadoop.security.token.service.use_ip=false configured, SecurityUtil's
static block executes setTokenServiceUseIp(false), which instantiates
QualifiedHostResolver.
4. QualifiedHostResolver requires org.xbill.DNS.ResolverConfig from dnsjava.
5. hdds-hadoop-dependency-client explicitly excludes dnsjava from
hadoop-common. Because CLI modules inherit through this dependency tree without
restoring it, the class is absent from the CLI runtime classpath.
6. The CLI fails with an ExceptionInInitializerError during early bootstrap,
before reaching OMProxyInfo or executing buildTokenService.
h3. Impact
This blocks secure cluster configurations that rely on
ozone.client.failover.resolve-needed.
When security is enabled, that setting requires
hadoop.security.token.service.use_ip=false. Under the default use_ip=true,
delegation token service keys are bound to IP:port. When an OzoneManager's IP
changes (e.g., following a DNS refresh), OzoneDelegationTokenSelector fails to
select the existing token because the stored token key does not match the new
IP service string. Setting use_ip=false keeps service keys as stable
hostname:port identifiers, but currently causes any CLI command connecting to
the cluster to crash.
h3. Why Tests Did Not Catch This
SecurityUtil.setTokenServiceUseIp(false) is already exercised in tests like
TestDelegationToken within ozone-integration-test. Those tests pass because
ozone-integration-test pulls in standard hadoop-common test dependencies
without the client exclusion, placing dnsjava on the test classpath. What is
currently uncovered is the CLI packaging, where dnsjava is excluded.
h3. Proposed Solution
1. Remove the dnsjava exclusion from hadoop-common in
hadoop-hdds/hadoop-dependency-client/pom.xml.
2. Add a unit test in ozone-cli-admin (which inherits the exclusion and does
not contain dnsjava on its classpath). The test should:
- Apply a configuration containing
hadoop.security.token.service.use_ip=false via
SecurityUtil.setConfiguration(conf).
- Verify that NetUtils.createSocketAddr, SecurityUtil.buildTokenService, and
OMProxyInfo.getDelegationTokenService() successfully produce a hostname:port
token service string without throwing NoClassDefFoundError.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
