[
https://issues.apache.org/jira/browse/FLINK-29622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl reopened FLINK-29622:
-----------------------------------
I noticed a similar warning when looking into FLINK-30507 for the
{{HBaseDeletionTokenProvider}}:
{code}
03:55:25,784 [ main] WARN
org.apache.flink.runtime.security.token.DefaultDelegationTokenManager [] -
Failed to initialize delegation token provider hbase
java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/HdfsConfiguration
at
org.apache.flink.runtime.security.token.hadoop.HBaseDelegationTokenProvider.getHBaseConfiguration(HBaseDelegationTokenProvider.java:69)
~[flink-runtime-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.security.token.hadoop.HBaseDelegationTokenProvider.init(HBaseDelegationTokenProvider.java:61)
~[flink-runtime-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.loadProviders(DefaultDelegationTokenManager.java:106)
~[flink-runtime-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManager.<init>(DefaultDelegationTokenManager.java:91)
~[flink-runtime-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.security.token.DefaultDelegationTokenManagerFactory.create(DefaultDelegationTokenManagerFactory.java:48)
~[flink-runtime-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:431)
~[flink-runtime-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.testutils.MiniClusterResource.startMiniCluster(MiniClusterResource.java:237)
~[flink-runtime-1.17-SNAPSHOT-tests.jar:1.17-SNAPSHOT]
at
org.apache.flink.runtime.testutils.MiniClusterResource.before(MiniClusterResource.java:109)
~[flink-runtime-1.17-SNAPSHOT-tests.jar:1.17-SNAPSHOT]
at
org.apache.flink.test.util.MiniClusterWithClientResource.before(MiniClusterWithClientResource.java:64)
~[flink-test-utils-1.17-SNAPSHOT.jar:1.17-SNAPSHOT]
at
org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase.setupTestCluster(EventTimeWindowCheckpointingITCase.java:253)
~[test-classes/:?]
at
org.apache.flink.test.checkpointing.LocalRecoveryITCase.executeTest(LocalRecoveryITCase.java:73)
~[test-classes/:?]
at
org.apache.flink.test.checkpointing.LocalRecoveryITCase.executeTest(LocalRecoveryITCase.java:66)
~[test-classes/:?]
[...]
{code}
Is this something to look into as part of this Jira issue or shall I create a
follow-up ticket?
> KerberosDelegationTokenManager fails to load DelegationTokenProvider due to
> NoClassDefFoundError in various ITCases
> -------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-29622
> URL: https://issues.apache.org/jira/browse/FLINK-29622
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
> Affects Versions: 1.16.0, 1.17.0
> Reporter: Matthias Pohl
> Assignee: Gabor Somogyi
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.16.0
>
>
> There are multiple ITCases (e.g. {{EventTimeWindowCheckpointingITCase}}) that
> print an error when trying to load the {{HadoopFSDelegationTokenProvider}}
> which is on the classpath through {{flink-runtime}} but the corresponding
> hadoop dependency seems to be missing:
> {code}
> 186348 02:25:25,492 [ main] INFO
> org.apache.flink.runtime.security.token.KerberosDelegationTokenManager [] -
> Loading delegation token providers
> 186349 02:25:25,493 [ main] ERROR
> org.apache.flink.runtime.security.token.KerberosDelegationTokenManager [] -
> Failed to initialize delegation token provider hadoopfs
> 186350 java.lang.NoClassDefFoundError:
> org/apache/hadoop/hdfs/HdfsConfiguration
> 186351 at
> org.apache.flink.runtime.security.token.HadoopFSDelegationTokenProvider.init(HadoopFSDelegationTokenProvider.java:68)
> ~[flink-runtime-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186352 at
> org.apache.flink.runtime.security.token.KerberosDelegationTokenManager.loadProviders(KerberosDelegationTokenManager.java:124)
> ~[flink-runtime-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186353 at
> org.apache.flink.runtime.security.token.KerberosDelegationTokenManager.<init>(KerberosDelegationTokenManager.java:109)
> ~[flink-runtime-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186354 at
> org.apache.flink.runtime.security.token.KerberosDelegationTokenManager.<init>(KerberosDelegationTokenManager.java:91)
> ~[flink-runtime-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186355 at
> org.apache.flink.runtime.security.token.KerberosDelegationTokenManagerFactory.create(KerberosDelegationTokenManagerFactory.java:47)
> ~[flink-runtime-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186356 at
> org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:431)
> ~[flink-runtime-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186357 at
> org.apache.flink.runtime.testutils.MiniClusterResource.startMiniCluster(MiniClusterResource.java:234)
> ~[flink-runtime-1.16-SNAPSHOT-tests.jar:1.16-SNAPSHOT]
> 186358 at
> org.apache.flink.runtime.testutils.MiniClusterResource.before(MiniClusterResource.java:109)
> ~[flink-runtime-1.16-SNAPSHOT-tests.jar:1.16-SNAPSHOT]
> 186359 at
> org.apache.flink.test.util.MiniClusterWithClientResource.before(MiniClusterWithClientResource.java:64)
> ~[flink-test-utils-1.16-SNAPSHOT.jar:1.16-SNAPSHOT]
> 186360 at
> org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase.setupTestCluster(EventTimeWindowCheckpointingITCase.java:253)
> ~[test-classes/:?]
> 186361 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) ~[?:1.8.0_292]
> [...]
> {code}
> This error might be misleading/confusing to people investigating the logs. It
> looks like this error is actually expected since the tests not necessarily
> require Kerberos delegation tokens.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
