[ https://issues.apache.org/jira/browse/HADOOP-14441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16017966#comment-16017966 ]
Yongjun Zhang commented on HADOOP-14441: ---------------------------------------- No problem [~jojochuang]. Thanks for the updated patch. I looked at have a high level comment: Looks to me that the following operations need to have similar fix, given a token to renew and cancel, we can either derive the KMS from the service field in the token, and operate on this KMS directly, or use a loop like the one you changed with addDelegationToken. {code} @Override public long renewDelegationToken(final Token<?> token) throws IOException { return doOp(new ProviderCallable<Long>() { @Override public Long call(KMSClientProvider provider) throws IOException { return provider.renewDelegationToken(token); } }, nextIdx()); } @Override public Void cancelDelegationToken(final Token<?> token) throws IOException { return doOp(new ProviderCallable<Void>() { @Override public Void call(KMSClientProvider provider) throws IOException { provider.cancelDelegationToken(token); return null; } }, nextIdx()); } {code} Do you agree? Thanks. > LoadBalancingKMSClientProvider#addDelegationTokens should add delegation > tokens from all KMS instances > ------------------------------------------------------------------------------------------------------ > > Key: HADOOP-14441 > URL: https://issues.apache.org/jira/browse/HADOOP-14441 > Project: Hadoop Common > Issue Type: Bug > Components: kms > Affects Versions: 2.7.0 > Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption > Reporter: Wei-Chiu Chuang > Assignee: Wei-Chiu Chuang > Attachments: HADOOP-14441.001.patch, HADOOP-14441.002.patch > > > LoadBalancingKMSClientProvider only gets delegation token from one KMS > instance, in a round-robin fashion. This is arguably a bug, as JavaDoc for > {{KeyProviderDelegationTokenExtension#addDelegationTokens}} states: > {quote} > /** > * The implementer of this class will take a renewer and add all > * delegation tokens associated with the renewer to the > * <code>Credentials</code> object if it is not already present, > ... > **/ > {quote} > This bug doesn't pop up very often, because HDFS clients such as MapReduce > unintentionally calls {{FileSystem#addDelegationTokens}} multiple times. > We have a custom client that accesses HDFS/KMS-HA using delegation token, and > we were puzzled why it always throws "Failed to find any Kerberos tgt" > exceptions talking to one KMS but not the other. Turns out that client > couldn't talk to the KMS because {{FileSystem#addDelegationTokens}} only gets > one KMS delegation token at a time. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org