[ https://issues.apache.org/jira/browse/HBASE-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guanghao Zhang reassigned HBASE-24403: -------------------------------------- Assignee: wuchang > FsDelegationToken should cache Token > ------------------------------------ > > Key: HBASE-24403 > URL: https://issues.apache.org/jira/browse/HBASE-24403 > Project: HBase > Issue Type: Bug > Reporter: wuchang > Assignee: wuchang > Priority: Major > Attachments: 24403.patch > > > When doing Bulkload, we find that FsDelegationToken will acquire token of > NameNode everytime for a single file, although, from the comment of > acquireDelegationToken(), it claims that it is trying to find token in cache > firstly, but the newly requested token is never put to cache and thus the > cache is still empty for the following request; > In cases there are many files to do the bulk load, the token request will > cause big short to NameNode. > > {code:java} > public void acquireDelegationToken(final FileSystem fs) > throws IOException { > if (userProvider.isHadoopSecurityEnabled()) { > this.fs = fs; > userToken = userProvider.getCurrent().getToken("HDFS_DELEGATION_TOKEN", > fs.getCanonicalServiceName()); > if (userToken == null) { > hasForwardedToken = false; > try { > userToken = fs.getDelegationToken(renewer); > } catch (NullPointerException npe) { > // we need to handle NullPointerException in case HADOOP-10009 is missing > LOG.error("Failed to get token for " + renewer); > } > } else { > hasForwardedToken = true; > LOG.info("Use the existing token: " + userToken); > } > } > }{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)