[ https://issues.apache.org/jira/browse/HDFS-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053571#comment-15053571 ]
Daryn Sharp commented on HDFS-9525: ----------------------------------- bq. The whole underlying point of this JIRA is to fix WebHDFS which ... doesn't work with multiple Kerberos realms # There's nothing wrong with spnego. # There's nothing wrong with webhdfs. # You can't fix something that's not broken by breaking it. Kerberos wasn't designed to be used in a non-trusting environment. All web clients, ex. curl the jdk's, I'm aware of fail if a webserver in 1 realm redirects to another webserver in a 2nd non-trusted realm. They don't have multiple identity/TGT support. You've discovered that feeding in tokens via an external tool like fetchdt is rather painful. So you probably worked backwards from the end goal. Used fetchdt as the remote identity to get a token, fed that via a token cache into your distcps, became dismayed that it eventually expired, possibly had to restart a deamon to read in a new token cache every week or two, thus ultimately decided to break the security model to allow getting new tokens from prior tokens. What I would do is write a wrapper over distcp and change nothing in hadoop core. Off the top of my head, something like: {noformat} UserGroupInformation.loginUserFromKeytab("local-principal", keytab); UserGroupInformation whyDoYouNotTrustMe = UserGroupInformation.loginUserFromKeytabAndReturnUGI("other-principal", keytab); whyDoYouNotTrustMe.doAs( new PrivilegedExceptionAction<Void>() { @Override public Void run() throws IOException { remotePath.getFileSystem(conf).addDelegationTokens( renewer, UserGroupInformation.getLoginUser().getCredentials()); return null; }}); DistCpOptions options = new DistCpOptions(listFile, target); options.setXYZ(...); new DistCp(conf, options).execute(); {noformat} Snapping back the token files if that's the route you chose to follow, rather than a yack (yet another config key), why not add an option to fetchdt to add tokens to a file instead of overwriting the entire file? > hadoop utilities need to support provided delegation tokens > ----------------------------------------------------------- > > Key: HDFS-9525 > URL: https://issues.apache.org/jira/browse/HDFS-9525 > Project: Hadoop HDFS > Issue Type: New Feature > Components: security > Affects Versions: 3.0.0 > Reporter: Allen Wittenauer > Assignee: HeeSoo Kim > Priority: Blocker > Fix For: 3.0.0 > > Attachments: HDFS-7984.001.patch, HDFS-7984.002.patch, > HDFS-7984.003.patch, HDFS-7984.004.patch, HDFS-7984.005.patch, > HDFS-7984.006.patch, HDFS-7984.007.patch, HDFS-7984.patch > > > When using the webhdfs:// filesystem (especially from distcp), we need the > ability to inject a delegation token rather than webhdfs initialize its own. > This would allow for cross-authentication-zone file system accesses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)