[ https://issues.apache.org/jira/browse/YARN-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169846#comment-17169846 ]
bianqi edited comment on YARN-10382 at 8/3/20, 9:36 AM: -------------------------------------------------------- Thank you for your reply. Our scenario is that there are two big data clusters, one of which is a non-secure cluster and the other is a secure cluster. Now there is a unified client who wants to submit tasks to the non-secure yarn cluster for access Safe HDFS. Non-secure HDFS and Secure yarn are not the same cluster. When kerberos is enabled on my unified client, the yarn and hdfs of the client are considered safe. But the yarn on the server is not safe. The error is as follows: {quote} java.io.IOException: Can't get Master Kerberos principal for use as renewer at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:138) at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:104) at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:82) at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:142) at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:141) {quote} {quote} static void obtainTokensForNamenodesInternal(Credentials credentials, Path[] ps, Configuration conf) throws IOException { Set<FileSystem> fsSet = new HashSet<FileSystem>(); for(Path p: ps) { fsSet.add(p.getFileSystem(conf)); } String masterPrincipal = Master.getMasterPrincipal(conf); for (FileSystem fs : fsSet) { obtainTokensForNamenodesInternal(fs, credentials, conf, masterPrincipal); } }{quote} masterPrincipal is null. How should I modify the client code so that the client can access non-secure yarn and read and write secure hdfs. Or do I need to modify the server code? was (Author: bianqi): Thank you for your reply. Our scenario is that there are two big data clusters, one of which is a non-secure cluster and the other is a secure cluster. Now there is a unified client who wants to submit tasks to the non-secure yarn cluster for access Safe HDFS. Non-secure HDFS and Secure yarn are not the same cluster. When kerberos is enabled on my unified client, the yarn and hdfs of the client are considered safe. But the yarn on the server is not safe. The error is as follows: {quote} java.io.IOException: Can't get Master Kerberos principal for use as renewer at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:138) at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:104) at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:82) at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:142) at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:141) {quote} How should I modify the client code so that the client can access non-secure yarn and read and write secure hdfs. Or do I need to modify the server code? > Non-secure yarn access secure hdfs > ---------------------------------- > > Key: YARN-10382 > URL: https://issues.apache.org/jira/browse/YARN-10382 > Project: Hadoop YARN > Issue Type: New Feature > Components: yarn > Reporter: bianqi > Priority: Minor > > In our production environment, yarn cannot enable kerberos due to yarn > environment problems, but our hdfs is to enable kerberos, and now we need > non-secure yarn to access secure hdfs. > It is known that yarn and hdfs are both safe after security is turned on. > I hope that after enabling hdfs security, you can use non-secure yarn to > access secure hdfs, or use secure yarn to access non-secure hdfs. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org