Hi, I tried to run secure containers in cluster, and followed the instructions from http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/SecureMode.html#LinuxContainerExecutor After that, I ran the mapreduce example pi and got the following exception:
2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : command provided 0 2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : run as user is hadoop 2017-02-06 17:46:24,018 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : requested yarn user is hadoop 2017-02-06 17:46:24,019 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Failed to create directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop - No such file or directory 2017-02-06 17:46:24,019 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer failed java.io.IOException: Application application_1486374303689_0001 initialization failed (exitCode=20) with output: main : command provided 0 main : run as user is hadoop main : requested yarn user is hadoop Failed to create directory /data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm/usercache/hadoop - No such file or directory at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:241) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088) Caused by: ExitCodeException exitCode=20: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:232) ... 1 more Here is the container-executor.cfg under /etc/hadoop owned by root (configurations are same to the yarn-site.xml): yarn.nodemanager.local-dirs=/data1/yarn/nm,/data2/yarn/nm,/data3/yarn/nm yarn.nodemanager.log-dirs=/data1/hadoop/logs/container yarn.nodemanager.linux-container-executor.group=hadoop banned.users=yarn min.user.id=500 allowed.system.users=hadoop,dm Hope to get some helps. Regards, Reid Chan.