[ https://issues.apache.org/jira/browse/MAPREDUCE-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13947317#comment-13947317 ]
Mohammad Kamrul Islam commented on MAPREDUCE-5799: -------------------------------------------------- +1 [~liangly] : Please assign the JIRA to yourself. > add default value of MR_AM_ADMIN_USER_ENV > ----------------------------------------- > > Key: MAPREDUCE-5799 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5799 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Affects Versions: 2.3.0 > Reporter: Liyin Liang > Priority: Minor > Attachments: MAPREDUCE-5799.diff > > > Submit a 1 map + 1 reduce sleep job with the following config: > {code} > <property> > <name>mapreduce.map.output.compress</name> > <value>true</value> > </property> > <property> > <name>mapreduce.map.output.compress.codec</name> > <value>org.apache.hadoop.io.compress.SnappyCodec</value> > </property> > <property> > <name>mapreduce.job.ubertask.enable</name> > <value>true</value> > </property> > {code} > And the LinuxContainerExecutor is enable on NodeManager. > This job will fail with the following error: > {code} > 2014-03-18 21:28:20,153 FATAL [uber-SubtaskRunner] > org.apache.hadoop.mapred.LocalContainerLauncher: Error running local > (uberized) 'child' : java.lang.UnsatisfiedLinkError: > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native > Method) > at > org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63) > at > org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:132) > at > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:148) > at > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:163) > at org.apache.hadoop.mapred.IFile$Writer.<init>(IFile.java:115) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1583) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1462) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:700) > at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:1990) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:774) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > at > org.apache.hadoop.mapred.LocalContainerLauncher$SubtaskRunner.runSubtask(LocalContainerLauncher.java:317) > at > org.apache.hadoop.mapred.LocalContainerLauncher$SubtaskRunner.run(LocalContainerLauncher.java:232) > at java.lang.Thread.run(Thread.java:662) > {code} > When create a ContainerLaunchContext for task in > TaskAttemptImpl.createCommonContainerLaunchContext(), the > DEFAULT_MAPRED_ADMIN_USER_ENV which is > "LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native" is added to the environment. > Where when create a ContainerLaunchContext for mrappmaster in > YARNRunner.createApplicationSubmissionContext(), there is no default > environment. So the ubermode job fails to find native lib. -- This message was sent by Atlassian JIRA (v6.2#6252)