[ https://issues.apache.org/jira/browse/MAPREDUCE-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242671#comment-13242671 ]
Kihwal Lee commented on MAPREDUCE-4072: --------------------------------------- Users should not include -Djava.library.path in task jvm options. The code change looks good, but we need to make sure this is documented somewhere as an incompatible change. > User set java.library.path seems to overwrite default creating problems > native lib loading > ------------------------------------------------------------------------------------------ > > Key: MAPREDUCE-4072 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4072 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Reporter: Anupam Seth > Assignee: Anupam Seth > Attachments: MAPREDUCE-4072-branch-23.patch > > > This was found by Peeyush Bishnoi. > While running a distributed cache example with Hadoop-0.23, > tasks are failing as follows: > ------------------------------------------------------------------------------------------------------------ > Exception from container-launch: > org.apache.hadoop.util.Shell$ExitCodeException: at > org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at > org.apache.hadoop.util.Shell.run(Shell.java:188) at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at > org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:207) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:241) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at > java.util.concurrent.FutureTask.run(FutureTask.java:138) at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) main : command provided 1 main : user > is <user> > ------------------------------------------------------------------------------------------------------------ > Same Pig script and command work successfully on 0.20 > See this in the stderr: > Exception in thread "main" java.lang.ExceptionInInitializerError > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:247) > at > org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1179) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1149) > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1238) > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1264) > at org.apache.hadoop.security.Groups.(Groups.java:54) > at > org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:178) > at > org.apache.hadoop.security.UserGroupInformation.initUGI(UserGroupInformation.java:252) > at > org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:223) > at > org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:265) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:75) > Caused by: java.lang.RuntimeException: Bailing out since native library > couldn't be loaded > at > org.apache.hadoop.security.JniBasedUnixGroupsMapping.(JniBasedUnixGroupsMapping.java:48) > ... 12 more > Pig command: > $ pig -Dmapred.job.queue.name=<queue> -Dmapred.cache.archives=<archives> > -Dmapred.child.java.opts="-Djava.library.path=./ygeo/lib > -Dip2geo.preLoadLibraries=<some other libs>" -Djava.io.tmpdir=/grid/0/tmp > -Dmapred.create.symlink=yes -Dmapred.job.map.memory.mb=3072 piggeoscript.pig -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira