[ https://issues.apache.org/jira/browse/PIG-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2979: ------------------------------- Attachment: PIG-2979.patch The problem is that the provider-configuration file of hadoop-hdfs.jar overwrites that of hadoop-common.jar since both files have the same name. Attached is a patch that includes two changes: 1) bundle slf4j\*.jar in pig.jar 2) exclude hadoop-hdfs.jar from pig.jar. Since pig.jar is meant to be used in local mode only, I don't think that hadoop-hdfs.jar is needed at all. Please correct me if you think otherwise. With the patch, "ant e2e-test-local" as well as "./bin/pig -x local" runs fine. Thanks! > ant test-e2e-local fails due to missing run-time dependencies in classpath > with hadoop-2.0.x > -------------------------------------------------------------------------------------------- > > Key: PIG-2979 > URL: https://issues.apache.org/jira/browse/PIG-2979 > Project: Pig > Issue Type: Sub-task > Reporter: Cheolsoo Park > Assignee: Cheolsoo Park > Fix For: 0.11 > > Attachments: PIG-2979.patch > > > To reproduce, please run on machine where no Hadoop is installed: > {code} > ant clean > ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir > -Dharness.cluster.bin=hadoop_script test-e2e-deploy-local -Dhadoopversion=23 > ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir > -Dharness.cluster.bin=hadoop_script test-e2e-local -Dhadoopversion=23 > {code} > The ant test-e2e-local fails with the following error: > {code} > java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory > at > org.apache.hadoop.security.authentication.util.KerberosName.<clinit>(KerberosName.java:42) > at > org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:211) > at > org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:274) > at > org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:531) > at > org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:512) > {code} > In fact, this is also an issue with running Pig in local mode with the fat > jar where no hadoop dependencies are available in classpath. For example, the > following command also fails with the same error: > {code} > ant clean jar -Dhadoopversion=23 > ./bin/pig -x local > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira