[
https://issues.apache.org/jira/browse/PIG-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-2979:
-------------------------------
Attachment: PIG-2979.patch
The problem is that the provider-configuration file of hadoop-hdfs.jar
overwrites that of hadoop-common.jar since both files have the same name.
Attached is a patch that includes two changes:
1) bundle slf4j\*.jar in pig.jar
2) exclude hadoop-hdfs.jar from pig.jar. Since pig.jar is meant to be used in
local mode only, I don't think that hadoop-hdfs.jar is needed at all. Please
correct me if you think otherwise.
With the patch, "ant e2e-test-local" as well as "./bin/pig -x local" runs fine.
Thanks!
> ant test-e2e-local fails due to missing run-time dependencies in classpath
> with hadoop-2.0.x
> --------------------------------------------------------------------------------------------
>
> Key: PIG-2979
> URL: https://issues.apache.org/jira/browse/PIG-2979
> Project: Pig
> Issue Type: Sub-task
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2979.patch
>
>
> To reproduce, please run on machine where no Hadoop is installed:
> {code}
> ant clean
> ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir
> -Dharness.cluster.bin=hadoop_script test-e2e-deploy-local -Dhadoopversion=23
> ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir
> -Dharness.cluster.bin=hadoop_script test-e2e-local -Dhadoopversion=23
> {code}
> The ant test-e2e-local fails with the following error:
> {code}
> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
> at
> org.apache.hadoop.security.authentication.util.KerberosName.<clinit>(KerberosName.java:42)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:211)
> at
> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:274)
> at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:531)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:512)
> {code}
> In fact, this is also an issue with running Pig in local mode with the fat
> jar where no hadoop dependencies are available in classpath. For example, the
> following command also fails with the same error:
> {code}
> ant clean jar -Dhadoopversion=23
> ./bin/pig -x local
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
