Bonjour,
I'm in the process of installing pig on a hadoop cluster (2.6) and do
have the following issue :
5 steps :
1\ Starting the pig command on the server
2\ adding hadoop in the PIG_CLASSPATH=/usr/local/hadoop/etc/hadoop
3\ adding the HADOOP_HOME=/usr/local/hadoop
4\ Hadoop version is 2.6.0
5\ "PIG secret command"...
Either I have the "NoClassDefFoundError: org/joda/time/ReadableInstant"
issue if I setup the HADOOP_HOME either I do have the Server IPC version
9 cannot communicate with clientversion 4
I've spent some time today digging around this iusse without any chance.
Any help appreciated :-)
Thanks
All steps below
1\ Starting the pig command on the server
15/07/14 15:14:58 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
15/07/14 15:14:58 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
15/07/14 15:14:58 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the
ExecType
2015-07-14 15:14:58,521 [main] INFO org.apache.pig.Main - Apache Pig
version 0.15.0 (r1682971) compiled Jun 01 2015, 11:43:55
2015-07-14 15:14:58,521 [main] INFO org.apache.pig.Main - Logging error
messages to: /usr/local/pig/pig-0.15.0/bin/pig_1436886898515.log
2015-07-14 15:14:58,581 [main] INFO org.apache.pig.impl.util.Utils -
Default bootup file /home/pig/.pigbootup not found
2015-07-14 15:14:58,772 [main] ERROR org.apache.pig.Main - ERROR 4010:
Cannot find hadoop configurations in classpath (neither hadoop-site.xml
nor core-site.xml was found in the classpath). If you plan to use local
mode, please put -x local option in command line
Details at logfile: /usr/local/pig/pig-0.15.0/bin/pig_1436886898515.log
2015-07-14 15:14:58,832 [main] INFO org.apache.pig.Main - Pig script
completed in 724 milliseconds (724 ms)
2\ adding hadoop in the PIG_CLASSPATH=/usr/local/hadoop/etc/hadoop
pig@master:/usr/local/pig/pig-0.15.0/bin$ pig
15/07/14 15:20:20 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
15/07/14 15:20:20 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
15/07/14 15:20:20 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the
ExecType
2015-07-14 15:20:20,287 [main] INFO org.apache.pig.Main - Apache Pig
version 0.15.0 (r1682971) compiled Jun 01 2015, 11:43:55
2015-07-14 15:20:20,287 [main] INFO org.apache.pig.Main - Logging error
messages to: /usr/local/pig/pig-0.15.0/bin/pig_1436887220285.log
2015-07-14 15:20:20,333 [main] INFO org.apache.pig.impl.util.Utils -
Default bootup file /home/pig/.pigbootup not found
2015-07-14 15:20:20,759 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: hdfs://master:54310
2015-07-14 15:20:21,600 [main] ERROR org.apache.pig.Main - ERROR 2999:
Unexpected internal error. Failed to create DataStorage
Details at logfile: /usr/local/pig/pig-0.15.0/bin/pig_1436887220285.log
2015-07-14 15:20:21,648 [main] INFO org.apache.pig.Main - Pig script
completed in 1 second and 703 milliseconds (1703 ms)
Log : Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC
version 9 cannot communicate with client version 4
3\ adding the HADOOP_HOME=/usr/local/hadoop
pig@master:/usr/local/pig/pig-0.15.0/bin$ pig
Exception in thread "main" java.lang.NoClassDefFoundError:
org/joda/time/ReadableInstant
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.joda.time.ReadableInstant
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 4 more
4\ Hadoop version is 2.6.0
hduser@master:/usr/local/pig$ hadoop version
Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
5\ "PIG secret command"...
Find hadoop at /usr/local/hadoop/bin/hadoop
dry run:
HADOOP_CLASSPATH:
/usr/local/pig/pig-0.15.0/bin/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/etc/hadoop:/usr/local/pig/pig-0.15.0/bin/../lib/ST4-4.0.4.jar:/usr/local/pig/pig-0.15.0/bin/../lib/accumulo-core-1.5.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/accumulo-fate-1.5.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/accumulo-server-1.5.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/accumulo-start-1.5.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/accumulo-trace-1.5.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/antlr-runtime-3.4.jar:/usr/local/pig/pig-0.15.0/bin/../lib/asm-3.3.1.jar:/usr/local/pig/pig-0.15.0/bin/../lib/automaton-1.11-8.jar:/usr/local/pig/pig-0.15.0/bin/../lib/avro-1.7.5.jar:/usr/local/pig/pig-0.15.0/bin/../lib/avro-tools-1.7.5-nodeps.jar:/usr/local/pig/pig-0.15.0/bin/../lib/groovy-all-1.8.6.jar:/usr/local/pig/pig-0.15.0/bin/../lib/guava-11.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/hive-common-0.14.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/hive-exec-0.14.0-core.jar:/usr/local/pig/pig-0.15.0/bin/../lib/hive-serde-0.14.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/hive-shims-common-0.14.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/hive-shims-common-secure-0.14.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/jackson-core-asl-1.8.8.jar:/usr/local/pig/pig-0.15.0/bin/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/pig/pig-0.15.0/bin/../lib/jansi-1.9.jar:/usr/local/pig/pig-0.15.0/bin/../lib/jline-1.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/joda-time-2.5.jar:/usr/local/pig/pig-0.15.0/bin/../lib/jruby-complete-1.6.7.jar:/usr/local/pig/pig-0.15.0/bin/../lib/js-1.7R2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/json-simple-1.1.jar:/usr/local/pig/pig-0.15.0/bin/../lib/jython-standalone-2.5.3.jar:/usr/local/pig/pig-0.15.0/bin/../lib/kryo-2.22.jar:/usr/local/pig/pig-0.15.0/bin/../lib/piggybank.jar:/usr/local/pig/pig-0.15.0/bin/../lib/protobuf-java-2.5.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/snappy-java-1.1.0.1.jar:/usr/local/pig/pig-0.15.0/bin/../lib/trevni-avro-1.7.5.jar:/usr/local/pig/pig-0.15.0/bin/../lib/trevni-core-1.7.5.jar:/usr/local/pig/pig-0.15.0/bin/../lib/zookeeper-3.4.5.jar:/usr/local/pig/pig-0.15.0/bin/../pig-0.15.0-core-h2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/avro-mapred-1.7.5-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/commons-collections4-4.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hbase-client-0.98.12-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hbase-common-0.98.12-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hbase-hadoop-compat-0.98.12-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hbase-hadoop2-compat-0.98.12-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hbase-protocol-0.98.12-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hbase-server-0.98.12-hadoop2.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/hive-shims-0.23-0.14.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-api-0.7.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-common-0.7.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-dag-0.7.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-mapreduce-0.7.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-runtime-internals-0.7.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-runtime-library-0.7.0.jar:/usr/local/pig/pig-0.15.0/bin/../lib/h2/tez-yarn-timeline-history-with-acls-0.7.0.jar:
HADOOP_OPTS:
HADOOP_CLIENT_OPTS: -Xmx1000m
-Dpig.log.dir=/usr/local/pig/pig-0.15.0/bin/../logs
-Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/pig/pig-0.15.0/bin/..
/usr/local/hadoop/bin/hadoop jar
/usr/local/pig/pig-0.15.0/bin/../pig-0.15.0-core-h2.jar
