Re: setting PIG_INPUT_INITIAL_ADDRESS environment . variable in Oozie for cassandra ...¿?
¡Eureka! At last !!! the trick?? As well as putting the jar libraries dependecies in the sharelib folder in hdfs ... I had define this and the other environment variables in the bash_profile and works fine if I launch pig scripts from command line shell. I have to define these variables: PIG_INITIAL_ADDRESS, PIG_CONF_DIR, PIG_RPC_PORT, PIG_PARTITIONER . in the hadoop_env.sh and works fine. The others exceptions were like null exceptions in cassandra's class, etc Now, It is working with pig 0.10 ... Run pig script using PigRunner.run() for Pig version 0.8+ Apache Pig version 0.10.0 (r1328203) ... Although I configue to use pig 0.11 and put this pig jar version in the oozie shrelib, pom , classpath, etc and , I can not run pig script by shell actions in oozie, but this another song!!! Regards 2013/12/12 Aaron Morton aa...@thelastpickle.com Caused by: java.io.IOException: PIG_INPUT_INITIAL_ADDRESS or PIG_INITIAL_ADDRESS environment variable not set at org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:314) at org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorage.java:358) at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151) ... 35 more Have you checked these are set ? Cheers - Aaron Morton New Zealand @aaronmorton Co-Founder Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 11/12/2013, at 4:00 am, Miguel Angel Martin junquera mianmarjun.mailingl...@gmail.com wrote: Hi, I have an error with pig action in oozie 4.0.0 using cassandraStorage. (cassandra 1.2.10) I can run pig scripts right with cassandra. but whe I try to use cassandraStorage to load data I have this error: Run pig script using PigRunner.run() for Pig version 0.8+ Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 Run pig script using PigRunner.run() for Pig version 0.8+ 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log 2013-12-10 12:24:39,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://10.228.243.18:9000 2013-12-10 12:24:39,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://10.228.243.18:9000 2013-12-10 12:24:39,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: 10.228.243.18:9001 2013-12-10 12:24:39,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: 10.228.243.18:9001 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245: file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:155) at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110) at org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68) at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60) at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77) at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) at org.apache.pig.PigServer$Graph.compile(PigServer.java:1617) at
Re: setting PIG_INPUT_INITIAL_ADDRESS environment . variable in Oozie for cassandra ...¿?
Caused by: java.io.IOException: PIG_INPUT_INITIAL_ADDRESS or PIG_INITIAL_ADDRESS environment variable not set at org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:314) at org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorage.java:358) at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151) ... 35 more Have you checked these are set ? Cheers - Aaron Morton New Zealand @aaronmorton Co-Founder Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 11/12/2013, at 4:00 am, Miguel Angel Martin junquera mianmarjun.mailingl...@gmail.com wrote: Hi, I have an error with pig action in oozie 4.0.0 using cassandraStorage. (cassandra 1.2.10) I can run pig scripts right with cassandra. but whe I try to use cassandraStorage to load data I have this error: Run pig script using PigRunner.run() for Pig version 0.8+ Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 Run pig script using PigRunner.run() for Pig version 0.8+ 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log 2013-12-10 12:24:39,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://10.228.243.18:9000 2013-12-10 12:24:39,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://10.228.243.18:9000 2013-12-10 12:24:39,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: 10.228.243.18:9001 2013-12-10 12:24:39,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: 10.228.243.18:9001 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245: file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:155) at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110) at org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68) at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60) at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77) at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) at org.apache.pig.PigServer$Graph.compile(PigServer.java:1617) at org.apache.pig.PigServer$Graph.compile(PigServer.java:1611) at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1334) at org.apache.pig.PigServer.execute(PigServer.java:1239) at org.apache.pig.PigServer.executeBatch(PigServer.java:362) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) at org.apache.pig.Main.run(Main.java:430) at org.apache.pig.PigRunner.run(PigRunner.java:49) at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283) at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223) at
setting PIG_INPUT_INITIAL_ADDRESS environment . variable in Oozie for cassandra ...¿?
Hi, I have an error with pig action in oozie 4.0.0 using cassandraStorage. (cassandra 1.2.10) I can run pig scripts right with cassandra. but whe I try to use cassandraStorage to load data I have this error: *Run pig script using PigRunner.run() for Pig version 0.8+* *Apache Pig version 0.10.0 (r1328203) * *compiled Apr 20 2012, 00:33:25* *Run pig script using PigRunner.run() for Pig version 0.8+* *2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25* *2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25* *2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log* *2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error messages to: /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log* *2013-12-10 12:24:39,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://10.228.243.18:9000 http://10.228.243.18:9000* *2013-12-10 12:24:39,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://10.228.243.18:9000 http://10.228.243.18:9000* *2013-12-10 12:24:39,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: 10.228.243.18:9001 http://10.228.243.18:9001* *2013-12-10 12:24:39,510 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: 10.228.243.18:9001 http://10.228.243.18:9001* *2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: * *file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage* *2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: * *file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage* *2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245: * *file testCassandra.pig, line 7, column 7 Cannot get schema from loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage* * at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:155)* * at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)* * at org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68)* * at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60)* * at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84)* * at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)* * at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)* * at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)* * at org.apache.pig.PigServer$Graph.compile(PigServer.java:1617)* * at org.apache.pig.PigServer$Graph.compile(PigServer.java:1611)* * at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1334)* * at org.apache.pig.PigServer.execute(PigServer.java:1239)* * at org.apache.pig.PigServer.executeBatch(PigServer.java:362)* * at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)* * at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)* * at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)* * at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)* * at org.apache.pig.Main.run(Main.java:430)* * at org.apache.pig.PigRunner.run(PigRunner.java:49)* * at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)* * at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)* * at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)* * at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)* * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* * at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)* * at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* * at java.lang.reflect.Method.invoke(Method.java:601)* * at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)* * at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)*
