Re: setting PIG_INPUT_INITIAL_ADDRESS environment . variable in Oozie for cassandra ...¿?
¡Eureka! At last !!! the trick?? As well as putting the jar libraries dependecies in the sharelib folder in hdfs ... I had define this and the other environment variables in the bash_profile and works fine if I launch pig scripts from command line shell. I have to define these variables: PIG_INITIAL_ADDRESS, PIG_CONF_DIR, PIG_RPC_PORT, PIG_PARTITIONER . in the hadoop_env.sh and works fine. The others exceptions were like null exceptions in cassandra's class, etc Now, It is working with pig 0.10 ... Run pig script using PigRunner.run() for Pig version 0.8+ Apache Pig version 0.10.0 (r1328203) ... Although I configue to use pig 0.11 and put this pig jar version in the oozie shrelib, pom , classpath, etc and , I can not run pig script by shell actions in oozie, but this another song!!! Regards 2013/12/12 Aaron Morton > > Caused by: java.io.IOException: PIG_INPUT_INITIAL_ADDRESS or > PIG_INITIAL_ADDRESS environment variable not set > > at > org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:314) > > at > org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorage.java:358) > > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151) > > ... 35 more > > Have you checked these are set ? > > Cheers > > - > Aaron Morton > New Zealand > @aaronmorton > > Co-Founder & Principal Consultant > Apache Cassandra Consulting > http://www.thelastpickle.com > > On 11/12/2013, at 4:00 am, Miguel Angel Martin junquera < > mianmarjun.mailingl...@gmail.com> wrote: > > > Hi, > > > > I have an error with pig action in oozie 4.0.0 using cassandraStorage. > (cassandra 1.2.10) > > > > I can run pig scripts right with cassandra. but whe I try to use > cassandraStorage to load data I have this error: > > > > > > Run pig script using PigRunner.run() for Pig version 0.8+ > > Apache Pig version 0.10.0 (r1328203) > > compiled Apr 20 2012, 00:33:25 > > > > Run pig script using PigRunner.run() for Pig version 0.8+ > > 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 > > 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 > > 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging > error messages to: > /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log > > 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging > error messages to: > /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log > > 2013-12-10 12:24:39,501 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to hadoop file system at: hdfs://10.228.243.18:9000 > > 2013-12-10 12:24:39,501 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to hadoop file system at: hdfs://10.228.243.18:9000 > > 2013-12-10 12:24:39,510 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to map-reduce job tracker at: 10.228.243.18:9001 > > 2013-12-10 12:24:39,510 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to map-reduce job tracker at: 10.228.243.18:9001 > > 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 2245: > > Cannot get schema from > loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage > > 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 2245: > > Cannot get schema from > loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage > > 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245: > > Cannot get schema from > loadFunc org.apache.cassandra.hadoop.pig.CassandraStorage > > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:155) > > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110) > > at > org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68) > > at > org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60) > > at > org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84) > > at > org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77) > > at > org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) > > at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) > > at org.apache.pig.PigServer$Graph.compile(PigServer.java:1617) > > at org.apache.pig.PigServer$
Re: setting PIG_INPUT_INITIAL_ADDRESS environment . variable in Oozie for cassandra ...¿?
> Caused by: java.io.IOException: PIG_INPUT_INITIAL_ADDRESS or > PIG_INITIAL_ADDRESS environment variable not set > at > org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:314) > at > org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorage.java:358) > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151) > ... 35 more Have you checked these are set ? Cheers - Aaron Morton New Zealand @aaronmorton Co-Founder & Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 11/12/2013, at 4:00 am, Miguel Angel Martin junquera wrote: > Hi, > > I have an error with pig action in oozie 4.0.0 using cassandraStorage. > (cassandra 1.2.10) > > I can run pig scripts right with cassandra. but whe I try to use > cassandraStorage to load data I have this error: > > > Run pig script using PigRunner.run() for Pig version 0.8+ > Apache Pig version 0.10.0 (r1328203) > compiled Apr 20 2012, 00:33:25 > > Run pig script using PigRunner.run() for Pig version 0.8+ > 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 > 2013-12-10 12:24:39,084 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0 (r1328203) compiled Apr 20 2012, 00:33:25 > 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error > messages to: > /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log > 2013-12-10 12:24:39,095 [main] INFO org.apache.pig.Main - Logging error > messages to: > /tmp/hadoop-ec2-user/mapred/local/taskTracker/ec2-user/jobcache/job_201312100858_0007/attempt_201312100858_0007_m_00_0/work/pig-job_201312100858_0007.log > 2013-12-10 12:24:39,501 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to hadoop file system at: hdfs://10.228.243.18:9000 > 2013-12-10 12:24:39,501 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to hadoop file system at: hdfs://10.228.243.18:9000 > 2013-12-10 12:24:39,510 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to map-reduce job tracker at: 10.228.243.18:9001 > 2013-12-10 12:24:39,510 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to map-reduce job tracker at: 10.228.243.18:9001 > 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 2245: > Cannot get schema from loadFunc > org.apache.cassandra.hadoop.pig.CassandraStorage > 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 2245: > Cannot get schema from loadFunc > org.apache.cassandra.hadoop.pig.CassandraStorage > 2013-12-10 12:24:40,505 [main] ERROR org.apache.pig.tools.grunt.Grunt - > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245: > Cannot get schema from loadFunc > org.apache.cassandra.hadoop.pig.CassandraStorage > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:155) > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110) > at > org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68) > at > org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60) > at > org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84) > at > org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77) > at > org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) > at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) > at org.apache.pig.PigServer$Graph.compile(PigServer.java:1617) > at org.apache.pig.PigServer$Graph.compile(PigServer.java:1611) > at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1334) > at org.apache.pig.PigServer.execute(PigServer.java:1239) > at org.apache.pig.PigServer.executeBatch(PigServer.java:362) > at > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) > at org.apache.pig.Main.run(Main.java:430) > at org.apache.pig.PigRunner.run(PigRunner.java:49) > at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283) > at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223) > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37) > at org.apache.oozie.action.hadoop.Pi
