Is the jar that has com.example.thrift.VectorSequence both "register"ed and
on the pig classpath?
D
On Wed, Mar 16, 2011 at 3:23 AM, Torben Brodt <t.br...@gmail.com> wrote:
> Hey folks,
> i still try to setup elephant bird in pig. I am using the pig-08 branch of
> dvryaboy.
> i managed to create my example loader using the pig8.util.ThriftToPig
>
> my pig code looks like this..
> raw_data = load '/tmp/thrift/vi_base64.txt.lzo' using
>
> com.twitter.elephantbird.pig8.load.LzoThriftB64LinePigLoader('com.example.thrift.VectorSequence');
>
> When i run it, i get the following exception after the map/reduce phase:
>
> elephantbird.thrift.class.for.com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat
> is not set
>
> Seems like i miss a classpath again? But of course the elephantbird libs
> are
> included, otherwise the script would fail much earlier withe the
> LzoThriftB64LinePigLoader missing itself?
> May the data be corrupted?
>
> See the stack trace attached, i hope you have some idea.
>
> Best regards,
> Torben
>
> Backend error message
> ---------------------
> java.lang.RuntimeException: java.lang.RuntimeException:
>
> elephantbird.thrift.class.for.com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat
> is not set
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:236)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:118)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
> Caused by: java.lang.RuntimeException:
>
> elephantbird.thrift.class.for.com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat
> is not set
> at com.twitter.elephantbird.util.ThriftUtils.getTypeRef(Unknown Source)
> at
>
> com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat.createRecordReader(Unknown
> Source)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:227)
> ... 9 more
>
> Pig Stack Trace
> ---------------
> ERROR 2997: Unable to recreate exception from backed error:
> java.lang.RuntimeException: java.lang.RuntimeException:
>
> elephantbird.thrift.class.for.com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat
> is not set
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias raw_data. Backend error : Unable to recreate
> exception from backed error: java.lang.RuntimeException:
> java.lang.RuntimeException:
>
> elephantbird.thrift.class.for.com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat
> is not set
> at org.apache.pig.PigServer.openIterator(PigServer.java:742)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at
>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:406)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
> 2997:
> Unable to recreate exception from backed error: java.lang.RuntimeException:
> java.lang.RuntimeException:
>
> elephantbird.thrift.class.for.com.twitter.elephantbird.mapreduce.input.LzoThriftB64LineInputFormat
> is not set
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:221)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:151)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:337)
> at
>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1198)
> at org.apache.pig.PigServer.storeEx(PigServer.java:874)
> at org.apache.pig.PigServer.store(PigServer.java:816)
> at org.apache.pig.PigServer.openIterator(PigServer.java:728)
> ... 7 more
>
> ================================================================================
>
