I could only make AvroStorage work with Avro 1.4.1.
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
datasyndrome.com
On Jan 9, 2012, at 1:16 AM, Andrew Kenworthy <adwkenwor...@yahoo.com> wrote:
Hallo,
When I run a simple pig script to LOAD and STORE avro data, I get:-
java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be
cast to org.apache.avro.generic.IndexedRecord
Script:
REGISTER /tmp/avro-1.6.0.jar;
--REGISTER /tmp/avro-1.5.4.jar
--REGISTER /tmp/avro-1.4.1.jar;
REGISTER /tmp/piggybank-0.9.1.jar;
REGISTER /tmp/json-simple-1.1.jar;
REGISTER /tmp/jackson-core-asl-1.8.4.jar;
REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;
avroData=LOAD '$DATA_INPUTDIR' USING
org.apache.pig.piggybank.storage.avro.AvroStorage();
dataSubset = FOREACH avroData GENERATE myField1, myField2;
describe dataSubset;
-----------------------------------------------
-- shows:
-- dataSubset : { myField1: int, myField2: int}
-----------------------------------------------
STORE dataSubset INTO '$OUTPUTDIR' USING
org.apache.pig.piggybank.storage.avro.AvroStorage();
If I use the 1.5.4 jar I get the same error, but the script works with the
1.4.1 version. If I just write one field, then it works with 1.6.0.
I see there's been a related issue fixed here:
https://issues.apache.org/jira/browse/PIG-2202
https://issues.apache.org/jira/browse/PIG-2195
Can anyone confirm that this or similar works with avro 1.6.0, and/or point
me in the right direction concering where the problem may lie?
Many thanks,
Andrew
