I'd be cautious of using AvroStorage in it's current state with 1.6.0.
Running the piggybank unit tests against 1.6.0 causes compile failures, due
to non-backward compatible Avro changes in 1.6.0.
GenericDatumReader.newRecord(Object old, Schema schema) has gone away in
Avro 1.6.0.
[javac]
/Users/billg/ws/git/pig/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroDatumReader.java:136:
method does not override or implement a method from a supertype
[javac] @Override
[javac] ^
I've just created this FYI:
https://issues.apache.org/jira/browse/PIG-2463
On Mon, Jan 9, 2012 at 1:21 PM, Russell Jurney <russell.jur...@gmail.com>wrote:
> Avro 1.4.1 only works for me with PIG-2411 applied
>
> Russell Jurney
> twitter.com/rjurney
> russell.jur...@gmail.com
> datasyndrome.com
>
> On Jan 9, 2012, at 12:52 PM, Stan Rosenberg
> <srosenb...@proclivitysystems.com> wrote:
>
> > Generally, AvroStorage works fine for us with Avro 1.6. However, we
> > also patched AvroStorage on a couple of occasions, e.g., see PIG-2330.
> >
> > stan
> >
> > On Mon, Jan 9, 2012 at 3:47 PM, Russell Jurney <russell.jur...@gmail.com>
> wrote:
> >> I could only make AvroStorage work with Avro 1.4.1.
> >>
> >> Russell Jurney
> >> twitter.com/rjurney
> >> russell.jur...@gmail.com
> >> datasyndrome.com
> >>
> >> On Jan 9, 2012, at 1:16 AM, Andrew Kenworthy <adwkenwor...@yahoo.com>
> wrote:
> >>
> >> Hallo,
> >>
> >> When I run a simple pig script to LOAD and STORE avro data, I get:-
> >>
> >> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot
> be
> >> cast to org.apache.avro.generic.IndexedRecord
> >>
> >> Script:
> >>
> >> REGISTER /tmp/avro-1.6.0.jar;
> >> --REGISTER /tmp/avro-1.5.4.jar
> >> --REGISTER /tmp/avro-1.4.1.jar;
> >>
> >> REGISTER /tmp/piggybank-0.9.1.jar;
> >> REGISTER /tmp/json-simple-1.1.jar;
> >> REGISTER /tmp/jackson-core-asl-1.8.4.jar;
> >> REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;
> >>
> >> avroData=LOAD '$DATA_INPUTDIR' USING
> >> org.apache.pig.piggybank.storage.avro.AvroStorage();
> >>
> >> dataSubset = FOREACH avroData GENERATE myField1, myField2;
> >> describe dataSubset;
> >> -----------------------------------------------
> >> -- shows:
> >> -- dataSubset : { myField1: int, myField2: int}
> >> -----------------------------------------------
> >> STORE dataSubset INTO '$OUTPUTDIR' USING
> >> org.apache.pig.piggybank.storage.avro.AvroStorage();
> >>
> >> If I use the 1.5.4 jar I get the same error, but the script works with
> the
> >> 1.4.1 version. If I just write one field, then it works with 1.6.0.
> >>
> >> I see there's been a related issue fixed here:
> >>
> >> https://issues.apache.org/jira/browse/PIG-2202
> >> https://issues.apache.org/jira/browse/PIG-2195
> >>
> >> Can anyone confirm that this or similar works with avro 1.6.0, and/or
> point
> >> me in the right direction concering where the problem may lie?
> >>
> >> Many thanks,
> >>
> >> Andrew
>
