Ok, I'm still a little confused. Let's say I use Avros.generics(), and then I modify my code to use GenericData.Records. Those Records still don't implement the Writable interface, so I'm still getting a class cast exception. Did I do something totally wrong?
On Thu, Dec 6, 2012 at 5:19 PM, Jonathan Natkins <[email protected]> wrote: > Well, the problem with that is that I really want to work with my objects, > rather than use Avros.generics, because then I'm forced to treat everything > as a GenericData.Record. It's just a pain in the butt. > > > On Thu, Dec 6, 2012 at 5:17 PM, Josh Wills <[email protected]> wrote: > >> You don't want to create an AvroType yourself, you want to call >> o.a.c.types.avro.Avros.records or one of its friends and pass it a Class >> object. >> >> Interesting though, I would still want that case to work correctly. >> >> Josh >> On Dec 6, 2012 5:14 PM, "Jonathan Natkins" <[email protected]> wrote: >> >>> So I've been futzing with Crunch a bit, and trying to understand how to >>> build a pipeline that outputs Avro data files. Roughly, I'm doing something >>> along these lines: >>> >>> Schema.Parser schemaParser = new Schema.Parser(); >>> final Schema avroObjSchema = schemaParser.parse( >>> schemaJsonString); >>> >>> AvroType avroType = new AvroType<MyAvroObject>(MyAvroObject.class, >>> avroObjSchema, new >>> AvroDeepCopier.AvroReflectDeepCopier<MyAvroObject>( >>> MyAvroObject.class, avroObjSchema)); >>> >>> PCollection<MyAvroObject> words = logs.parallelDo(new DoFn<String, >>> MyAvroObject>() { >>> public void process(String line, Emitter<MyAvroObject> emitter) { >>> emitter.emit(convertStringToAvroObj(line)); >>> } >>> }, avroType); >>> >>> However, this results in a class cast exception: >>> >>> Exception in thread "main" java.lang.ClassCastException: class >>> com.company.MyAvroObject >>> at java.lang.Class.asSubclass(Class.java:3039) >>> at >>> org.apache.crunch.types.writable.Writables.records(Writables.java:250) >>> at >>> org.apache.crunch.types.writable.WritableTypeFamily.records(WritableTypeFamily.java:86) >>> at org.apache.crunch.types.PTypeUtils.convert(PTypeUtils.java:61) >>> at org.apache.crunch.types.writable.WritableTypeFamily.as >>> (WritableTypeFamily.java:135) >>> at >>> org.apache.crunch.impl.mr.MRPipeline.writeTextFile(MRPipeline.java:319) >>> >>> Anybody have any thoughts? There's got to be a magical incantation that >>> I have slightly off. >>> >> >
