Well, the problem with that is that I really want to work with my objects, rather than use Avros.generics, because then I'm forced to treat everything as a GenericData.Record. It's just a pain in the butt.
On Thu, Dec 6, 2012 at 5:17 PM, Josh Wills <[email protected]> wrote: > You don't want to create an AvroType yourself, you want to call > o.a.c.types.avro.Avros.records or one of its friends and pass it a Class > object. > > Interesting though, I would still want that case to work correctly. > > Josh > On Dec 6, 2012 5:14 PM, "Jonathan Natkins" <[email protected]> wrote: > >> So I've been futzing with Crunch a bit, and trying to understand how to >> build a pipeline that outputs Avro data files. Roughly, I'm doing something >> along these lines: >> >> Schema.Parser schemaParser = new Schema.Parser(); >> final Schema avroObjSchema = schemaParser.parse( >> schemaJsonString); >> >> AvroType avroType = new AvroType<MyAvroObject>(MyAvroObject.class, >> avroObjSchema, new >> AvroDeepCopier.AvroReflectDeepCopier<MyAvroObject>( >> MyAvroObject.class, avroObjSchema)); >> >> PCollection<MyAvroObject> words = logs.parallelDo(new DoFn<String, >> MyAvroObject>() { >> public void process(String line, Emitter<MyAvroObject> emitter) { >> emitter.emit(convertStringToAvroObj(line)); >> } >> }, avroType); >> >> However, this results in a class cast exception: >> >> Exception in thread "main" java.lang.ClassCastException: class >> com.company.MyAvroObject >> at java.lang.Class.asSubclass(Class.java:3039) >> at >> org.apache.crunch.types.writable.Writables.records(Writables.java:250) >> at >> org.apache.crunch.types.writable.WritableTypeFamily.records(WritableTypeFamily.java:86) >> at org.apache.crunch.types.PTypeUtils.convert(PTypeUtils.java:61) >> at org.apache.crunch.types.writable.WritableTypeFamily.as >> (WritableTypeFamily.java:135) >> at >> org.apache.crunch.impl.mr.MRPipeline.writeTextFile(MRPipeline.java:319) >> >> Anybody have any thoughts? There's got to be a magical incantation that I >> have slightly off. >> >
