thanks! On Tue, Aug 2, 2016 at 12:28 AM, Ross Black <ross.w.bl...@gmail.com> wrote: > I raised https://issues.apache.org/jira/browse/AVRO-1891. > > Thanks. > Ross > > On 2 August 2016 at 01:46, Sean Busbey <bus...@cloudera.com> wrote: >> >> That's definitely looks like a bug. Would you mind filing a JIRA for this >> issue? >> >> On Sun, Jul 31, 2016 at 8:34 PM, Ross Black <ross.w.bl...@gmail.com> >> wrote: >> > Hi, >> > >> > I just encountered an issue that I suspect is a bug when using a logical >> > type within a union. >> > I am using Avro 1.8.1 with JDK8. >> > >> > >> > My schema is: >> > { >> > "type": "record", >> > "name": "RecordV1", >> > "namespace": "org.brasslock.event", >> > "fields": [ >> > { "name": "first", "type": ["null", {"type": "long", >> > "logicalType":"timestamp-millis"}]} >> > ] >> > } >> > >> > The avro compiler generates a field using the relevant joda class: >> > public org.joda.time.DateTime first >> > >> > >> > Running the following code to perform encoding: >> > final RecordV1 record = new >> > RecordV1(DateTime.parse("2016-07-29T10:15:30.00Z")); >> > final DatumWriter<RecordV1> datumWriter = new >> > SpecificDatumWriter<>(record.getSchema()); >> > final ByteArrayOutputStream stream = new >> > ByteArrayOutputStream(8192); >> > final BinaryEncoder encoder = >> > EncoderFactory.get().directBinaryEncoder(stream, null); >> > datumWriter.write(record, encoder); >> > encoder.flush(); >> > final byte[] bytes = stream.toByteArray(); >> > >> > fails with the exception stacktrace: >> > org.apache.avro.AvroRuntimeException: Unknown datum type >> > org.joda.time.DateTime: 2016-07-29T10:15:30.000Z >> > >> > at >> > org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741) >> > at >> > >> > org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:293) >> > at >> > org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706) >> > at >> > >> > org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192) >> > at >> > >> > org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110) >> > at >> > >> > org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87) >> > at >> > >> > org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143) >> > at >> > >> > org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105) >> > at >> > >> > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) >> > at >> > >> > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60) >> > at >> > >> > org.brasslock.avro.compiler.GeneratedRecordTest.shouldEncodeLogicalTypeInUnion(GeneratedRecordTest.java:82) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> > at >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > at >> > >> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) >> > at >> > >> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) >> > at >> > >> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) >> > at >> > >> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) >> > at >> > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) >> > at >> > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) >> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) >> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) >> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) >> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) >> > at >> > >> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117) >> > at >> > >> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) >> > at >> > >> > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:253) >> > at >> > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> > at >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > at >> > com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) >> > >> > >> > >> > The failure can be fixed by explicitly adding the relevant conversion(s) >> > to >> > DatumWriter / SpecificData: >> > final RecordV1 record = new >> > RecordV1(DateTime.parse("2007-12-03T10:15:30.00Z")); >> > final SpecificData specificData = new SpecificData(); >> > specificData.addLogicalTypeConversion(new >> > TimeConversions.TimestampConversion()); >> > final DatumWriter<RecordV1> datumWriter = new >> > SpecificDatumWriter<>(record.getSchema(), specificData); >> > final ByteArrayOutputStream stream = new >> > ByteArrayOutputStream(AvroUtil.DEFAULT_BUFFER_SIZE); >> > final BinaryEncoder encoder = >> > EncoderFactory.get().directBinaryEncoder(stream, null); >> > datumWriter.write(record, encoder); >> > encoder.flush(); >> > final byte[] bytes = stream.toByteArray(); >> > >> > >> > Is this expected behaviour? To me it is not particularly obvious that >> > generated code does not work with a standard datum writer. >> > >> > >> > Thanks, >> > Ross >> > >> >> >> >> -- >> busbey > >
-- busbey
