[ https://issues.apache.org/jira/browse/AVRO-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587525#comment-16587525 ]
ASF GitHub Bot commented on AVRO-1891: -------------------------------------- scatrin opened a new pull request #329: Improved conversions handling + pluggable conversions support [AVRO-1891, AVRO-2065] URL: https://github.com/apache/avro/pull/329 Fixes AVRO-1891, AVRO-2065 Adds support for pluggable conversion classes -- new feature Changes the compiler to add used conversions to the SpecificData instance already stored in the generated class. Adds static utility methods in SpecificData for getting the correct SpecificData instance from a class or a schema (using reflection). Adds calls to these utility methods from contexts where the schema is known. Also: Adds a possibility to add custom conversion classes to the compiler using a parameter to the Maven plugin. The custom conversion instances used are stored in the generated class with the same mechanism as above. They must of course be on the classpath at compile and serde time and whenever else the generated class is used. This feature is not required for the bug fixes to work but the addition is not large and IMO it adds nice value. Not implemented yet: * Custom conversions support needs to be added to the compiler tool. It is only available when using the Maven plugin ATM. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Generated Java code fails with union containing logical type > ------------------------------------------------------------ > > Key: AVRO-1891 > URL: https://issues.apache.org/jira/browse/AVRO-1891 > Project: Avro > Issue Type: Bug > Components: java, logical types > Affects Versions: 1.8.1 > Reporter: Ross Black > Priority: Blocker > Fix For: 1.8.3 > > Attachments: AVRO-1891.patch, AVRO-1891.yshi.1.patch, > AVRO-1891.yshi.2.patch, AVRO-1891.yshi.3.patch, AVRO-1891.yshi.4.patch > > > Example schema: > {code} > { > "type": "record", > "name": "RecordV1", > "namespace": "org.brasslock.event", > "fields": [ > { "name": "first", "type": ["null", {"type": "long", > "logicalType":"timestamp-millis"}]} > ] > } > {code} > The avro compiler generates a field using the relevant joda class: > {code} > public org.joda.time.DateTime first > {code} > Running the following code to perform encoding: > {code} > final RecordV1 record = new > RecordV1(DateTime.parse("2016-07-29T10:15:30.00Z")); > final DatumWriter<RecordV1> datumWriter = new > SpecificDatumWriter<>(record.getSchema()); > final ByteArrayOutputStream stream = new ByteArrayOutputStream(8192); > final BinaryEncoder encoder = > EncoderFactory.get().directBinaryEncoder(stream, null); > datumWriter.write(record, encoder); > encoder.flush(); > final byte[] bytes = stream.toByteArray(); > {code} > fails with the exception stacktrace: > {code} > org.apache.avro.AvroRuntimeException: Unknown datum type > org.joda.time.DateTime: 2016-07-29T10:15:30.000Z > at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741) > at > org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:293) > at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706) > at > org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192) > at > org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110) > at > org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87) > at > org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143) > at > org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105) > at > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) > at > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60) > at > org.brasslock.avro.compiler.GeneratedRecordTest.shouldEncodeLogicalTypeInUnion(GeneratedRecordTest.java:82) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117) > at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) > at > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:253) > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) > {code} > The failure can be fixed by explicitly adding the relevant conversion(s) to > DatumWriter / SpecificData: > {code} > final RecordV1 record = new > RecordV1(DateTime.parse("2007-12-03T10:15:30.00Z")); > final SpecificData specificData = new SpecificData(); > specificData.addLogicalTypeConversion(new > TimeConversions.TimestampConversion()); > final DatumWriter<RecordV1> datumWriter = new > SpecificDatumWriter<>(record.getSchema(), specificData); > final ByteArrayOutputStream stream = new > ByteArrayOutputStream(AvroUtil.DEFAULT_BUFFER_SIZE); > final BinaryEncoder encoder = > EncoderFactory.get().directBinaryEncoder(stream, null); > datumWriter.write(record, encoder); > encoder.flush(); > final byte[] bytes = stream.toByteArray(); > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)