Hi Fokko, Thanks for the response. I have been trying other things before I respond.
This is a not well documented and obscure feature of the resolution. It does work in 1.8.2 and also in 1.9.0 (there was an issue with my schema). This is a very specific feature, which doesn't work with primitives The specific case logically makes sense when there is a hierarchical types in the branches, where in I can define a base type and I return that as default. For example for Writer schema I have aa addtional "string" branch in my union > { > "type" : "record", > "name" : "TestSample11", > "namespace" : "com.kvajjala.avro.test.samples", > "fields" : [ { > "name" : "id", > "type" : "long" > }, { > "name" : "name", > "type" : "string" > }, { > "name" : "unionField", > "type" : [ "null", "long", "string" ], > "default" : null > } ] > } > The Reader schema is > { > "type" : "record", > "name" : "TestSample11", > "namespace" : "com.kvajjala.avro.test.samples", > "fields" : [ { > "name" : "id", > "type" : "long" > }, { > "name" : "name", > "type" : "string" > }, { > "name" : "unionField", > "type" : [ "null", "long" ], > "default" : null > } ] > } In my Writer if I set the string value then the reader throws an exception(In both and 1.8.2 and 1.9.0 versions). org.apache.avro.AvroTypeException: Found string, expecting union > at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:308) > at org.apache.avro.io.parsing.Parser.advance(Parser.java:86) > at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:275) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:178) > at > org.apache.avro.specific.SpecificDatumReader.readField(SpecificDatumReader.java:136) > at > org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:237) > at > org.apache.avro.specific.SpecificDatumReader.readRecord(SpecificDatumReader.java:123) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:170) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144) Why can't the union resolution return a null if it cannot find an appropriate or a branch of the same type. I think for a union which is bound to evolve, a null as default (the first branch in the union) should be an acceptable solution which will also be consistent with primitives. Appreciate response Thanks Kishore On Mon, Jul 22, 2019 at 11:13 PM Driesprong, Fokko <fo...@driesprong.frl> wrote: > Hi Kishore, > > The easiest way to find out is compile it against Avro 1.8.2. This is a > regression bug which will be fixed in 1.9.1 which will be released in the > upcoming weeks. > > Cheers, Fokko > > Op ma 22 jul. 2019 om 22:38 schreef KV 59 <kvajjal...@gmail.com> > >> Hi, >> >> I'm trying to define a union schema which is backward compatible. >> >> This is similar to the example in >> >> >> http://apache-avro.679487.n3.nabble.com/Avro-union-compatibility-mode-enhancement-proposal-td4034377.html >> >> I have the original schema as below >> >> >> record BaseOrg { >>> long orgId; >>> } >>> record Org { >>> long orgId; >>> string name; >>> } >>> record Address { >>> string city; >>> string state; >>> } >>> record ExtendedOrg { >>> long orgId; >>> string name; >>> string industry; >>> } >> >> >> union { >>> null, >>> BaseOrg, >>> Org, >>> ExtendedOrg >>> } org=null; >> >> >> This is just some example code (I could have added some of the fields to >> the original org ) >> >> Now I have evolved this schema to >> >>> >>> union { >>> null, >>> BaseOrg, >>> Org, >>> ExtendedOrg >>> ,ExtendedOrg2 >>> } org=null; >>> record ExtendedOrg2 { >>> long orgId; >>> string name; >>> boolean active; >>> string geography; >>> string industry; >>> } >> >> >> I have a consumer with the old schema and a producer with the new schema >> >> I saw in the JIRA >> >> https://issues.apache.org/jira/browse/AVRO-1590?focusedCommentId=14150780&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14150780 >> >> considering this I would imagine when a producer sets the "ExtendedOrg2" >> branch in the producer, The consumer who doesn't understand that should >> match to either of BaseOr, Org, ExtendedOrg. But I get the below exception >> >> java.lang.ArrayIndexOutOfBoundsException: 4 >>> at >>> org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:460) >>> at >>> org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:283) >>> at >>> org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:178) >>> at >>> org.apache.avro.specific.SpecificDatumReader.readField(SpecificDatumReader.java:136) >>> at >>> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:237) >>> at >>> org.apache.avro.specific.SpecificDatumReader.readRecord(SpecificDatumReader.java:123) >>> at >>> org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:170) >>> at >>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) >>> at >>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144) >>> at com.five9.dataservices.avro.AgentReader.toEvent(AgentReader.java:101) >>> at com.five9.dataservices.avro.AgentReader.read(AgentReader.java:83) >>> at com.five9.dataservices.avro.AgentReader.main(AgentReader.java:59) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282) >>> at java.lang.Thread.run(Thread.java:748) >> >> >> I would like to know if I understand the JIRA comments wrong or if >> something's wrong with my schema >> >> Thanks >> Kishore >> >>