[ https://issues.apache.org/jira/browse/AVRO-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan Skraba resolved AVRO-2490. ------------------------------- Fix Version/s: (was: 1.8.2) (was: 1.9.0) Resolution: Not A Bug > Unable to add field to existing avro schema > ------------------------------------------- > > Key: AVRO-2490 > URL: https://issues.apache.org/jira/browse/AVRO-2490 > Project: Apache Avro > Issue Type: Bug > Reporter: Martin Mucha > Priority: Major > > As described in > [https://docs.oracle.com/database/nosql-12.1.3.4/GettingStartedGuide/schemaevolution.html#changeschema-rules] > > I should be able to extend AVRO schema by adding optional and(or?) > default-having field. If I do that though, deserialization fails for me. It > happens for version 1.8.2 and 1.9.0 as well. > > Some cases: > A) If I add new field defined as: > {code:java} > { "name": "newColumn", "type": ["null","string"], "default": null}{code} > deserialization will fail with: > > {code:java} > Caused by: java.lang.ArrayIndexOutOfBoundsException: 5 at > org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:424) at > org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290) at > org.apache.avro.io.parsing.Parser.advance(Parser.java:88) at > org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267) at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) > at > org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) > at > org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145) > at > tech.allegro.schema.json2avro.converter.JsonAvroConverter.convertToJson(JsonAvroConverter.java:83){code} > > if I add new field defined as: > {code:java} > { "name": "newColumn", "type": "string", "default": ""}{code} > {{or}} > {code:java} > { "name": "newColumn", "type": "string", "default": "incorrect"}{code} > (assuming I will fix it later in app) > deserialization will fail with: > > {code:java} > Caused by: org.apache.avro.AvroRuntimeException: Malformed data. Length is > negative: -1 at > org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:336) at > org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:263) at > org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) at > org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422) > at > org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:414) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:181) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) > at > org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) > at > org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) > at > org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145) > at > tech.allegro.schema.json2avro.converter.JsonAvroConverter.convertToJson(JsonAvroConverter.java:83){code} > > {{what code does fail like that(that ± content of method convertToJson in > JsonAvroConverter as seen in stacktrace.):}} > {code:java} > BinaryDecoder binaryDecoder = DecoderFactory.get().binaryDecoder(avro, > (BinaryDecoder)null); > GenericRecord record = (GenericRecord)(new > GenericDatumReader(schema)).read((Object)null, binaryDecoder);{code} > the second line produces given errors. Without new column added, > deserialization works. > -- This message was sent by Atlassian Jira (v8.3.4#803005)