Hi, I think your usage of default for field "agentType" is invalid here.
When generating code from invalid schema, it tends to fail: [INFO] > [INFO] --- avro-maven-plugin:1.7.6-cdh5.4.4:schema (default) @ test-app --- > [WARNING] Avro: Invalid default for field agentType: "APP_AGENT" not a > ["null","string"] Try: { > "namespace": "xx.xxxx.xxxxx.xxxxx", > "type": "record", > "name": "MyPayLoad", > "fields": [ > {"name": "filed1", "type": "string"}, > {"name": "filed2", "type": "long"}, > {"name": "filed3", "type": "boolean"}, > { > "name" : "metrics", > "type": > { > "type" : "array", > "items": > { > "name": "MyRecord", > "type": "record", > "fields" : > [ > {"name": "min", "type": "long"}, > {"name": "max", "type": "long"}, > {"name": "sum", "type": "long"}, > {"name": "count", "type": "long"} > ] > } > } > }, > {"name": "agentType", "type": ["null", "string"], "default": null} > ] > } On Mon, Feb 1, 2016 at 8:31 PM, Raghvendra Singh <rsi...@appdynamics.com> wrote: > > > down votefavorite > <http://stackoverflow.com/questions/34733604/avro-schema-doesnt-honor-backward-compatibilty#> > > I have this avro schema > > { > "namespace": "xx.xxxx.xxxxx.xxxxx", > "type": "record", > "name": "MyPayLoad", > "fields": [ > {"name": "filed1", "type": "string"}, > {"name": "filed2", "type": "long"}, > {"name": "filed3", "type": "boolean"}, > { > "name" : "metrics", > "type": > { > "type" : "array", > "items": > { > "name": "MyRecord", > "type": "record", > "fields" : > [ > {"name": "min", "type": "long"}, > {"name": "max", "type": "long"}, > {"name": "sum", "type": "long"}, > {"name": "count", "type": "long"} > ] > } > } > } > ]} > > Here is the code which we use to parse the data > > public static final MyPayLoad parseBinaryPayload(byte[] payload) { > DatumReader<MyPayLoad> payloadReader = new > SpecificDatumReader<>(MyPayLoad.class); > Decoder decoder = DecoderFactory.get().binaryDecoder(payload, null); > MyPayLoad myPayLoad = null; > try { > myPayLoad = payloadReader.read(null, decoder); > } catch (IOException e) { > logger.log(Level.SEVERE, e.getMessage(), e); > } > > return myPayLoad; > } > > Now i want to add one more field int the schema so the schema looks like > below > > { > "namespace": "xx.xxxx.xxxxx.xxxxx", > "type": "record", > "name": "MyPayLoad", > "fields": [ > {"name": "filed1", "type": "string"}, > {"name": "filed2", "type": "long"}, > {"name": "filed3", "type": "boolean"}, > { > "name" : "metrics", > "type": > { > "type" : "array", > "items": > { > "name": "MyRecord", > "type": "record", > "fields" : > [ > {"name": "min", "type": "long"}, > {"name": "max", "type": "long"}, > {"name": "sum", "type": "long"}, > {"name": "count", "type": "long"} > ] > } > } > } > {"name": "agentType", "type": ["null", "string"], "default": > "APP_AGENT"} > ]} > > Note the filed added and also the default is defined. The problem is that > if we receive the data which was written using the older schema i get this > error > > java.io.EOFException: null > at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) > ~[avro-1.7.4.jar:1.7.4] > at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) > ~[avro-1.7.4.jar:1.7.4] > at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423) > ~[avro-1.7.4.jar:1.7.4] > at > org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229) > ~[avro-1.7.4.jar:1.7.4] > at org.apache.avro.io.parsing.Parser.advance(Parser.java:88) > ~[avro-1.7.4.jar:1.7.4] > at > org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206) > ~[avro-1.7.4.jar:1.7.4] > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152) > ~[avro-1.7.4.jar:1.7.4] > at > org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177) > ~[avro-1.7.4.jar:1.7.4] > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148) > ~[avro-1.7.4.jar:1.7.4] > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139) > ~[avro-1.7.4.jar:1.7.4] > at > com.appdynamics.blitz.shared.util.XXXXXXXXXXXXX.parseBinaryPayload(BlitzAvroSharedUtil.java:38) > ~[blitz-shared.jar:na] > > What i understood from this > <https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html> > document > that this should have been backward compatible but somehow that doesn't > seem to be the case. Any idea what i am doing wrong? > -- -- Cheers, Praj