Thanks Prajwal
I tried what you suggested but i still get the same error.
On Mon, Feb 1, 2016 at 2:05 PM, Prajwal Tuladhar <p...@infynyxx.com> wrote:
> Hi,
>
> I think your usage of default for field "agentType" is invalid here.
>
> When generating code from invalid schema, it tends to fail:
>
> [INFO]
>> [INFO] --- avro-maven-plugin:1.7.6-cdh5.4.4:schema (default) @ test-app
>> ---
>> [WARNING] Avro: Invalid default for field agentType: "APP_AGENT" not a
>> ["null","string"]
>
>
> Try:
>
> {
>> "namespace": "xx.xxxx.xxxxx.xxxxx",
>> "type": "record",
>> "name": "MyPayLoad",
>> "fields": [
>> {"name": "filed1", "type": "string"},
>> {"name": "filed2", "type": "long"},
>> {"name": "filed3", "type": "boolean"},
>> {
>> "name" : "metrics",
>> "type":
>> {
>> "type" : "array",
>> "items":
>> {
>> "name": "MyRecord",
>> "type": "record",
>> "fields" :
>> [
>> {"name": "min", "type": "long"},
>> {"name": "max", "type": "long"},
>> {"name": "sum", "type": "long"},
>> {"name": "count", "type": "long"}
>> ]
>> }
>> }
>> },
>> {"name": "agentType", "type": ["null", "string"], "default": null}
>> ]
>> }
>
>
>
>
>
> On Mon, Feb 1, 2016 at 8:31 PM, Raghvendra Singh <rsi...@appdynamics.com>
> wrote:
>
>>
>>
>> down votefavorite
>> <http://stackoverflow.com/questions/34733604/avro-schema-doesnt-honor-backward-compatibilty#>
>>
>> I have this avro schema
>>
>> {
>> "namespace": "xx.xxxx.xxxxx.xxxxx",
>> "type": "record",
>> "name": "MyPayLoad",
>> "fields": [
>> {"name": "filed1", "type": "string"},
>> {"name": "filed2", "type": "long"},
>> {"name": "filed3", "type": "boolean"},
>> {
>> "name" : "metrics",
>> "type":
>> {
>> "type" : "array",
>> "items":
>> {
>> "name": "MyRecord",
>> "type": "record",
>> "fields" :
>> [
>> {"name": "min", "type": "long"},
>> {"name": "max", "type": "long"},
>> {"name": "sum", "type": "long"},
>> {"name": "count", "type": "long"}
>> ]
>> }
>> }
>> }
>> ]}
>>
>> Here is the code which we use to parse the data
>>
>> public static final MyPayLoad parseBinaryPayload(byte[] payload) {
>> DatumReader<MyPayLoad> payloadReader = new
>> SpecificDatumReader<>(MyPayLoad.class);
>> Decoder decoder = DecoderFactory.get().binaryDecoder(payload, null);
>> MyPayLoad myPayLoad = null;
>> try {
>> myPayLoad = payloadReader.read(null, decoder);
>> } catch (IOException e) {
>> logger.log(Level.SEVERE, e.getMessage(), e);
>> }
>>
>> return myPayLoad;
>> }
>>
>> Now i want to add one more field int the schema so the schema looks like
>> below
>>
>> {
>> "namespace": "xx.xxxx.xxxxx.xxxxx",
>> "type": "record",
>> "name": "MyPayLoad",
>> "fields": [
>> {"name": "filed1", "type": "string"},
>> {"name": "filed2", "type": "long"},
>> {"name": "filed3", "type": "boolean"},
>> {
>> "name" : "metrics",
>> "type":
>> {
>> "type" : "array",
>> "items":
>> {
>> "name": "MyRecord",
>> "type": "record",
>> "fields" :
>> [
>> {"name": "min", "type": "long"},
>> {"name": "max", "type": "long"},
>> {"name": "sum", "type": "long"},
>> {"name": "count", "type": "long"}
>> ]
>> }
>> }
>> }
>> {"name": "agentType", "type": ["null", "string"], "default":
>> "APP_AGENT"}
>> ]}
>>
>> Note the filed added and also the default is defined. The problem is that
>> if we receive the data which was written using the older schema i get this
>> error
>>
>> java.io.EOFException: null
>> at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
>> ~[avro-1.7.4.jar:1.7.4]
>> at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
>> ~[avro-1.7.4.jar:1.7.4]
>> at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
>> ~[avro-1.7.4.jar:1.7.4]
>> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
>> ~[avro-1.7.4.jar:1.7.4]
>> at
>> com.appdynamics.blitz.shared.util.XXXXXXXXXXXXX.parseBinaryPayload(BlitzAvroSharedUtil.java:38)
>> ~[blitz-shared.jar:na]
>>
>> What i understood from this
>> <https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html>
>> document
>> that this should have been backward compatible but somehow that doesn't
>> seem to be the case. Any idea what i am doing wrong?
>>
>
>
>
> --
> --
> Cheers,
> Praj
>
