OLD:
{
namespace: com.hello.world,
type: record,
name: Toto,
fields: [
{
name: a,
type: [
string,
null
]
},
{
name: b,
type: string
}
]
}
NEW:
{
namespace: com.hello.world,
type: record,
name: Toto,
fields: [
{
name: a,
type: [
string,
null
]
},
{
name: b,
type: string
},
{
name: c,
type: string,
default: na
}
]
}
Sorry bad copy paste. The Avro Schema should be fine because I'm using
sbt-avro to generate the class from it.
On Wed, Apr 8, 2015 at 6:57 PM, Lukas Steiblys lu...@doubledutch.me wrote:
The schema is not valid JSON. Maybe you forgot the “[“ after “fields:”?
Lukas
*From:* Nicolas Phung nicolas.ph...@gmail.com
*Sent:* Wednesday, April 8, 2015 9:45 AM
*To:* user@avro.apache.org
*Subject:* Issue with reading old data with a new Avro Schema
Hello,
I'm trying to read old avro binary data with a new schema (I add a new
field).
This is the Avro Schema (OLD) I was using to write Avro binary data
before:
{
namespace: com.hello.world,
type: record,
name: Toto,
fields:
{
name: a,
type: [
string,
null
]
},
{
name: b,
type: string
}
]
}
This is the Avro Schema (NEW) I'm using to read the Avro binary data :
{
namespace: com.hello.world,
type: record,
name: Toto,
fields:
{
name: a,
type: [
string,
null
]
},
{
name: b,
type: string
},
{
name: c,
type: string,
default: na
}
]
}
However, I can't read the old data with the new Schema. I've got the
following errors :
15/04/08 17:32:22 ERROR executor.Executor: Exception in task 0.0 in stage
3.0 (TID 3)
java.io.EOFException
at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259)
at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:272)
at
org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:113)
at
org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:353)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157)
at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
at com.miguno.kafka.avro.AvroDecoder.fromBytes(AvroDecoder.scala:31)
From my understanding, I should be able to read the old data with the new
schema that contains a new field with a default value. But it doesn't seem
to work. Am I doing something wrong ?
I have posted a report https://issues.apache.org/jira/browse/AVRO-1661
Regards,
Nicolas PHUNG
