[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13755019#comment-13755019
]
Ramana Suvarapu commented on AVRO-1360:
---------------------------------------
Hi Thiru,
I tested the AVRO-1360 and it fixed only "Don't know how to handle excess
fields for reader.β error and handling the default values. But resolving
decoder functionality is not working as per avro specification. Here are the
issues I noticed when testing.
1) If the field ordering of writer schema and reader schema is different,
resolving decoder is unable to decode and itβs throwing exception.
2) If one of the additional field of reader schema is optional field
[null, string], while decoding this optional field to null, resolving decoder
is throwing exceptions.
Please find the attached schemas and sample c++ program that I tried to
re-produce this issue. The same schema works fine in Java.
void testResolvingDecoder()
{
const char *reader = "C:\\temp\\testreader";
const char *writer = "C:\\temp\tempwriter;
avro::ValidSchema readerSchema = load(reader);
avro::ValidSchema writerSchema = load(writer);
std::auto_ptr<avro::OutputStream> out = avro::memoryOutputStream();
avro::EncoderPtr e = avro::binaryEncoder();
e->init(*out);
writer::MyRecord wr;
wr.kind = writer::Kind::FOO;
wr.name = "Foo";
wr.longType = 999;
wr.floatType = 10.25;
wr.doubleType = 20.23;
wr.extra = "extra";
wr.hash.empty();
wr.arrayTest.empty();
wr.mapTest.empty();
wr.unionTest1 = "union1";
wr.unionTest2.set_Kind(writer::Kind::BAZ);
avro::encode(*e, wr);
std::auto_ptr<avro::InputStream> in = avro::memoryInputStream(*out);
avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
readerSchema,avro::binaryDecoder());
d->init(*in);
reader::MyRecord rr;
avro::decode(*d, rr);
}
Thanks,
Ramana
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> ------------------------------------------------------------------------------------------
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
> Affects Versions: 1.7.4
> Reporter: Ramana Suvarapu
> Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360.patch, reader.json, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.β with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.β But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
