[
https://issues.apache.org/jira/browse/AVRO-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733196#action_12733196
]
Thiruvalluvan M. G. commented on AVRO-50:
-----------------------------------------
Raymie and I have reimplemented the JSON encoder and decoder using the parser
instead of direct implementation of Encoder and Decoder. This enabled us to
implement JSON encoder and decoder with minimal additional support from the
clients of Encoder and Decoder. The additional information required such as
record boundaries etc. are derived from the parser's state. The only change
I've made to Encoder and Decoder is to distingush between map key and string.
This distinction makes JSON implementation easy. Consequently, some minor
changes were required in the clients of Encoder and Decoder. The only other
change was to move reset() as part of Encoder. This must have been part of the
interface when we separated Encoder from its implementation BinaryEncoder.
When trying to implement this, we realized that the original implementation of
the parser was too inflexible and unreadable. We have completely reimplemented
the parser generator. The new version is slightly faster than the original one.
As part of this refactoring, we have moved all those classes that are not
required by users of avro.io package into a subpackage called parsing under it.
When trying to validate various uses of the Decoder and Encoder interfaces, I
discovered there were some wrong uses. These errors do not have any impact with
the binary format but will have trouble with JSON format. for example, at one
place, writeInt() was instead of writeEnum(). All such cases have been fixed.
The JSON decoder streams contents from its input, so it expects the fields of
records in the order of their definition in the schema. There is no problem in
consuming the data generated by JSON encoder. But if the such data is passed
through some other JSON tool, the ordering of field may get changed and hence
may not get accepted by JSON decoder.
One additional functionality in this patch that was not present earlier is that
the ResolvingDecoder can supply values for the fields for whcih the reader's
schema has defined default values and the writer's schema has omitted. This
feature, naturally, uses JSON decoder. Consequently, the default values for
records should be defined the correct field order.
> bidrectional text representation of AVRO data
> ---------------------------------------------
>
> Key: AVRO-50
> URL: https://issues.apache.org/jira/browse/AVRO-50
> Project: Avro
> Issue Type: New Feature
> Components: spec
> Reporter: Philip Zeyliger
> Assignee: Doug Cutting
> Attachments: AVRO-50.patch, AVRO-50.patch, AVRO-50.patch, AVRO-50.sh
>
>
> It would be very useful to add a text representation of AVRO data to the
> spec, and implement toString() and fromString() in all implementations.
> Faced with binary data, it'll be a useful operation to decode it for
> debugging, ad-hoc manipulation, etc.
> I suspect the text format will:
> * be JSON
> * require the schema for full interpretation
> * map easily onto the binary format (if the binary format has a signifier to
> take a specific branch of a union, the text format will have such a signifier
> as well)
> * not be unique (there's more than one way to encode a given number (e.g.,
> {{0x0 == 0}}) or string (e.g., {{"\u0061" == "a"}}, not to mention flexible
> whitespace)
> * be compatible, for the binary type, with whatever is decided in AVRO-36
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.