[
https://issues.apache.org/jira/browse/AVRO-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768766#action_12768766
]
Philip Zeyliger commented on AVRO-160:
--------------------------------------
bq. Note that the only change permitted to a schema as a file is written is to,
if it is a union, to add new branches at the end of that union. If it is not a
union, no changes may be made. So it is still the case that the final schema in
a file can read every entry in the file and thus may be used to randomly access
the file.
For some reason, I thought you could go from
{ "type": "record", "name": "foo", "namespace": "foo", "fields": [{ "name":
"x", "type": "string" }] }
to
{ "type": "record", "name": "foo", "namespace": "foo", "fields": [{ "name":
"x", "type": "string" }, {"name": "y", "type": ["null", "string"] ] }
(That is, adding an optional extra field within some record, not just modifying
the top-level.)
Is that not the case?
> file format should be friendly to streaming
> -------------------------------------------
>
> Key: AVRO-160
> URL: https://issues.apache.org/jira/browse/AVRO-160
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
>
> It should be possible to stream through an Avro data file without seeking to
> the end.
> Currently the interpretation is that schemas written to the file apply to all
> entries before them. If this were changed so that they instead apply to all
> entries that follow, and the initial schema is written at the start of the
> file, then streaming could be supported.
> Note that the only change permitted to a schema as a file is written is to,
> if it is a union, to add new branches at the end of that union. If it is not
> a union, no changes may be made. So it is still the case that the final
> schema in a file can read every entry in the file and thus may be used to
> randomly access the file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
