[
https://issues.apache.org/jira/browse/AVRO-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806039#action_12806039
]
Doug Cutting commented on AVRO-380:
-----------------------------------
I would prefer we use longs instead of ints for both the count and the block
size to better future-proof the format. 2GB isn't as big as it used to be and
it's only getting smaller. I don't think there are significant performance
benefits to using int over long here.
Other than that, +1.
> Avro Container File format change: add block size to block descriptor
> ----------------------------------------------------------------------
>
> Key: AVRO-380
> URL: https://issues.apache.org/jira/browse/AVRO-380
> Project: Avro
> Issue Type: Improvement
> Components: doc, java, spec
> Affects Versions: 1.3.0
> Reporter: Scott Carey
> Fix For: 1.3.0
>
> Attachments: AVRO-380.patch
>
>
> The new file format in AVRO-160 limits a few use cases that I have found to
> be important.
> A block currently contains a count of the number of records, the block data,
> and a sync marker.
> This change would add the block size, in bytes, along side the number of
> records.
> This allows efficient access to a block's data without the need to decode the
> data into individual Datums, which is useful for various use cases.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.