[
https://issues.apache.org/jira/browse/AVRO-61?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting resolved AVRO-61.
------------------------------
Resolution: Fixed
Fix Version/s: 1.0.1
I just committed this. Thanks, Ravi!
> Support for blocking arrays/maps in python
> ------------------------------------------
>
> Key: AVRO-61
> URL: https://issues.apache.org/jira/browse/AVRO-61
> Project: Avro
> Issue Type: Bug
> Components: python
> Reporter: Thiruvalluvan M. G.
> Assignee: Ravi Gummadi
> Fix For: 1.0.1
>
> Attachments: AVRO-61.patch
>
>
> We have introduced a blocking version of writer in Java that enables readers
> to efficiently skip large arrays and maps. The avro format encoded arrays and
> maps by encoding the number of elements by the the elements themselves. Zero
> element count indicates that the array/map has ended. The change we
> introduced is that if the element count is negative, it is followed by the
> byte-count of the encoded elements that follow. The reader, on seeing a
> negative element count should flip the sign to get the actual number of
> elements. In addition, if it is interested in supporting fast skip, it should
> use the byte count to skip the elements en-bloc instead of decoding them
> individually. If it does not want to support fast skip, it has to just read
> the byte count and ignore its value.
> The changes are already made in Java's ValueReader to support this. Similar
> changes need to done in Python as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.