[
https://issues.apache.org/jira/browse/AVRO-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267317#comment-13267317
]
alex gemini commented on AVRO-806:
----------------------------------
1.the file header should have some extra space in case we add some column or
append some value to block end.
2.trevni set different codec for each column,I think there is different between
compression,decompression and encoding,decoding:for example,run length encoding
will have high compression ratio by sort them first.we change the sequence of
data within a minor block(say 64k) but still guarantee the whole block(say hdfs
block 128M) will looks same as before(except that data sequence change).so,use
different Codec for each column maybe not get better performance or compression
ratio than a single codec for all column.I think different column need
different encodec (for example run length encodec which we didn't implement
yet).
> add a column-major codec for data files
> ---------------------------------------
>
> Key: AVRO-806
> URL: https://issues.apache.org/jira/browse/AVRO-806
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.7.0
>
> Attachments: AVRO-806-v2.patch, AVRO-806.patch, avro-file-columnar.pdf
>
>
> Define a codec that, when a data file's schema is a record schema, writes
> blocks within the file in column-major order. This would permit better
> compression and also permit efficient skipping of fields that are not of
> interest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
