[ https://issues.apache.org/jira/browse/HIVE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159122#comment-13159122 ]
Krishna Kumar commented on HIVE-2600: ------------------------------------- I have chosen to record that information in the file itself. Each block has a header which contains the type/compression mechanism used for the decoder to process. The reader needs no extra information to process the file. This also allows us to dynamically choose the compression mechanism on a per-block basis in the future. > Enable/Add type-specific compression for rcfile > ----------------------------------------------- > > Key: HIVE-2600 > URL: https://issues.apache.org/jira/browse/HIVE-2600 > Project: Hive > Issue Type: Sub-task > Components: Query Processor, Serializers/Deserializers > Reporter: Krishna Kumar > Assignee: Krishna Kumar > Priority: Minor > Attachments: HIVE-2600.v0.patch, HIVE-2600.v1.patch > > > Enable schema-aware compression codecs which can perform type-specific > compression on a per-column basis. I see this as in three-parts > 1. Add interfaces for the rcfile to communicate column information to the > codec > 2. Add an "uber compressor" which can perform column-specific compression on > a per-block basis. Initially, this can be config driven, but we can go for a > dynamic implementation later. > 3. A bunch of type-specific compressors > This jira is for the first part of the effort. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira