[
https://issues.apache.org/jira/browse/HIVE-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12704765#action_12704765
]
Zheng Shao commented on HIVE-352:
---------------------------------
>>>>Writer: how do you pass the column number from Hive to the configuration
>>>>and then to the RCFIle.Writer?
>>The code is in RCFileOutputFormat's getHiveRecordWriter(). It tries to parse
>>the columns from passed in Properties.
Thanks. I understand it now.
>>>>init(...): Cleaning out the object and recreate LazyObject is not efficient.
>>If we change it, it will not pass the TestRCFile test. The final extra else
>>if statements are rarely reached, and when reached, most time it only needs
>>one instruction to determine whether fields[fieldIndex] is null.
Can you add a boolean[] fieldIsNull to mark whether a field is null, instead of
throwing away and recreating the LazyObject?
Then getField can check fieldIsNull to decide whether to return null or the
LazyObject.
> Make Hive support column based storage
> --------------------------------------
>
> Key: HIVE-352
> URL: https://issues.apache.org/jira/browse/HIVE-352
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: 4-22 performace2.txt, 4-22 performance.txt, 4-22
> progress.txt, hive-352-2009-4-15.patch, hive-352-2009-4-16.patch,
> hive-352-2009-4-17.patch, hive-352-2009-4-19.patch,
> hive-352-2009-4-22-2.patch, hive-352-2009-4-22.patch,
> hive-352-2009-4-23.patch, hive-352-2009-4-27.patch,
> hive-352-2009-4-30-2.patch, hive-352-2009-4-30-3.patch,
> hive-352-2009-4-30-4.patch, hive-352-2009-5-1.patch,
> HIve-352-draft-2009-03-28.patch, Hive-352-draft-2009-03-30.patch
>
>
> column based storage has been proven a better storage layout for OLAP.
> Hive does a great job on raw row oriented storage. In this issue, we will
> enhance hive to support column based storage.
> Acctually we have done some work on column based storage on top of hdfs, i
> think it will need some review and refactoring to port it to Hive.
> Any thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
