[
https://issues.apache.org/jira/browse/HIVE-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13745136#comment-13745136
]
Francis Liu commented on HIVE-4331:
-----------------------------------
Ashutosh, I don't think what you're suggesting will work. Since HiveOFImpl is
merely a container for initializing real OFs that are contained in defined
FileSinkOps. Looking at HIVE-2764, HiveOFImpl was introduced to guarantee
checkOutputSpecs was called on each real OF so that OFs such as HBase has a
chance of adding a delegation token. It seems the HiveOFImpl name is misleading
since it doesn't extend HiveOF and is not used in the same way. Let me know if
I missed anything.
The requirement that a StorageHandler's OF has to extend HiveOF pervasive all
over the hive code. ie A lot of classes have HiveOutputFormat (tableDesc,
FileSinkOP, etc) as a member variable instead of OF. So it's not possible to
just change this expectation at the "end" point of the code path. With storage
handlers I don't see why we still need HiveOF. We should migrate all supported
OFs to use storage handlers and remove HiveOF. HivePassThru is an step in that
direction. StorageHandlers will no longer have to extend HiveOF and HiveOF does
not have to leak into HCat code to get interoperability working.
> Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
> ------------------------------------------------------------------------
>
> Key: HIVE-4331
> URL: https://issues.apache.org/jira/browse/HIVE-4331
> Project: Hive
> Issue Type: Task
> Components: HCatalog
> Affects Versions: 0.11.0, 0.12.0
> Reporter: Ashutosh Chauhan
> Assignee: Viraj Bhat
> Attachments: HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf
>
>
> 1) Deprecate the HCatHBaseStorageHandler and "RevisionManager" from HCatalog.
> These will now continue to function but internally they will use the
> "DefaultStorageHandler" from Hive. They will be removed in future release of
> Hive.
> 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will
> bypass the HiveOutputFormat. We will use this class in Hive's
> "HBaseStorageHandler" instead of the "HiveHBaseTableOutputFormat".
> 3) Write new unit tests in the HCat's "storagehandler" so that systems such
> as Pig and Map Reduce can use the Hive's "HBaseStorageHandler" instead of the
> "HCatHBaseStorageHandler".
> 4) Make sure all the old and new unit tests pass without backward
> compatibility (except known issues as described in the Design Document).
> 5) Replace all instances of the HCat source code, which point to
> "HCatStorageHandler" to use the"HiveStorageHandler" including the
> "FosterStorageHandler".
> I have attached the design document for the same and will attach a patch to
> this Jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
