[
https://issues.apache.org/jira/browse/HIVE-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528235#comment-13528235
]
Alan Gates commented on HIVE-3752:
----------------------------------
As a first note, this seems like a good use case of why moving HCat into Hive
is a good idea.
But in general could you explain more why HCat isn't a good solution? If it
doesn't work with Hadoop 0.20 we'd certainly be open to patches to fix that.
And if it proves useful to push more functionality down into the
HCatInput/HCatOutput format that you need that also seems fine. But is there
baggage in the current HCatInput and HCatOutput formats that you don't need or
want? Why not improve those rather than duplicate functionality?
> Add a non-sql API in hive to access data.
> -----------------------------------------
>
> Key: HIVE-3752
> URL: https://issues.apache.org/jira/browse/HIVE-3752
> Project: Hive
> Issue Type: Improvement
> Reporter: Nitay Joffe
>
> We would like to add an input/output format for accessing Hive data in Hadoop
> directly without having to use e.g. a transform. Using a transform
> means having to do a whole map-reduce step with its own disk accesses and its
> imposed structure. It also means needing to have Hive be the base
> infrastructure for the entire system being developed which is not the right
> fit as we only need a small part of it (access to the data).
> So we propose adding an API level InputFormat and OutputFormat to Hive that
> will make it trivially easy to select a table with partition spec and read
> from / write to it. We chose this design to make it compatible with Hadoop so
> that existing systems that work with Hadoop's IO API will just work out of
> the box.
> We need this system for the Giraph graph processing system
> (http://giraph.apache.org/) as running graph jobs which read/write from Hive
> is a common use case.
> [~namitjain] [~aching] [~kevinwilfong] [~apresta]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
