[
https://issues.apache.org/jira/browse/FLINK-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299781#comment-14299781
]
Robert Metzger commented on FLINK-1466:
---------------------------------------
Maybe we can add this to {{flink-contrib}} to avoid polluting our dependencies
with Hive?
Are you planning to use HCatalogs HCatInputFormat with the
HadoopInputFormatWrapper of Flink?
I think that the HCatInputFormat provides a nice abstraction for the different
serialization formats supported by HCatalog/Hive Metastore.
Does HCatalog/Metastore also provide statistics about the data that our
optimizer can use?
> Add InputFormat to read from (Hive) tables stored in MetaStore
> --------------------------------------------------------------
>
> Key: FLINK-1466
> URL: https://issues.apache.org/jira/browse/FLINK-1466
> Project: Flink
> Issue Type: New Feature
> Components: Java API, Scala API
> Reporter: Fabian Hueske
> Assignee: Fabian Hueske
> Priority: Minor
>
> Hive features a table catalog called MetaStore to store information about
> tables. This information includes:
> - Table schema (field names, types)
> - Storage location in HDFS
> - HadoopInputFormat to read the data
> - SerDe information to extract fields
> - Information about partitions
> This JIRA proposes to add an InputFormat to read Hive tables. For a given
> database and table name, the IF should access the megastore to get all
> relevant information to read the data. Further parameters are field names
> (selection) and partitions (limited predicates).
> For a first version, the IF can be limited to flat tables (Hive supports
> nested data) with primitive field types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
