[
https://issues.apache.org/jira/browse/SQOOP-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433797#comment-16433797
]
Daniel Voros commented on SQOOP-3311:
-------------------------------------
Attached review request.
> Importing as ORC file to support full ACID Hive tables
> ------------------------------------------------------
>
> Key: SQOOP-3311
> URL: https://issues.apache.org/jira/browse/SQOOP-3311
> Project: Sqoop
> Issue Type: New Feature
> Components: hive-integration
> Reporter: Daniel Voros
> Assignee: Daniel Voros
> Priority: Major
>
> Hive 3 will introduce a switch (HIVE-18294) to create eligible tables as ACID
> by default. This will probably result in increased usage of ACID tables and
> the need to support importing into ACID tables with Sqoop.
> Currently the only table format supporting full ACID tables is ORC.
> The easiest and most effective way to support importing into these tables
> would be to write out files as ORC and keep using LOAD DATA as we do for all
> other Hive tables (supported since HIVE-17361).
> Workaround could be to create table as textfile (as before) and then CTAS
> from that. This would push the responsibility of creating ORC format to Hive.
> However it would result in writing every record twice; in text format and in
> ORC.
> Note that ORC is only necessary for full ACID tables. Insert-only (aka.
> micromanaged) ACID tables can use arbitrary file format.
> Supporting full ACID tables would also be the first step in making
> "lastmodified" incremental imports work with Hive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
