[
https://issues.apache.org/jira/browse/IMPALA-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826527#comment-16826527
]
Todd Lipcon commented on IMPALA-8454:
-------------------------------------
Chatting with Gopal, it turns out that actually the Hive-on-Tez behavior is to
always recursively list directories, including for partitions of external
tables. This is actually important for interop on external tables because Tez
will write to subdirectories for an insert if the insert contains a 'UNION ALL'
(even in Hive 2 non-ACID).
We should probably consider making this a global flag and enabling by default
(with the ability to roll back in case it breaks someone)
> Recursively list files within transactional tables
> --------------------------------------------------
>
> Key: IMPALA-8454
> URL: https://issues.apache.org/jira/browse/IMPALA-8454
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Major
>
> For transactional tables, the data files are not directly within the
> partition directories, but instead are stored within subdirectories
> corresponding to writeIds, compactions, etc. To support this, we need to be
> able to recursively load file lists within partition directories.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
