[
https://issues.apache.org/jira/browse/HIVE-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030836#comment-13030836
]
Steven Wong commented on HIVE-2087:
-----------------------------------
This problem seems to happen only when there is no static partition column.
> Dynamic partition insert performance problem
> --------------------------------------------
>
> Key: HIVE-2087
> URL: https://issues.apache.org/jira/browse/HIVE-2087
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon EMR, S3
> Reporter: Q Long
>
> Create an external(backed by S3) table T, make it partitioned by column P.
> Populate table T so it has large number of partitions (say 100). Execute
> statement like
> insert overwrite table T partition (p) select * from another_table
> check hive server log, and it will show that all existing partitions will be
> read and loaded before any mapper starts working. This feels excessive, given
> that the insert statement may only create or overwrite a very small number of
> partitions. Is there other reason that insert using dynamic partition
> requires loading the whole table?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
