[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128955#comment-17128955 ] Apache Spark commented on SPARK-29295: -- User 'turboFei' has created a pull request for this issue:

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057772#comment-17057772 ] Dongjoon Hyun commented on SPARK-29295: --- This lands to `branch-2.4` via

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057283#comment-17057283 ] Dongjoon Hyun commented on SPARK-29295: --- I confirmed that Apache Spark 2.1.3 and older versions

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057260#comment-17057260 ] Dongjoon Hyun commented on SPARK-29295: --- I marked this as a `correctness` issue. > Duplicate

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17057261#comment-17057261 ] Dongjoon Hyun commented on SPARK-29295: --- Hi, [~viirya]. Could you make a backport against

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949886#comment-16949886 ] angerszhu commented on SPARK-29295: ---  This probelm seems start from hive 1.2 I test in our env

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941483#comment-16941483 ] L. C. Hsieh commented on SPARK-29295: - I created a PR for this issue. > Duplicate result when

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941475#comment-16941475 ] feiwang commented on SPARK-29295: - Yes, how can we resolve this issue? Only by upgrading the hive

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941060#comment-16941060 ] L. C. Hsieh commented on SPARK-29295: - I traced code last night, and yes it looks like a Hive bug.

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940754#comment-16940754 ] feiwang commented on SPARK-29295: - relative hive issue, https://issues.apache.org/jira/browse/HIVE-17063

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940718#comment-16940718 ] L. C. Hsieh commented on SPARK-29295: - Thanks for reporting this. Will take a look. > Duplicate

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940600#comment-16940600 ] feiwang commented on SPARK-29295: - cc [~cloud_fan] > Duplicate result when dropping partition of an

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940599#comment-16940599 ] feiwang commented on SPARK-29295: - I have tested in 2.3.1 branch, without SPARK-25271, it will always

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940598#comment-16940598 ] feiwang commented on SPARK-29295: - When we set CONVERT_METASTORE_PARQUET=true, it will use