Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/10543 )
Change subject: IMPALA-6119: Fix issue with multiple partitions sharing same location ...................................................................... Patch Set 20: (3 comments) http://gerrit.cloudera.org:8080/#/c/10543/20//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10543/20//COMMIT_MSG@19 PS20, Line 19: For managed tables it's no longer allowed to drop a partition that : shares it's location with other partitions. In this case an error is : displayed to the user. > what was the previous behavior here? it would drop and remove the files, an The behavior is that Hive would delete the dirs/files (see the discussion in the comments for June 1). We thought that may have been unintentional given the discussion on the related Hive jira so decided to prevent the case when dropping (see June 14 comment). If they do want to drop a such a partition, I'd expect that they point each duplicate to another location-- though it would be useful to try this first and include the workaround in the error message. http://gerrit.cloudera.org:8080/#/c/10543/18/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10543/18/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1435 PS18, Line 1435: / Only reload file metadata of partitions speci > not really. the code in that patch loops over the partitions to delete, and whoops, indeed, looks like I was looking at the wrong ps. the change in 20 looks fine. there was an earlier question in the review about benchmarking-- have you tried these operations with a table with many partitions to see how much of a slow-down we should expect? http://gerrit.cloudera.org:8080/#/c/10543/20/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10543/20/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1380 PS20, Line 1380: its easier wording: ... that have locations in common with partitions in partitionsToUpdate. -- To view, visit http://gerrit.cloudera.org:8080/10543 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a54bc8224bcefe65b83de2df58bb84629f2aa4a Gerrit-Change-Number: 10543 Gerrit-PatchSet: 20 Gerrit-Owner: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Michael Brown <mi...@cloudera.com> Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Tue, 17 Jul 2018 00:30:45 +0000 Gerrit-HasComments: Yes
