czxm opened a new pull request, #39595: URL: https://github.com/apache/spark/pull/39595
### What changes were proposed in this pull request? This is based on @coalchan 's previous work of https://github.com/apache/spark/pull/35549 . I enhanced it as per @jackylee-ch 's comment below so that no new parameter is needed. https://github.com/apache/spark/pull/35549#issuecomment-1062790013 ### Why are the changes needed? 1) method listPartitions is order to get locations of partitions and compute custom partition locations(variable customPartitionLocations), but customPartitionLocations is only used while overwriting static hive partitions 2) method listPartitions is costly and only used in one case. So we can use listPartitionNames to reduce requests on hive metastore db. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? 1) created a nested partitioned table 2) run the insert/overwrite static/dynamic partition cases such as: insert overwrite table p1 partition(p1="1",p2="1.1",p3="1.1.1") select 1 insert into table p1 partition(p1="1",p2="1.1",p3="1.1.1") select 1 insert overwrite table p1 partition(p1="1",p2,p3) select 1,"1.1","1.1.1" insert into table p1 partition(p1="1",p2,p3) select 1,"1.1","1.1.1" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
