----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72333/ -----------------------------------------------------------
Review request for hive, Ashutosh Chauhan and Thejas Nair. Bugs: HIVE-23154 https://issues.apache.org/jira/browse/HIVE-23154 Repository: hive-git Description ------- With rename(), we could run into a race condition between snapshot of files to be moved ("filesKept") and when the fs.rename happens. It is possible that run awaay task could have added more files to this. 1. Patch fixes the problem by relying on local threadpool to move the files instead of fs.rename (where S3AFileSystem's rename is inherently parallel). 2. Same race condition issue persists in "insert into" mode as well, which was relying on "fs.rename". Patch fixes this issue as well. Diffs ----- ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java e25dc54e7d Diff: https://reviews.apache.org/r/72333/diff/1/ Testing ------- Thanks, Rajesh Balamohan