-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63544/
-----------------------------------------------------------
Review request for hive, Ashutosh Chauhan and Owen O'Malley.
Bugs: HIVE-17963
https://issues.apache.org/jira/browse/HIVE-17963
Repository: hive-git
Description
-------
For non-blobstore FS:
1) Move the temp directory to a new directory name, to prevent additional files
from being added by any runaway processes.
2) Run removeTempOrDuplicateFiles() on this renamed temp directory
3) Run renameOrMoveFiles() to move the renamed temp directory to the final
location.
For blobstore FS:
1) Remove duplicates from tmpPath
2) Use moveSpecifiedFiles() to perform a file-by-file move of the de-duped
files to specPath
Also remove hive.exec.move.files.from.source.dir setting - the above changes
will always be enabled.
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cbe4de5
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1be7eab
ql/src/test/queries/clientpositive/insert_with_move_files_from_source_dir.q
0117755
ql/src/test/queries/clientpositive/skewjoin.q 9ad4b77
Diff: https://reviews.apache.org/r/63544/diff/1/
Testing
-------
Thanks,
Jason Dere
