Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18975 Even renaming staging location is not atomic, right? Hive users might still see the extra data during the rename, right? I am not sure how Hive works. If our underlying system is Linux, the [rename of Linux doc](https://linux.die.net/man/3/rename) shows > If one or more processes have the file open when the last link is removed, the link shall be removed before rename() returns, but the removal of the file contents shall be postponed until all references to the file are closed.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org