steveloughran commented on PR #5689: URL: https://github.com/apache/hadoop/pull/5689#issuecomment-1573949366
safe when you don't have incompatible clients trying to write to the same directory tree, more specifically using rename and delete, which effectively means "hive, mapreduce, spark jobs etc". the s3a committer doesn't use rename to commit work, but when spark starts a job it will delete the dest directory -so if delete() mistakes a marker at the root for an empty dir, it may not delete the old data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
