I've got a PR up to completely remove the v2 commit algorithm

https://github.com/apache/hadoop/pull/2320

That may seem overkill, but while *we* know there's a small window of risk
(task attempt 1 failing partway through a nonatomic commit), that's not
known/appreciated by others.

The patch removes the v2 codepath from FileOutputCommitter, making it a lot
less complicated, and when v2 is requested, a warning is printed and the
option ignored.

Overkill? Maybe. But it guarantees correctness

Reply via email to