Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/19294 As I play with commit logic all the way through the stack, I can' t help thinking everyone's lives would be better if we tagged the MRv1 commit APIs as deprecated in Hadoop 3. and uses of the commit protocols went fully onto the v2 committers: one codepath to get confused by, half as much complexity. The issue with the custom stuff is inevitably Hive related, isn't it? It's always liked to scatter data around a filesystem and pretend its a single dataset
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org