Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/19269 w.r.t init, I'm thinking it's critical to get the DataframeWriter.extraOptions down the tree. This lets committers be tuned on a query-by-query basis for things like conflict management, rather than decide upfront. I'm going have the Hadoop MR committer factory support a `Map[String, String]` for the options, with the expectation that options hwere will overrride any in the job conf.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org