steveloughran commented on PR #45740: URL: https://github.com/apache/spark/pull/45740#issuecomment-2067069380
So both those bindings hand off to PathOutputCommitterFactory(), which looks for a committer from the config key mapreduce.outputcommitter.factory.class FileOutputCommitterFactory: classic committer NamedCommitterFactory: class in mapreduce.outputcommitter.named.classname then fallback to mapreduce.outputcommitter.factory.scheme.SCHEMA factory definition. The idea being; you get an fs specific one unless asked for. * the parquet one is there because parquet is fussy about its committer subclasses; that should be reviewed (where?) * and PathOutputCommitProtocol is probably surplus now that spark can use PathOutputCommitter everywhere... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
