[GitHub] [spark] steveloughran commented on pull request #30141: [SPARK-33230][SQL] Hadoop committers to get unique job ID in "spark.sql.sources.writeJobUUID"

steveloughran commented on pull request #30141: URL: https://github.com/apache/spark/pull/30141#issuecomment-717324176 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] steveloughran commented on pull request #30141: [SPARK-33230][SQL] Hadoop committers to get unique job ID in "spark.sql.sources.writeJobUUID"

steveloughran commented on pull request #30141: URL: https://github.com/apache/spark/pull/30141#issuecomment-716545472 @dongjoon-hyun > Do you think you can add a test case, @steveloughran ? not easily. Would need a new Hadoop committer (subclass of FileOutputCommitter easiest) w

[GitHub] [spark] steveloughran commented on pull request #30141: [SPARK-33230][SQL] Hadoop committers to get unique job ID in "spark.sql.sources.writeJobUUID"

steveloughran commented on pull request #30141: URL: https://github.com/apache/spark/pull/30141#issuecomment-716544258 @rdblue I am going to add two things to the committers 1. Autogenerate in job setup. More specifically -generate in constructor, and in task setup, fail if that pat