ulysses-you opened a new pull request, #57650: URL: https://github.com/apache/spark/pull/57650
### What changes were proposed in this pull request? When planned write is enabled, `FileFormatWriter` builds a `concurrentOutputWriterSpecFunc` that always wraps the plan with a sort and creates a concurrent output writer spec. This PR skips building the concurrent output writer spec when the output ordering already matches the required ordering, by making the func return `None` in that case. ### Why are the changes needed? The concurrent output writer keeps one open writer per dynamic partition and falls back to the sort-based sequential writer once the number of open writers exceeds `spark.sql.maxConcurrentOutputFileWriters`. When the input is already sorted by the required ordering, enabling the concurrent writer is pure overhead: it may open many writers and then fall back to the sort-based writer anyway. Skipping it avoids this wasteful fall-back. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? New UT in `V1WriteCommandSuite` that asserts, via the fall-back log, that the concurrent writer is disabled when the ordering already matches, with a control case (unsorted input) proving the assertion is discriminating. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
