peterxcli commented on PR #4746: URL: https://github.com/apache/datafusion-comet/pull/4746#issuecomment-5832990701
Thanks, all addressed: - Merged `main` rather than rebasing, to avoid a force push. The merge commit `032354cb3` has exactly the tree you get by replaying `278f28e0a` and `e9f5e5e7f` onto `main`, so the diff against `main` is only this PR's changes. - Updated the `escapedHdfsDestination` and `ParquetWriter`/`ParquetWriterExec` docs. Dropped the `Some(work_dir)` arm and the three fields natively, and reserved them in the proto (`f1b5e493c`). - `createTaskContext` moved to the companion object and takes `jobTrackerID`. The task closure only captures locals now. - `runNativeWriteJob` takes the `parallelize(Seq.empty[ColumnarBatch], 1)` swap and the `partitionId == 0 || batches.hasNext` check. Removed `assume(isSpark40Plus)` from the SPARK-23271 and empty-partition tests; both pass on Spark 3.5 (`086bc1eef`). With the swap in place, the `hasEmptyRelationInput` guard from #5821 looks unnecessary on 3.x. Happy to drop it here or in a follow-up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
