andygrove opened a new pull request, #6151: URL: https://github.com/apache/datafusion-comet/pull/6151
## Which issue does this PR close? Closes #6142. ## Rationale for this change `IcebergWriteStrategy` is a planner strategy, and Spark runs planner strategies whether or not Comet is enabled. It only checked `spark.comet.write.iceberg.splitOperator.enabled`, so a session with `spark.comet.enabled=false` and the split flag on still planned Iceberg writes as Comet's `IcebergCommitExec` over `IcebergWriteExec`. The flag defaults to `false` today, so this has no effect for most users. It blocks flipping the split-operator default (#5644), because after that `spark.comet.enabled=false` would no longer restore Spark's own write plan. That setting is the first thing users reach for when they suspect Comet. ## What changes are included in this PR? - `IcebergWriteStrategy.apply` returns `Nil` unless `isCometLoaded(conf)` is true, the same check `CometExecRule` applies before converting a plan. `isCometLoaded` covers `spark.comet.enabled` and the other conditions under which the extension disables itself. - A new test in `CometIcebergWriteActionSuite` runs an `INSERT` with Comet disabled and the split flag on. It asserts one commit, no `IcebergCommitExec` or `IcebergWriteExec` in the captured plans, and the written rows. ## How are these changes tested? The new test, plus the rest of `CometIcebergWriteActionSuite` on the default profile: 70 succeeded, 0 failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
