malinjawi opened a new pull request, #13158: URL: https://github.com/apache/gluten/pull/13158
## What changes are proposed in this pull request? Delta's `IncrementMetric` adds one to a SQL metric each time it is evaluated. Gluten stripped it at any depth of a projection and credited the metric with the operator's output row count. That is wrong for the counters MERGE keeps inside its per-clause `CASE WHEN` branches: every MERGE commit reported inflated `numTargetRowsInserted` / `Updated` / `Deleted` / `Copied`, and a no-op MERGE produced phantom change-data-feed rows because `CDCReader.shouldSkipFileActionsInCommit` relies on those counters being zero. This PR offloads a project or filter carrying `IncrementMetric` only where the output row count equals Spark's evaluation count: a stack of `IncrementMetric` at the root of a projection alias, or a filter over a literal `true`. Everything else stays on Spark with an explicit fallback reason. DELETE, UPDATE and MERGE's source-row counters keep their offload; only MERGE's write projection falls back. The transformers now strip only the root stack and derive their metric list once. The `delta_spark_ut` baseline drops the 225 entries that the offload-disabled experiment in #13112 showed pass without this defect. Fixes: #9003 Related: #9945, #12743 ## How was this patch tested? - New `IncrementMetricOffloadSuite` (no SparkSession) covering the predicate and the stripping. - New `DeltaSuite` tests: exact MERGE metrics with delete, update and insert clauses; a no-op MERGE with change data feed producing no change rows; DELETE and UPDATE metrics with and without deletion vectors. Expected values verified against vanilla Delta. - Compiled with tests on the `spark-3.4`, `spark-3.5` and `spark-4.0`/`scala-2.13` Delta profiles. ## Was this patch authored or co-authored using generative AI tooling? Co-authored with IBM Bob. Generated-by: IBM Bob 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
