LuciferYang commented on PR #55612:
URL: https://github.com/apache/spark/pull/55612#issuecomment-4350126373
Hi @aokolnychyi, this looks to have broken master `sql - other tests`.
Parent commit `68b936a9c36` is green; merge commit `5ef2e1ba174` and HEAD
`384cd0d3e83` both have 25 failures, all in the 7 Delta-based row-level
operation suites.
The failing tests are the "transactional checks" ones added in this PR
(`MergeIntoTableSuiteBase.scala:72` and several around
`UpdateTableSuiteBase.scala:952`). Sample failure from
`DeltaBasedMergeIntoTableSuite.self merge with transactional checks`:
```
ArrayBuffer([IsNotNull(dep), EqualTo(dep,hr)], [IsNotNull(salary),
EqualTo(salary,100)],
[IsNotNull(dep), EqualTo(dep,hr)], [IsNotNull(salary),
EqualTo(salary,100)])
had size 4 instead of expected size 2
```
The doubling matches the new group-filter scan: `matchingRowsPlan` re-scans
the target, and for MERGE `RewritePredicateSubquery` also re-scans the source.
The hard-coded counts in the new base-class tests still assume the old delta
behavior:
```scala
val expectedNumScans = if (deltaMerge) 2 else 4 // delta now 4
val expectedNumTargetScans = if (deltaMerge) 1 else 2 // delta now 2
val expectedNumSourceScans = if (deltaMerge) 1 else 2 // delta now 2
```
The `DeltaBased*Suite.scala` files were updated in this PR but the
assertions in `MergeIntoTableSuiteBase` / `UpdateTableSuiteBase` were missed.
CI: check-run `73714758514` on `5ef2e1ba174`. Do you have time to push a
follow-up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
