mizulun opened a new pull request, #6382:
URL: https://github.com/apache/datafusion-comet/pull/6382
## Which issue does this PR close?
Closes #6122.
## Rationale for this change
In `AdaptiveQueryExecSuite`, the test `Reuse the default parallelism in
local shuffle read` had its assertion changed from Spark's
`assert(localReads.length == 2)` to `== 1` in the 3.5.9, 4.0.4 and 4.1.3 diffs.
The next statement reads `localReads(1)`, so the test cannot pass for any
length. With one local read it throws `IndexOutOfBoundsException`, and with two
the assertion fails.
CI stays green because the test carries `IgnoreComet`, so it only runs when
Comet is disabled. That is the Spark-only baseline run. The 4.2.0 diff already
keeps Spark's value, and this PR brings the other three diffs in line with it.
## What changes are included in this PR?
- `dev/diffs/3.5.9.diff`, `dev/diffs/4.0.4.diff`, `dev/diffs/4.1.3.diff`:
drop the
`// Comet shuffle changes shuffle metrics` comment and restore
`assert(localReads.length == 2)`.
The `IgnoreComet` tag on the test stays.
- Each diff was regenerated as described in `spark-sql-tests.md`: apply it
to the Spark tag, edit the source, then run `git diff`. Apart from the removed
hunk, the only changes are the line offsets of later hunks in
`AdaptiveQueryExecSuite.scala`.
## How are these changes tested?
For each Spark version, the diff was applied to its tag checkout and the
test was run in both modes:
NOLINT_ON_COMPILE=true ENABLE_COMET=<false|true> build/sbt \
"sql/testOnly
org.apache.spark.sql.execution.adaptive.AdaptiveQueryExecSuite -- -z \"Reuse
the default parallelism in local shuffle read\""
| Spark | `ENABLE_COMET=false` | `ENABLE_COMET=true` |
|---|---|---|
| 3.5.9 | succeeded 1 | ignored 1 |
| 4.0.4 | succeeded 1 | ignored 1 |
| 4.1.3 | succeeded 1 | ignored 1 |
The baseline run now exercises the restored assertion, and the Comet run
still skips the test through the existing `IgnoreComet` tag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
