LuciferYang opened a new pull request, #12946:
URL: https://github.com/apache/gluten/pull/12946
## What changes are proposed in this pull request?
The `iceberg` and `iceberg-test` profiles each declared a
`build-helper-maven-plugin` execution named `add-iceberg-test-sources`, and
another named `add-iceberg-test-resources`. Maven merges plugin executions by
id, so activating both profiles kept only one `<sources>` list per id. This
gives the `iceberg-test` pair its own ids and changes nothing else.
Fixes #12933.
Before the change, `help:effective-pom` with `-Pbackends-clickhouse
-Pspark-3.5 -Pspark-ut -Pdelta -Piceberg -Piceberg-test` resolved
`add-iceberg-test-sources` to `src-iceberg-spark35/test/{scala,java}` alone.
The `iceberg` profile's `src-iceberg/test/{scala,java}` and
`src-iceberg${iceberg.binary.version}/test/{scala,java}` were dropped with no
warning. After the change all four executions survive in both backends.
For ClickHouse the old behaviour made that profile combination unbuildable.
`ClickHouseIcebergHiveTableSupport` lives in `src-iceberg/test/scala` and the
Java test under `src-iceberg-spark35/test/java` instantiates it, so the class
left the source path at exactly the moment the test joined it:
```
$ ./build/mvn -Pbackends-clickhouse -Pspark-3.5 -Pscala-2.13 -Pspark-ut
-Pdelta \
-Piceberg -Piceberg-test -DskipTests test-compile -pl
backends-clickhouse -am
[ERROR]
.../src-iceberg-spark35/test/java/.../TestPositionDeletesTableGluten.java:[41,18]
cannot find symbol
[ERROR] symbol: class ClickHouseIcebergHiveTableSupport
```
Velox was unaffected because nothing there needed both directories at once:
the `src-iceberg/test` suites run in the jobs that pass `-Piceberg` alone, and
the files under `src-iceberg-spark34/test` run in the three groups that pass
both.
The one behaviour change a reviewer should weigh is that those three groups,
`velox_backend_x86.yml:817`, `:821` and `:825`, plus the Spark 4.0
`test-compile` at `:1277`, now also compile `src-iceberg/test/**`.
`VeloxIcebergSuite` and `VeloxTPCHIcebergSuite` therefore run twice across the
matrix, once there and once in the `-Piceberg`-only jobs at `:941` and `:1013`.
That costs job time and gains nothing by itself, so if it is unwelcome the
alternative is to drop `-Piceberg` from those three commands, which I did not
do here because it would change which dependencies they resolve as well.
`src-iceberg${iceberg.binary.version}/test` and every
`src-iceberg*/test/resources` directory are empty in all modules today, so the
resources rename and that half of the sources rename are latent fixes with no
effect on the current tree. Only `backends-velox` and `backends-clickhouse`
hold files under `src-iceberg*/test`, so nothing outside those two modules is
touched.
While checking for the same shape elsewhere in the root pom, the only other
cross-profile id reuse is `enforce-java-17+` and `enforce-scala-213`, shared by
the `spark-4.0` and `spark-4.1` profiles. Those two are mutually exclusive, so
the merge never happens and I left them alone.
## How was this patch tested?
All four runs are on `e0dce586b` with only this change applied, JDK 17.
| what | result |
|-|-|
| ClickHouse `test-compile`, `-Piceberg -Piceberg-test`, before the change |
fails on `ClickHouseIcebergHiveTableSupport` |
| the same command after the change | exit 0, emits
`ClickHouseIcebergHiveTableSupport.class` and
`TestPositionDeletesTableGluten.class` |
| Velox `test-compile` with the profile set the three x86 groups use
(`-Pjava-17 -Pspark-3.4 -Pspark-ut -Piceberg -Piceberg-test -Pdelta -Phudi
-Ppaimon`) | BUILD SUCCESS |
| `help:effective-pom` for both backends after the change | all four
executions present, six test source directories each |
The Velox run is the one that matters for regressions, since it is the
combination that newly compiles five more files. It emitted
`VeloxIcebergSuite`, `VeloxTPCHIcebergSuite`, `enhanced/VeloxIcebergSuite` and
`TestIcebergNestedFieldVisitor` while keeping the existing
`src-iceberg-spark34/test` classes.
I did not run the ClickHouse tests themselves.
`TestPositionDeletesTableGluten` compiles for the first time here but no job
passes `-Piceberg-test` to a ClickHouse build, so it still does not execute;
#12934 covers whether it should.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude claude-opus-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
