damccorm opened a new pull request, #41141: URL: https://github.com/apache/beam/pull/41141
### Summary Fixes `IcebergReadWriteIT` (`testSqlWriteAndRead`, `testSqlInsertOverwrite`, `testSqlFilterPushdown`, `testSqlProjectPushdown`) failing in `PostCommit SQL` ([#42721081845](https://github.com/apache/beam/actions/runs/42721081845)). ### Root Cause In commit `74f141d1931` (#40982), `sdks/java/extensions/sql/iceberg/build.gradle` added `testImplementation project(":sdks:java:extensions:sql:jdbc")` for `BeamSqlLineIcebergCdcTest`. Because `:sdks:java:extensions:sql:jdbc` uses `shadowClosure: {}`, its default runtime artifact includes `beam-sdks-java-extensions-sql-jdbc-*-SNAPSHOT.jar`—an unrelocated fat JAR that bundles Parquet `1.15.2` classes (from `:sdks:java:io:parquet`). When placed on `:sdks:java:extensions:sql:iceberg`'s `testRuntimeClasspath` ahead of `parquet-column:1.17.1` (required by Iceberg `1.12.0` after #40971), reading Parquet files in `IcebergReadWriteIT` threw: ``` java.lang.NoSuchMethodError: 'org.apache.parquet.schema.LogicalTypeAnnotation$VariantLogicalTypeAnnotation org.apache.parquet.schema.LogicalTypeAnnotation.variantType(byte)' at org.apache.iceberg.parquet.ParquetTypeVisitor.visit(ParquetTypeVisitor.java:50) ``` Because `NoSuchMethodError` is a `java.lang.Error` and `--blockOnRun=false` is set in `integrationTest`, `DirectRunner` aborted worker execution without transitioning pipeline state, causing `IcebergReadWriteIT` to fail with `Expected: <DONE> but: was <RUNNING>`. ### Fix - Depend on `project(":sdks:java:extensions:sql:jdbc").sourceSets.main.output` (following the pattern in `sdks/java/extensions/sql/jdbc/build.gradle`) plus `jline` and `sqlline` test runtime dependencies rather than the fat `shadowJar`, and align `iceberg-api`/`iceberg-core` test dependencies to `1.12.0`. - Verified locally with `./gradlew :sdks:java:extensions:sql:iceberg:test --tests ...BeamSqlLineIcebergCdcTest :sdks:java:extensions:sql:iceberg:integrationTest --tests ...IcebergReadWriteIT.testSqlWriteAndRead`. ### Verification - Triggered `beam_PostCommit_SQL.yml` workflow run on branch: [`42732863040`](https://github.com/apache/beam/actions/runs/42732863040) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
