LuciferYang opened a new pull request, #13019: URL: https://github.com/apache/gravitino/pull/13019
### What changes were proposed in this pull request? Adds a `spark-connector/v4.1` module mirroring `v4.0`, published as `gravitino-spark-4.1_2.13` and `gravitino-spark-connector-runtime-4.1_2.13`, pinned to Spark 4.1.3 and the Iceberg 1.11.0 the other lines already use. Four Spark 4.1 API changes needed more than a copy: - `ParserInterface.parsePlanWithParameters` is new in 4.1 with a default body, and `SparkSession.sql` calls it for every query. The authorization parser forwards it explicitly; inheriting the default routes through `parsePlan` and drops the parameter context, which leaves parameter markers unbound outside legacy parameter substitution. - Iceberg's Spark 4.1 module publishes only `SparkTable(Table)`, and Spark 4.1's `JDBCTable` takes a fourth constructor argument. The superclass constructor call for the two shared table classes now comes from a per-line base class under `spark-common/src/main/spark35`, `spark40` and `spark41`, so the table bodies stay shared. - `ProcedureCatalog.listProcedures` is abstract in 4.1, implemented the way Iceberg's own `BaseCatalog` does it. - Iceberg raises a different exception for a version reference that no longer exists: up to its Spark 4.0 module the check lives in `SparkCatalog` and throws `ValidationException`, and from its Spark 4.1 one it lives in `SparkTable` and throws `IllegalArgumentException`. The shared Iceberg IT asks the subclass for the expected type rather than hardcoding it. Paimon stays out of the build: no released Paimon version publishes `paimon-spark-4.1`. Two shared-tree cleanups the 4.1 compile surfaced: an unused `org.jetbrains.annotations.NotNull` import, which resolved only because ORC leaked that jar onto the 4.0 compile classpath and 4.1 no longer does, and two commons-lang3 helpers the newer commons-lang3 deprecates, replaced with plain Java. The deprecation suppression that `UpdateColumnDefaultValue#newDefaultValue` needs is scoped to that one declaration, because `-Xlint:deprecation -Werror` applies to every module and a method-wide suppression would hide the next deprecation in the same method. On CI, the fast `spark-connector-build` job now builds all three connector modules instead of 3.5 alone, so a compile break in a Spark 4 module fails in minutes instead of waiting for the integration job. ### Why are the changes needed? Spark 4.1 is released and users on it have no connector build. Iceberg 1.11.0 publishes an `iceberg-spark-runtime` for 4.1 and none for 4.2, so 4.1 is the newest line the connector can support. Fix: #13018 ### Does this PR introduce _any_ user-facing change? Two new published artifacts, `gravitino-spark-4.1_2.13` and `gravitino-spark-connector-runtime-4.1_2.13`. No property or API changes. Paimon is unavailable on 4.1, as it is on 4.0. ### How was this patch tested? A new unit test pins the parser forwarding: delete the override and it fails, both on the `verify` that the delegate received `parsePlanWithParameters` and on the assertion that the denied-table state was cleared before delegating. The shared integration suite runs against the new module in CI in both embedded and deploy mode, alongside 3.5 and 4.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
