nielspardon opened a new pull request, #12650: URL: https://github.com/apache/gluten/pull/12650
## What changes are proposed in this pull request? Next increment of the Substrait 0.98 proto rebase (#12597), following #12598 (dead-proto/`Expression.Enum` removal), #12604 (URI→URN referencing), and #12642 (`AdvancedExtension.optimization` singular→repeated) — all merged. This PR is rebased directly on `main`, so its diff is exactly this increment. Migrates the **type system** to the 0.98 temporal types. Substrait 0.98 replaced the parameterless temporal types with explicit-precision ones (substrait-io/substrait#682): `Timestamp`/`TimestampTZ`/`Time` were removed in favor of `PrecisionTimestamp`/`PrecisionTimestampTZ`/`PrecisionTime`, which carry an explicit `precision`. This is a **type-level migration only**; the `algebra.proto` `Literal` temporal fields are a separate, later increment. - **`type.proto`**: remove `Timestamp` (14), `TimestampTZ` (29), `Time` (17), and the deprecated `user_defined_type_reference` (31) — numbers `reserved`. Add `PrecisionTimestamp` (33), `PrecisionTimestampTZ` (34), `PrecisionTime` (36). Relocate Gluten's `Nothing` fork from kind 33 → 40 to free 33 for `precision_timestamp`; **`Nothing` is kept**, not dropped (the `NullType`→typed-null drop has cross-backend tentacles and is deferred to its own increment, so this PR is semantically neutral). - **JVM producer**: `TimestampTypeNode` emits `PrecisionTimestampTZ(precision=6)`; `TimestampNTZTypeNode` emits `PrecisionTimestamp(precision=6)`; `ConverterUtils.parseFromSubstraitType` reads `PRECISION_TIMESTAMP` → `TimestampNTZType` and `PRECISION_TIMESTAMP_TZ` → `TimestampType`. `Nothing` handling is unchanged. - **Native consumers**: Velox `SubstraitParser` (`kPrecisionTimestamp` → `TIMESTAMP_UTC`, `kPrecisionTimestampTz` → `TIMESTAMP`; `kNothing` kept) and `VeloxToSubstraitType` (`TIMESTAMP` → `PrecisionTimestampTZ(precision=6)`); ClickHouse `TypeParser` and `SerializedPlanBuilder` (`has_precision_timestamp_tz`, `precision=6`; `has_nothing` kept). `precision=6` is used everywhere because Spark timestamps are microsecond-precision and ClickHouse maps them to `DateTime64(6)` (also microseconds), so the migration is an exact round-trip with no precision loss or semantic change. ## How was this patch tested? Type-level migration with no behavioral change. A new round-trip test, `ConverterUtilsSuite`, asserts `TimestampType` → `PrecisionTimestampTZ(precision=6)` → `TimestampType` and `TimestampNTZType` → `PrecisionTimestamp(precision=6)` → `TimestampNTZType` (including nullability). Verified locally: the vendored proto compiles (`protoc`, no duplicate field numbers after the `Nothing` relocation); the `gluten-substrait` JVM build succeeds against the regenerated classes; `ConverterUtilsSuite` passes (2/2); and the Velox native library builds (`libgluten` / `libvelox`, recompiling `SubstraitParser` / `VeloxToSubstraitType`). The ClickHouse native changes (`TypeParser` / `SerializedPlanBuilder`) are covered by CI, which builds the CH backend. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) 🤖 Generated with AI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
