nielspardon opened a new pull request, #12704: URL: https://github.com/apache/gluten/pull/12704
## What changes are proposed in this pull request? Substrait 0.98 replaced `FetchRel`'s scalar `int64 offset = 3` / `int64 count = 4` with `Expression offset_expr = 5` / `Expression count_expr = 6`. This migrates Gluten's vendored proto, the JVM producer, and both native consumers in lockstep, as one step of the Substrait `v0.23.0` → `0.98.0` proto rebase (#12597). Gluten's only `FetchRel` producer (`LimitExecTransformer`) always supplies literal `Long` offset/count from Spark's Limit/Offset plans, so the change is mechanical: the producer wraps each value into an i64-literal `Expression`, and the consumers extract the literal back out — guarded by `has_offset_expr()`/`has_count_expr()`, falling back to `0` when unset (matching the 0.98 "offset unset is treated as 0" semantics). `FetchRel` stays at Rel-oneof field 3, so there is no oneof renumbering. - **Proto:** reserve fields 3/4, add `offset_expr = 5` / `count_expr = 6` (vendored verbatim from 0.98). - **Producer:** `FetchRelNode` wraps offset/count via `ExpressionBuilder.makeLongLiteral(...)`. - **Velox:** `SubstraitToVeloxPlan` and `SubstraitToVeloxPlanValidator` read the literals via `SubstraitParser::getLiteralValue<int64_t>`; `VeloxToSubstraitPlan` (the Velox→Substrait reverse converter) writes i64-literal expressions. - **ClickHouse:** `FetchRelParser` and `SortRelParser::parseLimit` read the `count_expr` / `offset_expr` literals. ## How was this patch tested? - New `FetchRelProtoSuite` pins the producer contract (offset/count land as i64 literal expressions). - Locally verified: `protoc` dup-field check; `gluten-substrait` builds and the new suite passes (`mvn -Pspark-3.5`); the Velox native backend compiles (consumer, validator, and reverse converter); `clang-format` (v15) clean on all changed C++ files. - Existing end-to-end LIMIT/OFFSET suites exercise the consumer extraction on both backends. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) 🤖 Generated with AI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
