nielspardon opened a new issue, #12597: URL: https://github.com/apache/gluten/issues/12597
### Background Gluten vendors its Substrait `.proto` files (`gluten-substrait/src/main/resources/substrait/proto/substrait/`) pinned near Substrait **v0.23.0** with local patches, while upstream Substrait is now at **0.98.0**. The JVM plan producer and both native consumers (Velox, ClickHouse) regenerate from this single proto source and ship together, and Gluten's Substrait plans are transient (built per query, serialized across JNI, discarded) — so there is **no wire-compatibility constraint** and the rebase is a pure code migration. ### Goal Bring the vendored proto to official 0.98.0 core (verbatim) plus Gluten's genuinely engine-specific additions grafted at non-colliding numbers, migrating the JVM producer and both native backends in lockstep. ### Approach Land as small, independently-green PRs, one per message family (each keeps `protoc` + JVM + both native builds compiling). Minimal rebase first — relocate only colliding fork fields and drop dead/decided-drop items; extracting engine-specific forks to formal Substrait extensions is deferred to a follow-up. ### Increments - [ ] **0** — Remove unused vendored proto files + dead proto-based type derivation + deprecated `Expression.Enum` - [ ] **1** — `plan.proto` / `extensions.proto` → 0.98 (URI→URN, `AdvancedExtension.optimization` repeated) - [ ] **2** — Type system: temporal types → `PrecisionTimestamp*`; drop `Type.Nothing` - [ ] **3** — `CrossRel` → `NestedLoopJoinRel` (+ join-type enum remap) - [ ] **4** — Windowing: `WindowRel` → `ConsistentPartitionWindowRel`; `WindowType` → `BoundsType` - [ ] **5** — `FetchRel` / `AggregateRel.Grouping` / `ExpandRel` - [ ] **6** — `TopNRel` / `WriteRel` - [ ] **7** — `ReadRel` (relocate `stream_kafka`; text-options redesign; `VirtualTable`) - [ ] **8** — `Rel` oneof reconciliation to exact 0.98 layout Related: upstream consolidation work is tracked under the `apache-gluten` label in `substrait-io/substrait`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
