LuciferYang opened a new pull request, #12999: URL: https://github.com/apache/gluten/pull/12999
## What changes are proposed in this pull request? `backends-bolt` does not compile on main. #12454 was merged without rebasing onto #12954, which removed 19 methods from the `SparkShims` trait, and #12981 then landed on top of it, removing the Spark-3.3-only native-write post-rule pipeline. No CI job builds the `backends-bolt` profile, so nobody noticed: `mvn test-compile -Pbackends-bolt` fails with twelve errors. This restores the build. The wider drift the failure exposed is filed separately as #12998. Six call sites move off deleted shim methods, each to what Velox already does: `enableNativeWriteFilesByDefault()` becomes the literal `true`, `generateMetadataColumns` becomes `FileMetadataUtil.generateMetadataColumns`, `withTryEvalMode` and `withAnsiEvalMode` become `ExpressionUtils.withTryEvalMode` / `withAnsiEvalMode`, `unBase64FunctionFailsOnError(expr)` becomes `expr.failOnError`, and `extractExpressionTimestampDiffUnit` becomes an inline match on `TimestampDiff`. The two that turn into a literal or a field read were checked against the four per-version shim bodies as they stood before #12954: every one of them returned `true` and `unBase64.failOnError` respectively, so the values are the ones the shim produced on 3.4, 3.5, 4.0 and 4.1. The trait defaults they replaced (`false` in both cases) were reachable only on 3.3. Five deletions, none of which loses behaviour. `skipNativeCtas` and `skipNativeInsertInto` no longer exist on `BackendSettingsApi` and nothing in the repo reads them, so the overrides cannot compile; the bucket restriction the second one expressed still holds on the live path, through `validateBucketSpec()` in `supportWriteFilesExec`. The `NativeWritePostRule` injection goes because that pipeline stopped being reachable when 3.3 left: since 3.4 a write is a real `WriteFilesExec` node, the only surviving user of the fake-row adaptor is the noop format's `GlutenNoopWriterRule`, and Velox has shipped native write without ever injecting the rule. The `getExtendedColumnarPostRules()` loop goes because all four shims returned an empty list, so it never iterated. `@NotNull` goes from `BoltBlockStripes.iterator()` because `org.jetbrains.annotations` is not on the main compile classpath under `-Pspark-4.1`, and the base method plus `VeloxBlockStripes.iterator()` carry no annotation either. Finally one `// Since spark 3.3.0` comment next to the changed `genTimestampDiffTransformer` is dropped, since its sibling lost the same comment here. `extractExpressionTimestampAddUnit` deliberately stays on the shim. Unlike the Diff variant, its 4.0 and 4.1 overrides carry an extra guard that rejects a quantity above `Int.MaxValue`, and inlining would silently drop it. Out of scope, and listed in #12998: `-Pbackends-bolt -Pdelta` still does not compile, because `BoltDeltaComponent` calls `OffloadDeltaScan()` while the case class has taken a parameter since it grew deletion-vector support. That one needs a decision about bolt rather than a mechanical copy, so it is not in this PR. ## How was this patch tested? Compile only. Scala 2.13 throughout, since Spark 4.x publishes no 2.12 artifacts. | profile | built | |-|-| | 3.4, 3.5 | `-Pbackends-bolt -Pspark-ut`, including bolt's own test sources | | 4.0, 4.1 | `-Pbackends-bolt` | `spotless:check` is clean. 4.0 and 4.1 leave `-Pspark-ut` off because `gluten-ut/spark40` and `gluten-ut/spark41` have no `src/test/backends-bolt` directory, which is a pre-existing gap in how bolt is wired into gluten-ut rather than anything this PR changes. No suite ran. Bolt's native library needs Linux with GCC 10-12 or Clang 16 per `docs/bolt-quick-start.md`, so no bolt test can execute on the machine this was written on. Someone with a Linux build, or a CI job of the kind #12998 asks for, is what would exercise these paths. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude claude-opus-5 Related issue: #12998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
