schumiyi opened a new pull request, #13657: URL: https://github.com/apache/gluten/pull/13657
## What changes are proposed in this pull request? Fixes #13655. Velox emits POSEXPLODE ordinality as a one-based BIGINT, but the native Generate rewrite declares it using Spark's original INT attribute. When the normalization cast runs in `ColumnarPartialProjectExec` (for example, with `cast` blacklisted), JVM expression binding selects `getInt` on the BIGINT Arrow vector and fails before the cast executes. Use a separate `LongType` attribute with a fresh exprId for the native output and the cast input. The post projection continues to return Spark's zero-based INT position with its original exprId and qualifier. ARRAY/MAP and OUTER handling preserve the existing layout and marker semantics; a rejected native candidate restores the original Spark Generate. ## How was this patch tested? - Added `PullOutGenerateProjectSuite` under `gluten-ut/spark35`: 10 JVM tests cover schema/identity, ARRAY/MAP, inner/outer, null values and padding, rejected-candidate fallback, and Arrow/MutableProjection reading. A source-isolated harness using the exact production post-project method and actual rewrite manager was run with public Spark 3.5.5, Scala 2.12.18, Arrow 15.0.0 and JDK 17: unpatched main failed 10/10; patched method passed 10/10. The unpatched Arrow case reproduced `UnsupportedOperationException` in `ArrowColumnVector.getInt`. - Added materialized Parquet integration coverage to `UDFPartialProjectSuite`: native Generate on/off, ARRAY/MAP, inner/outer, empty/null collections, and null elements/values. Assertions require native Generate and the partial project consuming its LONG ordinal, preventing complete fallback from producing a false pass. - `./dev/format-scala-code.sh`, `./.github/workflows/util/check.py header main --fix`, and `git diff --check` passed. The integration suite also passed Scala parsing. - Full Gluten/native build and native SQL integration were not run locally; the JVM harness does not replace those CI checks. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-6.1-sol and GPT-6-astra) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
