FelixYBW opened a new pull request, #13374: URL: https://github.com/apache/gluten/pull/13374
## What changes are proposed in this pull request? Adds `docs/developers/ArrowColumnarConversion.md` (and a mkdocs nav entry), which documents: 1. **Conventions**: `RowType` / `BatchType`, `ConventionReq`, `ConventionFunc`, the transition graph, and how `InsertTransitions` inserts transitions. 2. **Types and batch layout**: `VanillaRowType`, `VanillaBatchType`, `ArrowJavaBatchType` (heavy batch), `ArrowNativeBatchType` (light batch), `VeloxBatchType`, `VeloxCarrierRowType`, with the transition graph as a mermaid diagram and the composite paths the planner builds. 3. **What each transition does**: `LoadArrowDataExec`, `OffloadArrowDataExec`, `ArrowColumnarToVeloxColumnarExec` and the row transitions, down to the JNI / Velox calls, where data is copied, and the in-place, reference-counted batch lifecycle. 4. **Arrow-based operators today.** 5. **How to add an Arrow-based columnar operator**, step by step with a checklist, using the Lance scan offload (#12958) as the example: convention, batch ownership, validation, offload rule, component registration, tests. 6. **Ongoing work**: #13280, SPARK-57468 and the upstream Arrow batch ownership work. Based on #12289 and https://github.com/apache/gluten/issues/13140#issuecomment-6281254885. ## How was this patch tested? Docs only. Checked the described behavior against the code on `main`, and that the relative links resolve. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude claude-opus-5-5 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
