Abhishek-Mohan opened a new pull request, #7161:
URL: https://github.com/apache/datafusion-comet/pull/7161
## Which issue does this PR close?
Part of #4419.
## Rationale for this change
This applies the #4419 vocabulary to the native shuffle guide. The rule IDs
refer to the style guide in #7151.
## What changes are included in this PR?
- **"Comet native operators" → "Comet operators".** Native shuffle accepts
any `CometPlan` child, including JVM-side ones
(`CometShuffleExchangeExec.scala:516-524`), so "native" was wrong as well as
vague.
- **"native" used as a noun → the Rust component (N4).** Examples:
- "because native cannot sort them" → "because the Rust range partitioner
cannot sort them"
- "hands to native" → "hands to the Rust side"
- "Native must fully consume it" → "`ShuffleScanExec` must fully consume
it"
- **"not converted natively" → "not converted to Comet".**
- **"on the native path" → "in native shuffle"** in the config table.
- **Unchanged:** "native shuffle" (N2), contributor-guide forms such as
"native code" and "native plan" (N3), and config keys (N6).
Two things on this page look out of date, but I left them for a separate
change because they need new content rather than rewording:
- A Celeborn-backed native shuffle needs a `CometNativeExec` child, not just
any `CometPlan`.
- When the shuffle input was converted by `CometSparkToColumnarExec`,
`RevertNativeForTransitionHeavyStages` switches the exchange to JVM shuffle
instead of keeping native shuffle.
## How are these changes tested?
Docs only.
- `prettier --check` passes.
- I checked each changed claim against `main`.
- I built the docs locally with Sphinx.
AI assistance: OpenCode helped check the page against the code and draft the
edits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
