kentkwu opened a new pull request, #24890: URL: https://github.com/apache/datafusion/pull/24890
## Which issue does this PR close? - Closes #24886 ## Rationale for this change Multi-partition queries panic on `wasm32-unknown-unknown` when they funnel through `CoalescePartitionsExec` — most commonly `UNION ALL`, but any plan whose intermediate results have more than one partition (e.g. a multi-file external table scan) trips this. The operator's multi-input branch spawns via `JoinSet::spawn`, which requires a tokio reactor that isn't installed under `wasm-bindgen-futures`: ``` panicked at datafusion/common-runtime/src/join_set.rs:69:20: there is no reactor running, must be called from the context of a Tokio 1.x runtime ``` This is a follow-up to #24275, which fixed the same class of panic in `collect_partitioned`. ## What changes are included in this PR? This PR adds a single-threaded path to `CoalescePartitionsExec::execute`: when the session's `target_partitions` equals `1`, the operator drains its input partitions sequentially instead of spawning a task per partition. Behavior when `target_partitions > 1` is unchanged. Since the native default is `num_cpus::get()`, only callers who have explicitly opted out of parallelism take the new path. ## Are these changes tested? Yes. - Adds a `test_union_all` regression in `datafusion/wasmtest` that runs under both `wasm-pack test --chrome` and `tokio::test`. - Confirmed the reproducer panics on unpatched main and passes with the patch applied. - `cargo fmt --check` and `cargo clippy -- -D warnings` both run clean. ## Are there any user-facing changes? There are no API changes. On `wasm32-unknown-unknown`, multi-partition queries no longer panic when the session has `target_partitions = 1`. Native callers running with the default `target_partitions > 1` are unaffected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
