kentkwu opened a new issue, #24886:
URL: https://github.com/apache/datafusion/issues/24886
### Describe the bug
DataFusion compiled to `wasm32-unknown-unknown` panics on `UNION ALL` and
any other multi-partition query whose collect funnels through
`CoalescePartitionsExec` (aggregations over partitioned scans, sorts across
partitions, etc.).
`CoalescePartitionsExec::execute` spawns one task per input partition via
`JoinSet::spawn` on the `_ =>` (multi-input) branch. `JoinSet::spawn` needs a
tokio reactor that isn't installed under `wasm-bindgen-futures`. The panic:
```
there is no reactor running, must be called from the context of a Tokio 1.x
runtime
```
The single-input branch of `CoalescePartitionsExec::execute` works — it
passes the child stream through directly. Only the multi-input branch spawns.
Follow-up to #24275, which fixed the same class of panic at a different call
site (`collect_partitioned`).
### To Reproduce
Add this to `datafusion/wasmtest/src/lib.rs`:
```rust
#[wasm_bindgen_test(unsupported = tokio::test)]
async fn test_union_all() {
let ctx = get_ctx();
ctx.sql("SELECT 1 AS n UNION ALL SELECT 2 AS n")
.await.unwrap()
.collect().await.unwrap();
}
```
Run from the repo root:
```sh
RUSTFLAGS='--cfg getrandom_backend="wasm_js"' \
wasm-pack test --headless --chrome datafusion/wasmtest
```
The test panics:
```
panicked at datafusion/common-runtime/src/join_set.rs:69:20:
there is no reactor running, must be called from the context of a Tokio 1.x
runtime
```
### Expected behavior
Multi-partition collect should succeed under `wasm-bindgen-futures` when the
session has asked for single-threaded execution (`target_partitions == 1`).
Sequential drain of the input partitions is a valid interleaving —
`CoalescePartitionsExec`'s own docstring states *"No guarantees are made about
the order of the resulting partition."*
### Additional context
Related: #24274, #24275, #13815
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
