amitvijapur commented on issue #10952: URL: https://github.com/apache/arrow-rs/issues/10952#issuecomment-5782835315
I measured what `cargo test --benches` would cost, since the answer shapes where it goes. All numbers are from an M-series laptop, so a runner is probably 1.5 to 2x slower. Running every bench once per crate, debug profile, with the same features the `Test` jobs use: - arrow-arith, arrow-array, arrow-avro, arrow-buffer, arrow-cast, arrow-ipc, arrow-json, arrow-schema, arrow-flight, parquet-variant, parquet-variant-compute: all pass, seconds each. - `arrow` with `test_utils,csv,json` (everything its 43 benches require, and it keeps `pyarrow` out): all 43 pass after #11175, but one pass is about 8 minutes, and two benches are 5.5 of them: `comparison_kernels` (168 s) and `coalesce_kernels` (161 s). - `parquet --all-features`: `arrow_writer` alone exceeds 3 minutes and `row_selection_cursor` takes 2; the other 20 total under 3. `arrow_reader_clickbench` tries to `wget` the 1.5 GB `hits_1.parquet` when it is not on disk; I will open a small PR so it skips under `cargo test` instead of downloading. - The arrow crate set again in `--release`: about 3 minutes to compile the benches, about a minute to run them all, all pass. Two things worth knowing before writing the job. `--workspace --all-features` is not usable here: feature unification switches on `arrow/pyarrow`, and every bench binary then links against libpython. And `cargo clippy --all-targets` already type-checks the benches, so the gap is only executing them. So the choice is roughly: a debug pass catches things like the shift overflow in #11175 but is probably 15 minutes on a runner, mostly runtime; a release pass is closer to 10, mostly compile, and proves the benches execute. Either way it looks better as its own job in `arrow.yml` and `parquet.yml`, running alongside `Test` (6 minutes today), than as steps inside it. I have a draft with explicit per-crate commands and can open it for whichever shape you prefer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
