jayzhan211 opened a new pull request, #26649: URL: https://github.com/apache/datafusion/pull/26649
## Which issue does this PR close? - None. CI-only change to cut extended test time. ## Rationale for this change The `extended_tests` job runs for every merge queue entry (778 times in the last 30 days, on 32-CPU runners), and most of what it runs is a second copy of the workspace test suite. The `cargo test (amd64)` job already runs that suite on the same commit in the merge queue. In a recent merge queue run (39907001762) the job spent 555s running tests: | | time | |---|---| | `tests/fuzz.rs` (only in this job) | 266s | | sqllogictests (also in `cargo test (amd64)`) | 103s | | physical-plan unit tests (also in `cargo test (amd64)`, except the 1000-iteration spill pool fuzz) | 56s | | everything else (also in `cargo test (amd64)`) | ~130s | ## What changes are included in this PR? The `extended` xtask step now runs only the tests that the `workspace` step does not: - the fuzz tests and `memory_limit_validation`, which are behind the `extended_tests` feature - `spill_pool_scenario_fuzz`, which this job runs with 1000 iterations It builds only `datafusion` and `datafusion-physical-plan` test targets and uses test name filters to skip their other tests. The job name and required check are unchanged. Nothing that ran before is lost: - `extended_tests` is only defined in `datafusion` and only used by `tests/fuzz.rs` and `tests/memory_limit/mod.rs`. - The `workspace` step runs every other test with a superset of the features. `recursive_protection` is a default feature, and `json` only applies to the proto crates, which the `workspace` step tests with `json`. - `ffi_example_table_provider`, which the `workspace` step excludes, has no tests. ### Tradeoff The step now lists the tests it runs by name. If a new test is put behind the `extended_tests` feature outside `fuzz_cases` or `memory_limit_validation`, it also needs a filter in this step, otherwise it never runs in CI. A comment next to the filters says this. ## What is the testing strategy for this PR? The xtask snapshot test is updated. Running `cargo xtask ci step test extended` locally runs exactly the expected 156 tests and filters out the rest: | binary | ran | filtered out | |---|---|---| | `datafusion` lib | 0 | 490 | | `core_integration` | 21 | 1453 | | `fuzz` | 134 | 0 | | `datafusion-physical-plan` lib | 1 | 2576 | Without the filters, the same targets run 4675 tests, and all pass. Based on the run above, this should cut the job's test time from about 555s to about 325s, plus a smaller build. That is roughly 4 minutes per merge queue run. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
