> On Oct 29, 2026, at 2:53 PM, Ruifeng Zheng <[email protected]> wrote: > > A PR may still be tested against a state different from the one into which it > is ultimately merged, especially when several PRs merge concurrently. > Pre-merge CI against master also does not validate subsequent backports.
A merge queue for each active branch solves both of these problems. Every PR marked as ready-to-merge is tested against the latest state of the target branch and merged automatically on success. > Regarding test selection, would we be open to experimenting with AI-based > approaches? I have a POC in #57461 > <https://github.com/apache/spark/pull/57461> that uses the diff and > repository context to select likely affected test suites. Our existing > deterministic selection primarily operates at the module or job-group level; > it cannot select individual test suites at the same granularity. Once a > module is considered affected, a relatively broad set of tests may still run. I very much like the idea of improving our test selection to provide faster feedback. It improves the contributor experience, but it won’t save us build time if we cannot trust it enough to replace the current module-level test infrastructure. I would be interested in exploring whether for PySpark we can effectively deploy testmon <https://www.testmon.org/> to select just the tests that exercise whatever lines of code were touched in a given patch. Nick
