Should be green now and all comments were addressed. Sameer -> it is not **much** in 2 CPU worker scenarios, but it will matter when more parallelism is involved (when the work is complete with self hosted runners).
But there are no easy answers Currently the E2E test time is heavily impacted by queue of ASF projects - which is why self-hosted runners might bypass ASF-wide "stress` coming from AI BTW. Like Kaxil before - anyone is welcome to implement similar improvements :) J. On Mon, Jun 15, 2026 at 5:15 PM Sameer Mesiah <[email protected]> wrote: > Hi Jarek, > > This seems like a very good performance improvement. > > It would be interesting to see how it translates in terms of average E2E > runtime for the full CI workflow (honestly CI run times are currently a > significant damper on contributor experience). Looking forward to less > waiting for CI to turn green (or fail). > > Thanks, > Sameer. > > On Sun, 14 Jun 2026 at 11:27, Jarek Potiuk <[email protected]> wrote: > > > Hello here, > > > > Another day - another performance improvement in tests. > > > > I am helping Shahar to test and optimise the self-hosted runner > > experience to make us independent of the public GitHub Runner queue. > > > > Since test parallelism of our tests is **crucial** for faster and > > cheaper - I finally tackled the problem of "standard" provider DB > > tests taking by far longest "serialized" time (combination of the > > tests being "DB tests" and creating/removing a lot of venvs in the > > process. > > > > https://github.com/apache/airflow/pull/68533 - looks small but I > > **hope** it should solve the problem permanently and improve test > > speed even without self-hosted runners. > > > > It changes all PythonVenv/ExternalEnv tests to be "non-DB tests," > > meaning they no longer have to run serially. Consequently, we avoid > > artificially splitting the "standard" provider into smaller > > "sub-tests," as all these tests will now run in parallel using Xdist. > > This means parallelism scales automatically and naturally with size of > > the runner we will run it on. > > > > J. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
