> I have one question about the k8s tests in the future: should they live
under airflow-core, under task-sdk, or in a separate dedicated location?
This is an excellent question. And as surprising as it might be I think
there should be in .... chart
chart
tests
unit <- current "helm_tests"
e2e <- current "k8s_tests"
On Mon, Nov 17, 2025 at 7:09 PM Zhe-You Liu <[email protected]> wrote:
> Hi Jarek,
>
> Thanks for proposing the new e2e test structure, the task-sdk e2e setup
> looks great to me!
> I have one question about the k8s tests in the future: should they live
> under airflow-core, under task-sdk, or in a separate dedicated location?
>
> Best regards,
> Jason
>
>
> On Mon, Nov 17, 2025 at 11:56 PM Jarek Potiuk <[email protected]> wrote:
>
> > Hello here,
> >
> > *TL;DR; We want to have better organized, and much improved new `e2e`
> test
> > type, a common thing in Airflow - one that others will be able to add
> tests
> > to and add their own similar tests following the blueprint.*
> >
> > Over the last few weeks quite a few of us were working on something we
> have
> > not fully realise we will end up and it was mostly bottom -up grass root
> > ideas that simply "clicked" together and we would like to propose adding
> > (or more formalising) a new type of tests in airflow - e2e tests. The
> > people who worked on it were mostly Amogh, Zhe You lIu, Bugra, myself.
> >
> > Wa already have a few attempts to do it (in various stages - as top-level
> > folders: *docker-tests, k8s-tests, airflow-e2e-tests, airflow-ctl-tests,
> > task-sdk-integration-tests* - yes the lack of consistency in names is
> very
> > confusing) but with recent improvements in task-sdk-integration-tests -
> we
> > think we have a good proposal on how we can implement those tests and
> > standardise it across airflow distributions.
> >
> > Current (best) approach is
> > https://github.com/apache/airflow/tree/main/task-sdk-integration-tests
> >
> > And it has the following properties:
> >
> > 1) uses uv pytest (standard pytest tests), docker-compose and
> > python-on-whales together, also has nice breeze wrapper for CI
> >
> > 2) automatically sets-up (and keeps running during iteration)
> > docker-compose based airflow deployment - with components (any -
> > api-server, dag processor, scheduler, triggerer) necessary during the
> tests
> > and allows to interact with it, including triggering local dags and the
> > like
> >
> > 3) pytest tests are run in a local venv controlled by uv sync
> >
> > 4) it allows for extremely fast (almost like pytest-native in venv)
> > iteration speed with tests - it uses Zhe's hot-reload functionality
> added a
> > week ago in order to reload components inside the docker-containers, and
> > local sources are mounted to those - which means that just saving a file
> in
> > your IDE or local env will make automated (sub-second, really) reload of
> > the components you interact with. Basically at the time you press
> Shift-F10
> > (or whatever shortcut you have) to re-run your tests, your just
> auto-saved
> > modified Airflow code already runs in those docker-compose components (!)
> > - *this is the most important feature of it.*
> >
> > 5) All those tests are automatically run in CI - in relevant PRs (driven
> by
> > selective checks) - but also they are very easily runnable locally in
> local
> > venv (no breeze CI image needed). Basically:
> >
> > cd task-sdk-integration-tests
> > uv run pytest
> >
> > 6) Last minute addition - we've been literally asked by Kacper an hour
> ago
> > about having "something like that" for open lineage - every current
> > distribution is able to have their own set of tests like that if the
> > stewards of that part want it
> >
> > Enough bragging....
> >
> > *Our proposal:*
> >
> > * we consistently rename those tests as `*e2e*` tests
> >
> > * we adapt all of them (including *k8s* tests eventually - where we have
> to
> > hack kind a bit) - to follow the same patterns and principles
> >
> > * we extract common code for that to `*devel-common*` and reuse across
> > those tests
> >
> > * we get rid of all the top-level `*SMTH-test*` folders we have now and
> > move those different kinds of tests to `*tests/e2e*` (next to `unit`,
> > `system`, `integration` we already have) of distributions that the tests
> > are relevant to
> >
> > For example - instead of:
> >
> > task-sdk
> > src
> > tests
> > task_sdk
> > some_unit_test.py
> > task-sdk-integration-tests
> > dags <- here e2e test dags are stored
> > logs <- here logs from all components are kept (.gitignored)
> > tests
> > task_sdk_tests
> > some_end_2_end_test.py
> >
> > We propose:
> >
> >
> > task-sdk
> > src
> > tests
> > unit
> > task_sdk
> > some_unit_test.py
> > e2e
> > dags
> > logs
> > task_sdk
> > some_end_2_end_test.py
> >
> > Of course - this is a proposal - and if there are other ideas on how to
> > restructure the test folders, that might be a good idea to do it now.
> *Only
> > serious and complete offers are going to be considered :)*, so I propose
> > constructive, complete proposals rather than criticising the current one.
> >
> > *Let us know what you think,*
> >
> > Jarek, Zhe, Bugra, Amogh
> >
>
