Thanks for the summary, Rahul. Sounds like a good plan, you could coordinate the work better either by using a Github Board or Slack channel or both too.
Thanks & Regards, Amogh Desai On Fri, Oct 10, 2025 at 9:11 PM Rahul Vats <[email protected]> wrote: > Hi Everyone, > > Thank you all for the fantastic feedback and strong support! The community > consensus is clear and encouraging: > > - High priority for improving UI testing > - Playwright + TypeScript for e2e testing > - Parallel development of unit and e2e tests > - Test pyramid approach with focused e2e scope > - Multiple volunteers ready to contribute! > > > *Addressing Key Concerns* > Important points on e2e complexity are well taken. I completely agree that > we should follow the test pyramid. > > - E2E tests should cover only critical workflows (~10-15 scenarios) > - Unit tests are the foundation and should be the primary focus and we > should have an enforcement so that every UI change comes with a unit test. > - CI integration needs careful planning to avoid slowing development. > > *Next Steps* > > - I plan to work on a POC for E2E tests and will raise a PR for review > soon. After that, I'll share a detailed implementation plan and coordinate > with volunteers who offered to help. > - In the meantime, I suggest we should continue expanding unit test > coverage as planned. > > > It was great meeting you all at the Airflow Summit. Momentum here is > fantastic, and I'm excited to work with everyone who offered to help make > Airflow 3 UI much more reliable! > > Regards, > Rahul Vats > > On Thu, 25 Sept 2025 at 07:49, Vikram Koka via dev <[email protected] > > > wrote: > > > Rahul, > > Thank you for starting this thread. > > > > Strong +1 from me as well. > > I not only agree with this, but I believe this needs to be a high > priority > > going forward and before 3.2. > > > > I also agree with the comments on the "test pyramid" approach, of there > > needing to be FAR more unit tests as compared to end to end tests. > > I also believe that a certain level of rigor around this i.e. some form > of > > enforcement to ensure that all new UI changes come with a unit test. > > > > I would also add the Core API changes (for UI) to the mix here, since a > > fair number of UI changes requires underlying API changes. > > > > Vikram > > > > > > On Thu, Sep 25, 2025 at 12:04 AM Amogh Desai <[email protected]> > > wrote: > > > > > Valid points. > > > > > > I was just expressing concern from a CI/CD failure perspective but in > no > > > way > > > was I objecting to usage of typescript. > > > > > > Anything that works for the team that owns most of the tests works by > me! > > > > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > > > > On Thu, Sep 25, 2025 at 1:39 AM Jens Scheffler <[email protected]> > > > wrote: > > > > > > > Echoing a full agreement to all statement before. > > > > > > > > I also would emphasize the test pyramid. As we really have a low > > > > coverage we should maybe also have a metric. I would also propose > that > > > > we focus more on testing coverage == hopefully also improved quality > > > > preventing regression in 3.2. One option could a bit of force to > force > > > > all new changes applied to UI to request a unit test being added (all > > > > changes TS/x files need to have a corresponding .test.ts/x file) > except > > > > for cases where it does not make sense we allow exclusion of the rule > > by > > > > adding a label by maintainers. (just want to drop the idea, might be > > > > many will hate it) > > > > > > > > For language also have no strong preference but even though I am a TS > > > > noob I see that testing UI is closer to UI development, to using > Python > > > > for TS UI testing might make a knowledge gap also for potential > utility > > > > code. So as most voices would propose TS (maybe then finally I get to > > > > understand it) > > > > > > > > Jens > > > > > > > > On 24.09.25 18:33, Jarek Potiuk wrote: > > > > > Small poking at important things from holidays while having a > coffee > > > at a > > > > > place with internet :) > > > > > > > > > > +1 on having both. I have no strong opinion on the language and > > > > Playwright. > > > > > > > > > > But there are few things for consideration: > > > > > > > > > > We should have many, many more unit tests than e2e tests. This is > the > > > > > classic test pyramid case: The Practical Test Pyramid > > > > > https://share.google/YHVIt8EBptT1yYFur > > > > > > > > > > The e2e tests are slow, brittle and difficult to maintain and we > > should > > > > > have rather small number of those - mostly green path on all major > > > > > functionalities. Possibly with some cross browser checks - but > those > > > are > > > > > very difficult to get right. > > > > > > > > > > It will be a pain to maintain those. Like our k8s e2e tests - when > > they > > > > > break (rarely) people will have hard time to create and run the > tests > > > so > > > > > the tooling around that has to drive people who will be > reproducing > > > the > > > > > failures by hand - how to set it up and how to reproduce it and how > > to > > > > > iterate as fast as possible. > > > > > > > > > > Keeping the tests stable and side-effect free will be a challenge - > > so > > > > > likely a good 'reset the environment' before each test is a must. > > Also > > > > they > > > > > need to be run with all UI changes in CI which will make UI PR run > > > > longer. > > > > > It's an imperative to get them run not only in canary builds but in > > all > > > > PRs > > > > > for ui and API and likely whenever any core model change. It will > > slow > > > > down > > > > > UI development and create frustration for contributors unfamiliar > > with > > > > the > > > > > test framework so we need to have at least a few people proficient > in > > > > > intricacies of the test framework dedicated and willing to help > those > > > who > > > > > will struggle with issues. > > > > > > > > > > Language is secondary - e2e tests will require all stack - Python > > > > (backend) > > > > > and Typescript and react experiences anyway. Most likely people who > > > write > > > > > UI should be involved so Typescript is fine, and also even those > who > > do > > > > not > > > > > know TypeScript will be able to fix some obvious cases. > > > > > > > > > > J. > > > > > > > > > > śr., 24 wrz 2025, 09:50 użytkownik Shubham Raj < > > [email protected]> > > > > > napisał: > > > > > > > > > >> This addition will be really nice. > > > > >> Also, at this point of 3.x, I think UI has been relatively stable > > and > > > > >> writing e2e would be a good starting point. Unit tests can go > > > parallel. > > > > >> It's right that people maintaining CI/CD would be first persons to > > see > > > > any > > > > >> breakage, but I feel writing in ts will be easier for folks > working > > on > > > > >> frontend, and also for the new contributors who would like to > > > > contribute in > > > > >> this UI e2e tests coming from cypress or similar experience, since > > > > majority > > > > >> of other choices do come with ts/js support. > > > > >> I would be happy to help with the e2e tests. We should probably go > > in > > > > >> phases with P0/1/2. > > > > >> > > > > >> Thanks. > > > > >> > > > > >> > > > > >> On Wed, Sep 24, 2025 at 9:01 PM Pierre Jeambrun < > > > [email protected]> > > > > >> wrote: > > > > >> > > > > >>> I also think that having a good unit test coverage + e2e tests > > would > > > > >> really > > > > >>> help maintaining quality of releases and prevent regressions > early, > > > > >> making > > > > >>> all that much more manageable. > > > > >>> > > > > >>> My two cents regarding the questions above: > > > > >>> > > > > >>> - > > > > >>> Should we prioritize expanding unit test coverage first, or > > work > > > on > > > > >> unit > > > > >>> and E2E tests in parallel? > > > > >>> > > > > >>> Both in parallel sounds good to me. They do not depend on > each > > > > other > > > > >> (At > > > > >>> least in my mind) > > > > >>> - > > > > >>> For E2E testing, is Playwright a good choice, or are there > > other > > > > >>> preferred tools we should explore? > > > > >>> > > > > >>> I like playwright and I feel this is the modern solution to > go > > > to. > > > > >>> - > > > > >>> If Playwright, should we use TypeScript or Python as language > > for > > > > >> tests? > > > > >>> Good arguments mentioned above for both sides. I personally > > think > > > > >> (maybe > > > > >>> wrongly), that ideally people maintaining / developing e2e tests > > will > > > > be > > > > >>> people with > > > > >>> front-end knowledge. (events, identifiers, selectors, css > > > styling, > > > > >>> browser local states, etc..), so I tend to think that those > people > > > > will > > > > >>> most likely come from the front-end community and > > > > >>> having that in typescript would probably help them get > started. > > > > >>> > > > > >>> > > > > >>> On Wed, Sep 24, 2025 at 10:01 AM Amogh Desai < > > [email protected]> > > > > >>> wrote: > > > > >>> > > > > >>>> Thanks for starting this discussion, Rahul. > > > > >>>> > > > > >>>> I am aligned with your thoughts, we have seen many many UI > issues > > > > >> lately > > > > >>>> which I believe we should prioritize fixing or at least reducing > > > going > > > > >>>> forward. > > > > >>>> > > > > >>>> To answer your question(s) -- > > > > >>>> > > > > >>>> * I think the unit tests and e2e / user flow tests as > orthogonal, > > so > > > > >> they > > > > >>>> can be > > > > >>>> developed in parallel. > > > > >>>> > > > > >>>> * While I do not have real experience with Playwright but from > > what > > > I > > > > >>> hear > > > > >>>> / read on reddit, > > > > >>>> github, and related forums, it looks like Playwright can do > > similar > > > > job > > > > >>> as > > > > >>>> alternatives such as > > > > >>>> Selenium, Cypress, while being more beginner friendly, which > could > > > > play > > > > >>>> better in our favour. > > > > >>>> > > > > >>>> * Again, not very strong on this but I feel that having > Typescript > > > as > > > > >> the > > > > >>>> language of choice would limit > > > > >>>> the number of contributors we get to help on this. Python would > be > > > my > > > > >>>> weapon of choice, given > > > > >>>> the "testing" is not limited by it and it potentially could > > > encourage > > > > >>>> broader participation. > > > > >>>> > > > > >>>> (Folks active on CI/CD(atleast me), are not very comfortable > with > > > > >>>> Typescript tests compared to > > > > >>>> Python) > > > > >>>> > > > > >>>> One additional point: It would be valuable if we extract the > e2e / > > > > user > > > > >>>> flow testing into a GH job that > > > > >>>> a release manager could run as "sort" of last validation before > > > > >> releasing > > > > >>>> to avoid any last > > > > >>>> minute issues. > > > > >>>> > > > > >>>> Thanks & Regards, > > > > >>>> Amogh Desai > > > > >>>> > > > > >>>> > > > > >>>> On Tue, Sep 23, 2025 at 9:15 PM Brent Bovenzi via dev < > > > > >>>> [email protected]> wrote: > > > > >>>> > > > > >>>>> I agree that we need to improve UI tests. Happy to make that a > > > > >> priority > > > > >>>>> during 3.1.x and 3.2 development. > > > > >>>>> > > > > >>>>> Personally, I will start with expanding unit tests first. But > > that > > > > >>>> doesn't > > > > >>>>> mean others can't get started on E2E tests. > > > > >>>>> > > > > >>>>> I lean towards using typescript for playwright. With E2E we are > > > > >> testing > > > > >>>> the > > > > >>>>> UI more than we are testing the API so I think the code should > > sit > > > > >>> closer > > > > >>>>> to the UI code. > > > > >>>>> > > > > >>>>> On Tue, Sep 23, 2025 at 9:38 AM Vincent Beck < > > [email protected]> > > > > >>>> wrote: > > > > >>>>>> Not answering any of these questions (sorry) but strong +1 on > > > > >> adding > > > > >>>>>> end-to-end (E2E) tests. This will significantly make our UI > more > > > > >>> robust > > > > >>>>> but > > > > >>>>>> also will speed up some of our work. An example is front-end > > > > >>> dependency > > > > >>>>>> upgrades by dependabot. Today we have no way of knowing > whether > > > > >> these > > > > >>>> PRs > > > > >>>>>> break the UI, in order to merge these PRs we need to compile > the > > > > >>>>> front-end > > > > >>>>>> assets and test manually whether the UI work. Having > end-to-end > > > > >> tests > > > > >>>> on > > > > >>>>>> the front-end and making them run in the CI would speed up > this > > > > >> very > > > > >>>>>> example (on top of increasing code robustness, increase our > > > > >>> confidence > > > > >>>>> etc). > > > > >>>>>> On 2025/09/23 11:03:40 Rahul Vats wrote: > > > > >>>>>>> Hi all, > > > > >>>>>>> > > > > >>>>>>> With Airflow 3.0, the UI has undergone a complete revamp, and > > new > > > > >>>>> changes > > > > >>>>>>> are naturally more prone to regressions. We've already seen > > more > > > > >> UI > > > > >>>>>>> regressions that are hard to catch with our current limited > > test > > > > >>>>>> coverage. > > > > >>>>>>> Manual testing is also challenging given the wide variety of > > > > >>>> workflows > > > > >>>>>> and > > > > >>>>>>> screens in the UI. > > > > >>>>>>> > > > > >>>>>>> Currently, the Airflow UI has a few unit tests using Vitest > and > > > > >>> React > > > > >>>>>>> Testing Library, but coverage is quite limited — only about 4 > > > > >> test > > > > >>>>> files > > > > >>>>>>> covering major pages like DagsList and TaskInstance. Many > > > > >> critical > > > > >>>>>>> pages (Connections, > > > > >>>>>>> Variables, Pools, Dashboard, etc.) have no test coverage at > > all. > > > > >>>>>>> > > > > >>>>>>> I’m exploring a two-pronged approach to improve UI testing: > > > > >>>>>>> > > > > >>>>>>> 1. > > > > >>>>>>> > > > > >>>>>>> Expand unit test coverage for individual components and > > pages > > > > >>>>>>> > > > > >>>>>>> 2. > > > > >>>>>>> > > > > >>>>>>> Add end-to-end (E2E) tests using Playwright for complete > > user > > > > >>>>>> workflows > > > > >>>>>>> > > > > >>>>>>> This would give us both component-level safety and full > > > > >>> user-journey > > > > >>>>>>> testing across browsers, helping catch regressions early. > > > > >>>>>>> > > > > >>>>>>> A few questions: > > > > >>>>>>> > > > > >>>>>>> - > > > > >>>>>>> > > > > >>>>>>> Should we prioritize expanding unit test coverage first, > or > > > > >> work > > > > >>>> on > > > > >>>>>> unit > > > > >>>>>>> and E2E tests in parallel? > > > > >>>>>>> > > > > >>>>>>> - > > > > >>>>>>> > > > > >>>>>>> For E2E testing, is Playwright a good choice, or are > there > > > > >> other > > > > >>>>>>> preferred tools we should explore? > > > > >>>>>>> > > > > >>>>>>> - > > > > >>>>>>> > > > > >>>>>>> If Playwright, should we use TypeScript or Python as > > language > > > > >>> for > > > > >>>>>> tests? > > > > >>>>>>> - > > > > >>>>>>> > > > > >>>>>>> Any other suggestions to strengthen our UI testing > > strategy? > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> I’d love to hear your thoughts on whether this would be > > valuable > > > > >>> for > > > > >>>>> the > > > > >>>>>>> project before diving into implementation details. > > > > >>>>>>> > > > > >>>>>>> Regards, > > > > >>>>>>> Rahul Vats > > > > >>>>>>> > > > > >>>>>> > > > > >> > > --------------------------------------------------------------------- > > > > >>>>>> To unsubscribe, e-mail: [email protected] > > > > >>>>>> For additional commands, e-mail: [email protected] > > > > >>>>>> > > > > >>>>>> > > > > >> > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > >
