Valid points. I was just expressing concern from a CI/CD failure perspective but in no way was I objecting to usage of typescript.
Anything that works for the team that owns most of the tests works by me! Thanks & Regards, Amogh Desai On Thu, Sep 25, 2025 at 1:39 AM Jens Scheffler <[email protected]> wrote: > Echoing a full agreement to all statement before. > > I also would emphasize the test pyramid. As we really have a low > coverage we should maybe also have a metric. I would also propose that > we focus more on testing coverage == hopefully also improved quality > preventing regression in 3.2. One option could a bit of force to force > all new changes applied to UI to request a unit test being added (all > changes TS/x files need to have a corresponding .test.ts/x file) except > for cases where it does not make sense we allow exclusion of the rule by > adding a label by maintainers. (just want to drop the idea, might be > many will hate it) > > For language also have no strong preference but even though I am a TS > noob I see that testing UI is closer to UI development, to using Python > for TS UI testing might make a knowledge gap also for potential utility > code. So as most voices would propose TS (maybe then finally I get to > understand it) > > Jens > > On 24.09.25 18:33, Jarek Potiuk wrote: > > Small poking at important things from holidays while having a coffee at a > > place with internet :) > > > > +1 on having both. I have no strong opinion on the language and > Playwright. > > > > But there are few things for consideration: > > > > We should have many, many more unit tests than e2e tests. This is the > > classic test pyramid case: The Practical Test Pyramid > > https://share.google/YHVIt8EBptT1yYFur > > > > The e2e tests are slow, brittle and difficult to maintain and we should > > have rather small number of those - mostly green path on all major > > functionalities. Possibly with some cross browser checks - but those are > > very difficult to get right. > > > > It will be a pain to maintain those. Like our k8s e2e tests - when they > > break (rarely) people will have hard time to create and run the tests so > > the tooling around that has to drive people who will be reproducing the > > failures by hand - how to set it up and how to reproduce it and how to > > iterate as fast as possible. > > > > Keeping the tests stable and side-effect free will be a challenge - so > > likely a good 'reset the environment' before each test is a must. Also > they > > need to be run with all UI changes in CI which will make UI PR run > longer. > > It's an imperative to get them run not only in canary builds but in all > PRs > > for ui and API and likely whenever any core model change. It will slow > down > > UI development and create frustration for contributors unfamiliar with > the > > test framework so we need to have at least a few people proficient in > > intricacies of the test framework dedicated and willing to help those who > > will struggle with issues. > > > > Language is secondary - e2e tests will require all stack - Python > (backend) > > and Typescript and react experiences anyway. Most likely people who write > > UI should be involved so Typescript is fine, and also even those who do > not > > know TypeScript will be able to fix some obvious cases. > > > > J. > > > > śr., 24 wrz 2025, 09:50 użytkownik Shubham Raj <[email protected]> > > napisał: > > > >> This addition will be really nice. > >> Also, at this point of 3.x, I think UI has been relatively stable and > >> writing e2e would be a good starting point. Unit tests can go parallel. > >> It's right that people maintaining CI/CD would be first persons to see > any > >> breakage, but I feel writing in ts will be easier for folks working on > >> frontend, and also for the new contributors who would like to > contribute in > >> this UI e2e tests coming from cypress or similar experience, since > majority > >> of other choices do come with ts/js support. > >> I would be happy to help with the e2e tests. We should probably go in > >> phases with P0/1/2. > >> > >> Thanks. > >> > >> > >> On Wed, Sep 24, 2025 at 9:01 PM Pierre Jeambrun <[email protected]> > >> wrote: > >> > >>> I also think that having a good unit test coverage + e2e tests would > >> really > >>> help maintaining quality of releases and prevent regressions early, > >> making > >>> all that much more manageable. > >>> > >>> My two cents regarding the questions above: > >>> > >>> - > >>> Should we prioritize expanding unit test coverage first, or work on > >> unit > >>> and E2E tests in parallel? > >>> > >>> Both in parallel sounds good to me. They do not depend on each > other > >> (At > >>> least in my mind) > >>> - > >>> For E2E testing, is Playwright a good choice, or are there other > >>> preferred tools we should explore? > >>> > >>> I like playwright and I feel this is the modern solution to go to. > >>> - > >>> If Playwright, should we use TypeScript or Python as language for > >> tests? > >>> Good arguments mentioned above for both sides. I personally think > >> (maybe > >>> wrongly), that ideally people maintaining / developing e2e tests will > be > >>> people with > >>> front-end knowledge. (events, identifiers, selectors, css styling, > >>> browser local states, etc..), so I tend to think that those people > will > >>> most likely come from the front-end community and > >>> having that in typescript would probably help them get started. > >>> > >>> > >>> On Wed, Sep 24, 2025 at 10:01 AM Amogh Desai <[email protected]> > >>> wrote: > >>> > >>>> Thanks for starting this discussion, Rahul. > >>>> > >>>> I am aligned with your thoughts, we have seen many many UI issues > >> lately > >>>> which I believe we should prioritize fixing or at least reducing going > >>>> forward. > >>>> > >>>> To answer your question(s) -- > >>>> > >>>> * I think the unit tests and e2e / user flow tests as orthogonal, so > >> they > >>>> can be > >>>> developed in parallel. > >>>> > >>>> * While I do not have real experience with Playwright but from what I > >>> hear > >>>> / read on reddit, > >>>> github, and related forums, it looks like Playwright can do similar > job > >>> as > >>>> alternatives such as > >>>> Selenium, Cypress, while being more beginner friendly, which could > play > >>>> better in our favour. > >>>> > >>>> * Again, not very strong on this but I feel that having Typescript as > >> the > >>>> language of choice would limit > >>>> the number of contributors we get to help on this. Python would be my > >>>> weapon of choice, given > >>>> the "testing" is not limited by it and it potentially could encourage > >>>> broader participation. > >>>> > >>>> (Folks active on CI/CD(atleast me), are not very comfortable with > >>>> Typescript tests compared to > >>>> Python) > >>>> > >>>> One additional point: It would be valuable if we extract the e2e / > user > >>>> flow testing into a GH job that > >>>> a release manager could run as "sort" of last validation before > >> releasing > >>>> to avoid any last > >>>> minute issues. > >>>> > >>>> Thanks & Regards, > >>>> Amogh Desai > >>>> > >>>> > >>>> On Tue, Sep 23, 2025 at 9:15 PM Brent Bovenzi via dev < > >>>> [email protected]> wrote: > >>>> > >>>>> I agree that we need to improve UI tests. Happy to make that a > >> priority > >>>>> during 3.1.x and 3.2 development. > >>>>> > >>>>> Personally, I will start with expanding unit tests first. But that > >>>> doesn't > >>>>> mean others can't get started on E2E tests. > >>>>> > >>>>> I lean towards using typescript for playwright. With E2E we are > >> testing > >>>> the > >>>>> UI more than we are testing the API so I think the code should sit > >>> closer > >>>>> to the UI code. > >>>>> > >>>>> On Tue, Sep 23, 2025 at 9:38 AM Vincent Beck <[email protected]> > >>>> wrote: > >>>>>> Not answering any of these questions (sorry) but strong +1 on > >> adding > >>>>>> end-to-end (E2E) tests. This will significantly make our UI more > >>> robust > >>>>> but > >>>>>> also will speed up some of our work. An example is front-end > >>> dependency > >>>>>> upgrades by dependabot. Today we have no way of knowing whether > >> these > >>>> PRs > >>>>>> break the UI, in order to merge these PRs we need to compile the > >>>>> front-end > >>>>>> assets and test manually whether the UI work. Having end-to-end > >> tests > >>>> on > >>>>>> the front-end and making them run in the CI would speed up this > >> very > >>>>>> example (on top of increasing code robustness, increase our > >>> confidence > >>>>> etc). > >>>>>> On 2025/09/23 11:03:40 Rahul Vats wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> With Airflow 3.0, the UI has undergone a complete revamp, and new > >>>>> changes > >>>>>>> are naturally more prone to regressions. We've already seen more > >> UI > >>>>>>> regressions that are hard to catch with our current limited test > >>>>>> coverage. > >>>>>>> Manual testing is also challenging given the wide variety of > >>>> workflows > >>>>>> and > >>>>>>> screens in the UI. > >>>>>>> > >>>>>>> Currently, the Airflow UI has a few unit tests using Vitest and > >>> React > >>>>>>> Testing Library, but coverage is quite limited — only about 4 > >> test > >>>>> files > >>>>>>> covering major pages like DagsList and TaskInstance. Many > >> critical > >>>>>>> pages (Connections, > >>>>>>> Variables, Pools, Dashboard, etc.) have no test coverage at all. > >>>>>>> > >>>>>>> I’m exploring a two-pronged approach to improve UI testing: > >>>>>>> > >>>>>>> 1. > >>>>>>> > >>>>>>> Expand unit test coverage for individual components and pages > >>>>>>> > >>>>>>> 2. > >>>>>>> > >>>>>>> Add end-to-end (E2E) tests using Playwright for complete user > >>>>>> workflows > >>>>>>> > >>>>>>> This would give us both component-level safety and full > >>> user-journey > >>>>>>> testing across browsers, helping catch regressions early. > >>>>>>> > >>>>>>> A few questions: > >>>>>>> > >>>>>>> - > >>>>>>> > >>>>>>> Should we prioritize expanding unit test coverage first, or > >> work > >>>> on > >>>>>> unit > >>>>>>> and E2E tests in parallel? > >>>>>>> > >>>>>>> - > >>>>>>> > >>>>>>> For E2E testing, is Playwright a good choice, or are there > >> other > >>>>>>> preferred tools we should explore? > >>>>>>> > >>>>>>> - > >>>>>>> > >>>>>>> If Playwright, should we use TypeScript or Python as language > >>> for > >>>>>> tests? > >>>>>>> - > >>>>>>> > >>>>>>> Any other suggestions to strengthen our UI testing strategy? > >>>>>>> > >>>>>>> > >>>>>>> I’d love to hear your thoughts on whether this would be valuable > >>> for > >>>>> the > >>>>>>> project before diving into implementation details. > >>>>>>> > >>>>>>> Regards, > >>>>>>> Rahul Vats > >>>>>>> > >>>>>> > >> --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: [email protected] > >>>>>> For additional commands, e-mail: [email protected] > >>>>>> > >>>>>> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
