I also think that having a good unit test coverage + e2e tests would really help maintaining quality of releases and prevent regressions early, making all that much more manageable.
My two cents regarding the questions above: - Should we prioritize expanding unit test coverage first, or work on unit and E2E tests in parallel? Both in parallel sounds good to me. They do not depend on each other (At least in my mind) - For E2E testing, is Playwright a good choice, or are there other preferred tools we should explore? I like playwright and I feel this is the modern solution to go to. - If Playwright, should we use TypeScript or Python as language for tests? Good arguments mentioned above for both sides. I personally think (maybe wrongly), that ideally people maintaining / developing e2e tests will be people with front-end knowledge. (events, identifiers, selectors, css styling, browser local states, etc..), so I tend to think that those people will most likely come from the front-end community and having that in typescript would probably help them get started. On Wed, Sep 24, 2025 at 10:01 AM Amogh Desai <[email protected]> wrote: > Thanks for starting this discussion, Rahul. > > I am aligned with your thoughts, we have seen many many UI issues lately > which I believe we should prioritize fixing or at least reducing going > forward. > > To answer your question(s) -- > > * I think the unit tests and e2e / user flow tests as orthogonal, so they > can be > developed in parallel. > > * While I do not have real experience with Playwright but from what I hear > / read on reddit, > github, and related forums, it looks like Playwright can do similar job as > alternatives such as > Selenium, Cypress, while being more beginner friendly, which could play > better in our favour. > > * Again, not very strong on this but I feel that having Typescript as the > language of choice would limit > the number of contributors we get to help on this. Python would be my > weapon of choice, given > the "testing" is not limited by it and it potentially could encourage > broader participation. > > (Folks active on CI/CD(atleast me), are not very comfortable with > Typescript tests compared to > Python) > > One additional point: It would be valuable if we extract the e2e / user > flow testing into a GH job that > a release manager could run as "sort" of last validation before releasing > to avoid any last > minute issues. > > Thanks & Regards, > Amogh Desai > > > On Tue, Sep 23, 2025 at 9:15 PM Brent Bovenzi via dev < > [email protected]> wrote: > > > I agree that we need to improve UI tests. Happy to make that a priority > > during 3.1.x and 3.2 development. > > > > Personally, I will start with expanding unit tests first. But that > doesn't > > mean others can't get started on E2E tests. > > > > I lean towards using typescript for playwright. With E2E we are testing > the > > UI more than we are testing the API so I think the code should sit closer > > to the UI code. > > > > On Tue, Sep 23, 2025 at 9:38 AM Vincent Beck <[email protected]> > wrote: > > > > > Not answering any of these questions (sorry) but strong +1 on adding > > > end-to-end (E2E) tests. This will significantly make our UI more robust > > but > > > also will speed up some of our work. An example is front-end dependency > > > upgrades by dependabot. Today we have no way of knowing whether these > PRs > > > break the UI, in order to merge these PRs we need to compile the > > front-end > > > assets and test manually whether the UI work. Having end-to-end tests > on > > > the front-end and making them run in the CI would speed up this very > > > example (on top of increasing code robustness, increase our confidence > > etc). > > > > > > On 2025/09/23 11:03:40 Rahul Vats wrote: > > > > Hi all, > > > > > > > > With Airflow 3.0, the UI has undergone a complete revamp, and new > > changes > > > > are naturally more prone to regressions. We've already seen more UI > > > > regressions that are hard to catch with our current limited test > > > coverage. > > > > Manual testing is also challenging given the wide variety of > workflows > > > and > > > > screens in the UI. > > > > > > > > Currently, the Airflow UI has a few unit tests using Vitest and React > > > > Testing Library, but coverage is quite limited — only about 4 test > > files > > > > covering major pages like DagsList and TaskInstance. Many critical > > > > pages (Connections, > > > > Variables, Pools, Dashboard, etc.) have no test coverage at all. > > > > > > > > I’m exploring a two-pronged approach to improve UI testing: > > > > > > > > 1. > > > > > > > > Expand unit test coverage for individual components and pages > > > > > > > > 2. > > > > > > > > Add end-to-end (E2E) tests using Playwright for complete user > > > workflows > > > > > > > > > > > > This would give us both component-level safety and full user-journey > > > > testing across browsers, helping catch regressions early. > > > > > > > > A few questions: > > > > > > > > - > > > > > > > > Should we prioritize expanding unit test coverage first, or work > on > > > unit > > > > and E2E tests in parallel? > > > > > > > > - > > > > > > > > For E2E testing, is Playwright a good choice, or are there other > > > > preferred tools we should explore? > > > > > > > > - > > > > > > > > If Playwright, should we use TypeScript or Python as language for > > > tests? > > > > > > > > - > > > > > > > > Any other suggestions to strengthen our UI testing strategy? > > > > > > > > > > > > I’d love to hear your thoughts on whether this would be valuable for > > the > > > > project before diving into implementation details. > > > > > > > > Regards, > > > > Rahul Vats > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > >
