Hello, we had some experiments last year about using Airflow to run tests directly from the CI, it was good but at the end we had no way to complete the solution, also it was a bit too much. The good thing of that is that: - you can create dependencies between testsets - you can trigger via CI (Jenkins) at given moments - you can also automate the deployment with airflow. - you can embed tools, other party tools for testing like for example newman (postman API scripting) for the API or ui selenium. Just 1c. Giorgio
Il giorno mar 25 gen 2022 alle ore 17:58 Jarek Potiuk <[email protected]> ha scritto: > Just let me add a bit more context as I see it. > > The AIP-47 design (following the AIP-4 idea) is targeted to handle those > things: > > * streamline the execution of the System Tests (which are basically > executable "example_dags" of ours which we already have in our > documentation for providers) > * make them integrated and automatically execute in a "cloud" environment > - with real cloud services behind > * make use of the 100s of System Tests already written in Google Provider > (and many others which we have in in other providers - for example AWS > should be close to follow) > * test automation of those using Google Cloud Platform donated credits > (from what I understand Google is willing to sponsor credits to run those > tests regularly). > * I am quite sure when it works also AWS will donate credits for our CI > that we could use for that. > * and we should be able to extend it to other providers as well once the > "CI environment" is in place. > > When this one is implemented, we should be finally able not only to run > "unit" tests with all PRs for a lot of provider code, but also we will be > able to regularly (not with all PR but likely daily) run the example dags > with the real services and make sure that at least big part of our > Providers code (Hooks/Operators) are still "working as expected" - at least > that they pass "smoke tests". > > Currently we rely mostly on the manual tests done when we release > providers (and our community members are actually great here - as every > month we have a great deal of help from within the community), but when we > have system tests automation, the confidence of releasing new versions of > providers will go up significantly. > > I am really looking forward to discussion on this one and implementation > and deployment for a number of providers. > > j. > > > On Tue, Jan 25, 2022 at 2:13 PM Mateusz Nojek <[email protected]> wrote: > >> Hello everyone, >> >> I created a new AIP draft titled AIP-47 New design of Airflow System >> Tests >> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-47+New+design+of+Airflow+System+Tests> >> . >> It is related to the new design of system tests that improves how they >> are written and run which should contribute to higher quality of tests and >> thus to lower quantity of bugs related to the operators. The AIP-47 is >> located under AIP-4 >> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-4+Support+for+Automation+of+System+Tests+for+external+systems> >> as its direct successor (it more deeply describes some of the concepts). >> The basic target was to simplify tests execution and enable running them >> more regularly. >> >> I was working on it with Eugene Kostieiev and Bartłomiej Hirsz for some >> time and soon we are also going to publish a PR with changes based on the >> content of this AIP. >> >> For now, we would like to start a discussion with the developers and >> contributors of the community of Apache Airflow. The idea was already >> discussed with Jarek Potiuk, the original author of AIP-4, who cooperated >> with us on some of the design details. >> >> Please, tell me what you think about it and share your ideas, concerns >> and comments. >> >> Kind regards, >> Mateusz Nojek >> > -- Life is a chess game - Anonymous.
