Just let me add a bit more context as I see it. The AIP-47 design (following the AIP-4 idea) is targeted to handle those things:
* streamline the execution of the System Tests (which are basically executable "example_dags" of ours which we already have in our documentation for providers) * make them integrated and automatically execute in a "cloud" environment - with real cloud services behind * make use of the 100s of System Tests already written in Google Provider (and many others which we have in in other providers - for example AWS should be close to follow) * test automation of those using Google Cloud Platform donated credits (from what I understand Google is willing to sponsor credits to run those tests regularly). * I am quite sure when it works also AWS will donate credits for our CI that we could use for that. * and we should be able to extend it to other providers as well once the "CI environment" is in place. When this one is implemented, we should be finally able not only to run "unit" tests with all PRs for a lot of provider code, but also we will be able to regularly (not with all PR but likely daily) run the example dags with the real services and make sure that at least big part of our Providers code (Hooks/Operators) are still "working as expected" - at least that they pass "smoke tests". Currently we rely mostly on the manual tests done when we release providers (and our community members are actually great here - as every month we have a great deal of help from within the community), but when we have system tests automation, the confidence of releasing new versions of providers will go up significantly. I am really looking forward to discussion on this one and implementation and deployment for a number of providers. j. On Tue, Jan 25, 2022 at 2:13 PM Mateusz Nojek <[email protected]> wrote: > Hello everyone, > > I created a new AIP draft titled AIP-47 New design of Airflow System Tests > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-47+New+design+of+Airflow+System+Tests> > . > It is related to the new design of system tests that improves how they are > written and run which should contribute to higher quality of tests and thus > to lower quantity of bugs related to the operators. The AIP-47 is located > under AIP-4 > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-4+Support+for+Automation+of+System+Tests+for+external+systems> > as its direct successor (it more deeply describes some of the concepts). > The basic target was to simplify tests execution and enable running them > more regularly. > > I was working on it with Eugene Kostieiev and Bartłomiej Hirsz for some > time and soon we are also going to publish a PR with changes based on the > content of this AIP. > > For now, we would like to start a discussion with the developers and > contributors of the community of Apache Airflow. The idea was already > discussed with Jarek Potiuk, the original author of AIP-4, who cooperated > with us on some of the design details. > > Please, tell me what you think about it and share your ideas, concerns and > comments. > > Kind regards, > Mateusz Nojek >
