Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

Hello Mateuz, it's ok for me. Good. BR, Giorgio Il giorno mar 15 feb 2022 alle ore 10:51 Mateusz Nojek ha scritto: > Hello again, > > I agree with your (Niko, Giorgio) points that the environment needs to be > set up as easy as possible, but I will leave it to the providers, as > mentioned befor

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

For me that one looks good. On Tue, Feb 15, 2022 at 10:51 AM Mateusz Nojek wrote: > > Hello again, > > I agree with your (Niko, Giorgio) points that the environment needs to be set > up as easy as possible, but I will leave it to the providers, as mentioned > before. It's also in their interest

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

Hello again, I agree with your (Niko, Giorgio) points that the environment needs to be set up as easy as possible, but I will leave it to the providers, as mentioned before. It's also in their interest to simplify it. Regarding the "watcher" task. It's been discussed in another topic on the devli

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

> > > I think both these sticking points are really a trade-off of simplicity vs > consistency/reliability. And to be clear I'm not arguing for things to be > more complex just for the heck of it, I agree that simplicity is great! But > just that there needs to be a balance and we can't get caught

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

On Wed, Feb 9, 2022 at 10:35 PM Giorgio Zoppi wrote: > Hello, > we had some experiments last year about using Airflow to run tests > directly from the CI, it was good but at the end we had no way to complete > the solution, also > it was a bit too much. > Yeah. Having "full" airflow to run the t

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

Hello, we had some experiments last year about using Airflow to run tests directly from the CI, it was good but at the end we had no way to complete the solution, also it was a bit too much. The good thing of that is that: - you can create dependencies between testsets - you can trigger via CI (Jen

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

v@airflow.apache.org Subject: RE: [EXTERNAL] [DISCUSSION] AIP-47 New design of Airflow System Tests CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. I think Mateusz explained the

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

Thanks, Jarek, for your email. I think this completes the reasoning behind this AIP. However, I recently discovered the flaw in the design that was omitted. The problem is related to the state of the test (DAG) and the trigger rules. When the teardown task is used (e.g. to clean the resources afte

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

I think Mateusz explained the points very well. I have just a few comments to some of the points. > 3. In general the AIP reads as if it's solved this problem, but it's more like it has absolved itself from solving this problem, which is much different. I think this approach could possibly make th

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

need test infrastructure that's easier to use and > will also enforce these best practices/requirements which are needed for > the tests to run. > In general, it reads much more like a guideline on best practices rather > than a new and improved system test engine. > > &g

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

and improved system test engine. Thanks for taking the time to create this AIP I'm very eager to get system testing up and running in Airflow and I'd love to collaborate further on it! Cheers, Niko From: Jarek Potiuk Sent: Tuesday, January 25, 2022

Re: [DISCUSSION] AIP-47 New design of Airflow System Tests

Just let me add a bit more context as I see it. The AIP-47 design (following the AIP-4 idea) is targeted to handle those things: * streamline the execution of the System Tests (which are basically executable "example_dags" of ours which we already have in our documentation for providers) * make t

[DISCUSSION] AIP-47 New design of Airflow System Tests

Hello everyone, I created a new AIP draft titled AIP-47 New design of Airflow System Tests . It is related to the new design of system tests that improves how they are written and run which should contr