Re: [DISCUSS] AIP-4 System tests

2020-03-04 Thread Jarek Potiuk
> When these system tests would be run? With every PR? > I think we'll have to decide when we see how long the tests run. First I want to see if this is feasible and whether we can automate it. After we do it - we can decide how often we run it (when we see how long it takes and how stable they ar

Re: [DISCUSS] AIP-4 System tests

2020-03-04 Thread Michał Słowikowski
+1 from me to make more system tests. I do not have much experience with such tests but I am going to fix it, right away. :) When these system tests would be run? With every PR? I don't know if such tool will be helpful but I leave the link here. The description says: "This is a pytest plug-in whi

Re: [DISCUSS] AIP-4 System tests

2020-03-04 Thread Jarek Potiuk
Hello One small update. We are trying now with Bjorn Olsen to see how well the System Tests approach we did for Google Cloud Platform can be applied to AWS. This might be a good exercise to see if we can apply it to other services and make it part of releasing backport operators, fully automating

Re: [DISCUSS] AIP-4 System tests

2020-02-21 Thread Jarek Potiuk
Any more comments for system tests? I would love to vote on the AIP-4 and my current proposal would be : 1) Let's try to automate system test execution (starting with GCP as it is close to be ready). That would most likely be with Github Actions - details to be worked on. 2) We can do it to automa

Re: [DISCUSS] AIP-4 System tests

2020-02-15 Thread Jarek Potiuk
On Sat, Feb 15, 2020, 20:54 Ash Berlin-Taylor wrote: > Yeah, I'm for this. > > In fact I'm about to mark some of the Hive ones as system tests as they > require a running hive cluster. > I would be careful about which automatically marking unit tests that run > dags as system/integration though,

Re: [DISCUSS] AIP-4 System tests

2020-02-15 Thread Ash Berlin-Taylor
Yeah, I'm for this. In fact I'm about to mark some of the Hive ones as system tests as they require a running hive cluster. I would be careful about which automatically marking unit tests that run dags as system/integration though, a number of our unit tests rely on this to test the tasks in va

Re: [DISCUSS] AIP-4 System tests

2020-02-15 Thread Tomasz Urbaszek
+1 for introducing system tests. Lack of them is a big pain. I would like also to suggest to mark some actual tests (those running DAGs, etc) as system tests. Then we can simplify our units and probably speed up CI builds (not to mention the reduction of side effects). The approach used for GCP sy

[DISCUSS] AIP-4 System tests

2020-02-15 Thread Jarek Potiuk
TL;DR; I would like to revive a discussion (hopefully short :) and possibly cast a vote on "AIP-4 - Support for System Tests for external systems". https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-4+Support+for+System+Tests+for+external+systems This is the very first AIP I created almost 1