Re: [NON-TECHNICAL] [DISCUSS] Being even more welcoming community ?

2020-02-15 Thread Jarek Potiuk
On Fri, Jan 31, 2020, 22:21 Karolina Rosół wrote: > Hi guys, > > TL;DR; Non-technical people would love to build this community as > well. We just don't know how ;-) > Thanks for thone comments! 1. It is amazing that there's an info on the Apache website about the > devlist. However, I totally

Re: [DISCUSS] AIP-4 System tests

2020-02-15 Thread Jarek Potiuk
On Sat, Feb 15, 2020, 20:54 Ash Berlin-Taylor wrote: > Yeah, I'm for this. > > In fact I'm about to mark some of the Hive ones as system tests as they > require a running hive cluster. > I would be careful about which automatically marking unit tests that run > dags as system/integration though,

Re: "Powered By" (working title) section at the official Airflow Website

2020-02-15 Thread Jarek Potiuk
and I love the Ecosystem name. On Sat, Feb 15, 2020 at 8:51 PM Jarek Potiuk wrote: > Anyone else with an opinion against ? if not I will! proceed with adding > some of the Powered bys. > > Another example of great 'powered by' enabled by Airflow (amongst others) > is for example 'dbt' - which s

Re: [DISCUSS] AIP-4 System tests

2020-02-15 Thread Ash Berlin-Taylor
Yeah, I'm for this. In fact I'm about to mark some of the Hive ones as system tests as they require a running hive cluster. I would be careful about which automatically marking unit tests that run dags as system/integration though, a number of our unit tests rely on this to test the tasks in va

Re: "Powered By" (working title) section at the official Airflow Website

2020-02-15 Thread Jarek Potiuk
Anyone else with an opinion against ? if not I will! proceed with adding some of the Powered bys. Another example of great 'powered by' enabled by Airflow (amongst others) is for example 'dbt' - which seems to be very popular recently - Iam watching it closely. it would be great to pay those too

Re: [DISCUSS] AIP-4 System tests

2020-02-15 Thread Tomasz Urbaszek
+1 for introducing system tests. Lack of them is a big pain. I would like also to suggest to mark some actual tests (those running DAGs, etc) as system tests. Then we can simplify our units and probably speed up CI builds (not to mention the reduction of side effects). The approach used for GCP sy

Re: [DISCUSS] Reduce (remove?) automated imports in Airflow 2.0

2020-02-15 Thread Ash Berlin-Taylor
I'm massively in favour of this. And as a side effect it would solve an issue a reports almost two years ago https://issues.apache.org/jira/browse/AIRFLOW-1931 (https://issues.apache.org/jira/browse/AIRFLOW-1931?jql=project%20%3D%20AIRFLOW%20AND%20text%20~%20%22logging%20import%22) The one outst

[DISCUSS] Reduce (remove?) automated imports in Airflow 2.0

2020-02-15 Thread Jarek Potiuk
TL;DR; I would like to ask the community for opinion about reducing (or even removing) the number of automated imports we have in `airflow/__init__.py` for Airflow 2.0. This issue is plaguing us for quite a while already and I think we have a perfect opportunity to solve it in AIrflow 2.0. Current

[DISCUSS] AIP-4 System tests

2020-02-15 Thread Jarek Potiuk
TL;DR; I would like to revive a discussion (hopefully short :) and possibly cast a vote on "AIP-4 - Support for System Tests for external systems". https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-4+Support+for+System+Tests+for+external+systems This is the very first AIP I created almost 1