I’m not sure package structure based on whether major providers will fund development is the right approach. My $.02
Sent from a device with less than stellar autocorrect > On Jan 7, 2019, at 3:44 PM, Tim Swast <sw...@google.com.INVALID> wrote: > > In general it’s easier for cloud providers to fund development of operators > that bring data in. I’d say if there is overlap, put the operator in the > target system’s repo. > > On Mon, Jan 7, 2019 at 2:17 PM Maxime Beauchemin <maximebeauche...@gmail.com> > wrote: > >> Something to think about is how data transfer operators like the >> MysqlToHiveOperator usually rely on 2 hooks. With a package-specific >> approach that may mean something like an `airflow-hive`, `airflow-mysql` >> and `airflow-mysql-hive` packages, where the `airflow-mysql-hive` package >> depends on the two other packages. >> >> It's just a matter of having a clear strategy, good naming conventions and >> a nice central place in the docs that centralize a list of approved >> packages. >> >> Max >> >>> On Mon, Jan 7, 2019 at 9:05 AM Tim Swast <sw...@google.com.invalid> wrote: >>> >>> I've created AIP-8: >>> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303 >>> >>> To follow-up from the discussion about splitting hooks/operators out of >> the >>> core Airflow package at >>> >>> >> http://mail-archives.apache.org/mod_mbox/airflow-dev/201809.mbox/%3c308670db-bd2a-4738-81b1-3f6fb312c...@apache.org%3E >>> >>> I propose packaging based on the target system, informed by the existing >>> hooks in both core and contrib. This will allow those with the relevant >>> expertise in each target system to respond to contributions / issues >>> without having to follow the flood of everything Airflow-related. It will >>> also decrease the surface area of the core package, helping with >>> testability and long-term maintenance. >>> >>> * • **Tim Swast* >>> * • *Software Friendliness Engineer >>> * • *Google Cloud Developer Relations >>> * • *Seattle, WA, USA >>> >> > -- > * • **Tim Swast* > * • *Software Friendliness Engineer > * • *Google Cloud Developer Relations > * • *Seattle, WA, USA
