In general it’s easier for cloud providers to fund development of operators that bring data in. I’d say if there is overlap, put the operator in the target system’s repo.
On Mon, Jan 7, 2019 at 2:17 PM Maxime Beauchemin <maximebeauche...@gmail.com> wrote: > Something to think about is how data transfer operators like the > MysqlToHiveOperator usually rely on 2 hooks. With a package-specific > approach that may mean something like an `airflow-hive`, `airflow-mysql` > and `airflow-mysql-hive` packages, where the `airflow-mysql-hive` package > depends on the two other packages. > > It's just a matter of having a clear strategy, good naming conventions and > a nice central place in the docs that centralize a list of approved > packages. > > Max > > On Mon, Jan 7, 2019 at 9:05 AM Tim Swast <sw...@google.com.invalid> wrote: > > > I've created AIP-8: > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303 > > > > To follow-up from the discussion about splitting hooks/operators out of > the > > core Airflow package at > > > > > http://mail-archives.apache.org/mod_mbox/airflow-dev/201809.mbox/%3c308670db-bd2a-4738-81b1-3f6fb312c...@apache.org%3E > > > > I propose packaging based on the target system, informed by the existing > > hooks in both core and contrib. This will allow those with the relevant > > expertise in each target system to respond to contributions / issues > > without having to follow the flood of everything Airflow-related. It will > > also decrease the surface area of the core package, helping with > > testability and long-term maintenance. > > > > * • **Tim Swast* > > * • *Software Friendliness Engineer > > * • *Google Cloud Developer Relations > > * • *Seattle, WA, USA > > > -- * • **Tim Swast* * • *Software Friendliness Engineer * • *Google Cloud Developer Relations * • *Seattle, WA, USA
