Something to think about is how data transfer operators like the MysqlToHiveOperator usually rely on 2 hooks. With a package-specific approach that may mean something like an `airflow-hive`, `airflow-mysql` and `airflow-mysql-hive` packages, where the `airflow-mysql-hive` package depends on the two other packages.
It's just a matter of having a clear strategy, good naming conventions and a nice central place in the docs that centralize a list of approved packages. Max On Mon, Jan 7, 2019 at 9:05 AM Tim Swast <sw...@google.com.invalid> wrote: > I've created AIP-8: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303 > > To follow-up from the discussion about splitting hooks/operators out of the > core Airflow package at > > http://mail-archives.apache.org/mod_mbox/airflow-dev/201809.mbox/%3c308670db-bd2a-4738-81b1-3f6fb312c...@apache.org%3E > > I propose packaging based on the target system, informed by the existing > hooks in both core and contrib. This will allow those with the relevant > expertise in each target system to respond to contributions / issues > without having to follow the flood of everything Airflow-related. It will > also decrease the surface area of the core package, helping with > testability and long-term maintenance. > > * • **Tim Swast* > * • *Software Friendliness Engineer > * • *Google Cloud Developer Relations > * • *Seattle, WA, USA >
