Re: AIP-8 Split Hooks/Operators into Separate Packages

2019-01-10 Thread Maxime Beauchemin
A related though is around the fact that all these libs depend on Airflow itself to get the base class they are deriving (BaseHook and BaseOperator mostly). It's a bit upside down when the small library depends on a big library. That may be ok as is, but pushing the micro-package logic would

Re: AIP-8 Split Hooks/Operators into Separate Packages

2019-01-10 Thread Maxime Beauchemin
That's not what I meant. If I apply what I meant to your example we'd have a single package for each hook `airflow-hook-s3` and `airflow-hook-gcs`, and a package for `airflow-operator-s3-to-gcs`. The operator package would depend on both hook packages. There's no code or test duplication there.