YES - I strongly agree with this! I first did it this way because I wanted to follow the instructions, assuming there was some Airflow magic, and later found it really frustrating to maintain. We should be clear that standard Python packaging is the way to go.
That being said, what if Airflow had an official Cookiecutter template for Airflow operator packages, and suggested that as a way to manage your operators in the docs? That would probably help people who aren't as familiar with Python, and over time might increase quality of third party operators (if it ships by default with linting/etc.) On Fri, Apr 12, 2019 at 3:54 AM Ash Berlin-Taylor <a...@apache.org> wrote: > This is something I've said a number of times (but possibly never on the > list): > > I think we should deprecate adding Operators and Hooks via the Airflow > plugin mechanism. > > I think plugins should be reserved for any mechanism that a plain-ol > python module import won't work for (which is basically anything that needs > to tie deeply in to the Webserver or Scheduler process). > > To that endI think we should deprecate adding operators via plugins: > > from airflow.operators.my_plugin import MyOperator > > can become > > from my_plugin import MyOperator > > with no impact on functionality. > > The same could be said for hooks, with one possible feature addition: > Right now the list of connection types in the Connections screen is > hard-coded. Is it possible worth making that dynamic somehow based on the > Hook classes that are loaded? i.e. adding the ability for hooks added in > plguins to add items to that list? > > -ash
