Yeah I think this is an important point: > > Core Airflow so far hasn't had the dependencies > required to make those executors functional either - users either had to > use the extra or install the provider directly. So that doesn't really > change if we choose not to preinstall the providers. > And this is also:
> Why would it be beneficial to not pre-install them? The most obvious > benefit is you don't get a handful of new required deps you aren't using > (like celery/flower/billiard/etc if you don't use CeleryExecutor). > And to clarify a bit, cus I was a bit confused when I first read... Currently, when you install airflow, celery executor code is included but in a non-working state unless you install the celery extra. So moving it out won't in many cases hurt people using celery. Though e.g. it could hurt someone who maybe does pip install airflow and then pip install celery (i.e. not using the extra) but hey that's probably not so common. But given that we're moving the code into providers we have a choice to pre-install or not. Pre-installing might be more convenient for some, but the cost is, where previously you would not have celery and kubernetes libs installed, now you would have them. And installing any library is not risk free so if you force everyone to install them then there's risk there too. It seems there could be a third option too, where we do some shenanigans with maybe *provider* extras, or perhaps make k8s executor its *own* provider but ... those don't seem like the right long term path to me. So I lean rip off the bandaid and don't pre-install.
