I have bad news and good news :) Bad news - just making "providers" package an implicit one does not work - when you mix implicit and explicit packages there are some problems. Tools like pytest, mypy, pylint do not handle this well and think that "google", "amazon" are top-level packages when analysing the sources.
Good news - I managed to get providers packages for backporting work without having to change "providers" to implicit package. It was pretty much just a matter of filtering found packages in setup.py - I am sending a PR shortly to enable that. I also managed to use breeze environment to test the backported packages with system tests (at least for GCP operators where we have system test). We can therefore close the subject of implicit packages. We do not need them for now. Later we might want to convert to implicit packages but this is something that might never happen. J, On Wed, Feb 5, 2020 at 2:40 PM Jarek Potiuk <[email protected]> wrote: > Yep. Learning from the bigger move I will see how small is the "smaller" > one I would love to see and can make informed decision. > > On Wed, Feb 5, 2020 at 1:35 PM Ash Berlin-Taylor <[email protected]> wrote: > >> Ah right okay, not an impossible task then. >> >> Still, I am very strongly in favour of the smallest change for the goal >> (which is to allow back-ported providers to be installed on 1.10.x) - both >> cos then most things are more normal, including code under a provider, and >> then that way we don't have to fight our tools. >> On Feb 5 2020, at 12:08 pm, Jarek Potiuk <[email protected]> >> wrote: >> > > >> > > If we make every namespace under airflow.* implict, then we have to >> ensure >> > > that our modules do not clash with _any and every_ possible top level >> > > package that exists in PyPi, because we cannot know what might be >> installed >> > > by our users. That seems like an impossible feat. Have I >> mis-understood? >> > > If I haven't mis-understoond then I this is I think a strong argument >> for >> > > _not_ making more than we absolutely have to an implicit namepsace. >> > > >> > >> > Nope. This is a problem for MyPy and Pytest only so mostly for our >> tooling. >> > And only if our own modules import packages with the same name as the >> > module. So we are in full control here. >> > >> > > >> > >> > > -a >> > > >> > >> > -- >> > Jarek Potiuk >> > Polidea <https://www.polidea.com/> | Principal Software Engineer >> > >> > M: +48 660 796 129 <+48660796129> >> > [image: Polidea] <https://www.polidea.com/> >> > >> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
