And also how it failed when making it an implicit package? I did do some quick tests before suggesting it earlier in this thread and pytest and mypy ran seemingly without error (but I did only do very quick tests)
My understanding is that come pip install time without implicit packages that the order in which modules get installed can change/break the behaviour. -a On Feb 9 2020, at 9:09 pm, Kamil Breguła <[email protected]> wrote: > Can you show some code snippet? I am curious about this solution. > > On Sun, Feb 9, 2020 at 10:06 PM Jarek Potiuk <[email protected]> wrote: > > > > I have bad news and good news :) > > Bad news - just making "providers" package an implicit one does not work - > > when you mix implicit and explicit packages there are some problems. Tools > > like pytest, mypy, pylint do not handle this well and think that "google", > > "amazon" are top-level packages when analysing the sources. > > > > Good news - I managed to get providers packages for backporting work > > without having to change "providers" to implicit package. It was pretty > > much just a matter of filtering found packages in setup.py - I am sending > > a PR shortly to enable that. I also managed to use breeze environment to > > test the backported packages with system tests (at least for GCP operators > > where we have system test). > > > > We can therefore close the subject of implicit packages. We do not need > > them for now. Later we might want to convert to implicit packages but this > > is something that might never happen. > > > > J, > > > > > > > > > > On Wed, Feb 5, 2020 at 2:40 PM Jarek Potiuk <[email protected]> > > wrote: > > > > > Yep. Learning from the bigger move I will see how small is the "smaller" > > > one I would love to see and can make informed decision. > > > > > > On Wed, Feb 5, 2020 at 1:35 PM Ash Berlin-Taylor <[email protected]> wrote: > > > > Ah right okay, not an impossible task then. > > > > Still, I am very strongly in favour of the smallest change for the goal > > > > (which is to allow back-ported providers to be installed on 1.10.x) - > > > > both > > > > cos then most things are more normal, including code under a provider, > > > > and > > > > then that way we don't have to fight our tools. > > > > On Feb 5 2020, at 12:08 pm, Jarek Potiuk <[email protected]> > > > > wrote: > > > > > > > > > > > > If we make every namespace under airflow.* implict, then we have to > > > > ensure > > > > > > that our modules do not clash with _any and every_ possible top > > > > > > level > > > > > > package that exists in PyPi, because we cannot know what might be > > > > > > > > > > > > > installed > > > > > > by our users. That seems like an impossible feat. Have I > > > > > > > > > > > > > mis-understood? > > > > > > If I haven't mis-understoond then I this is I think a strong > > > > > > argument > > > > > > > > > > > > > for > > > > > > _not_ making more than we absolutely have to an implicit namepsace. > > > > > > > > > > > > > > > > Nope. This is a problem for MyPy and Pytest only so mostly for our > > > > tooling. > > > > > And only if our own modules import packages with the same name as the > > > > > module. So we are in full control here. > > > > > > > > > > > > > > > > > > > > > > -a > > > > > > > > > > -- > > > > > Jarek Potiuk > > > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > > > > > M: +48 660 796 129 <+48660796129> > > > > > [image: Polidea] <https://www.polidea.com/> > > > > > > > > > > > > > > > > > > > -- > > > Jarek Potiuk > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > M: +48 660 796 129 <+48660796129> > > > [image: Polidea] <https://www.polidea.com/> > > > > > > > > > > -- > > Jarek Potiuk > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > M: +48 660 796 129 <+48660796129> > > [image: Polidea] <https://www.polidea.com/> > >
