FYI, Beam dependency checker is still complaining currently since google-auth currently sets the restriction to rsa<4.1 for Py2.
There are 2 independent changes in flight, which should fix this: https://github.com/googleapis/google-auth-library-python/pull/535, https://github.com/apache/beam/pull/11996 On Thu, Jun 11, 2020 at 4:28 PM Udi Meiri <[email protected]> wrote: > BTW, the new pip resolver seems to do the right thing by installing > rsa==4.0 instead of 4.2 in this case (upgrade pip to 20.1.1): > > pip --unstable-feature=resolver install apache-beam[gcp] > > > https://discuss.python.org/t/an-update-on-pip-and-dependency-resolution/1898/4 > > On Thu, Jun 11, 2020 at 2:40 PM Valentyn Tymofieiev <[email protected]> > wrote: > >> > In python 2 oauth2client's rsa>3.14 requirement will resolve to >> latest python2 supporting version of rsa (4.0?) >> >> Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent the >> breakage of Py2 users, opened: >> https://github.com/sybrenstuvel/python-rsa/issues/152. >> >> On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <[email protected]> wrote: >> >>> >>> >>> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> google-auth has been released (with the wider pin >>>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> >>>> on >>>> rsa). >>>> >>> >>> Thank you! Much appreciated! >>> >>> >>>> >>>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <[email protected]> >>>>> wrote: >>>>> >>>>>> The fix to google-auth has been merged. Is the plan just to wait >>>>>> until a new version of google-auth is released and ignore the failing >>>>>> tests >>>>>> until then? (btw I filed a JIRA for this before I realized it was already >>>>>> being discussed here: >>>>>> https://issues.apache.org/jira/browse/BEAM-10232) >>>>>> >>>>> >>>>> Could we add it as a test dependency? Or if that is not possible, add >>>>> it but remove it before next release? >>>>> >>>>> It seems like there is a release PR on google-auth ( >>>>> https://github.com/googleapis/google-auth-library-python/pull/525). I >>>>> asked +Bu Sun Kim <[email protected]> on the PR, they usually >>>>> release pretty quickly. >>>>> >>>>> >>>>>> >>>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <[email protected]> wrote: >>>>>> >>>>>>> Yes you're right, Py2 envs are still using 4.0. >>>>>>> >>>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <[email protected]> wrote: >>>>>>>> >>>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for >>>>>>>>> that yet. >>>>>>>>> >>>>>>>> >>>>>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14 >>>>>>>> requirement will resolve to latest python2 supporting version of rsa >>>>>>>> (4.0?) >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Looks like there is an attempt to fix this: >>>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524 >>>>>>>>>> >>>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> You may be interested in following >>>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already. >>>>>>>>>>>>> >>>>>>>>>>>>> Kenn >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but >>>>>>>>>>>>>> pip doesn't do transitive deps well. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Would it be right to put a direct dependency >>>>>>>>>>>>>> on rsa<4.1,>=3.1.4 in setup.py? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> Did you find where the google-auth dependency is coming from? >>>>>>>>>>>> We might try to fix the problem at the source of that dependency >>>>>>>>>>>> instead of >>>>>>>>>>>> adding rsa to beam's setup.py. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 >>>>>>>>>>> was released today. >>>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency >>>>>>>>>>> tree. For example: >>>>>>>>>>> >>>>>>>>>>> google-cloud-bigquery==1.24.0 >>>>>>>>>>> - google-api-core [required: >=1.15.0,<2.0dev, installed: >>>>>>>>>>> 1.20.0] >>>>>>>>>>> - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1] >>>>>>>>>>> - rsa [required: >=3.1.4,<4.1, installed: 4.1] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <[email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :) >>>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my >>>>>>>>>>>>>>> pip install command above. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is >>>>>>>>>>>>>>>> coming from? >>>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a >>>>>>>>>>>>>>>> Jenkins job, and see if it helps. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/ >>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, >>>>>>>>>>>>>>>>> but you'll have rsa 4.1 which is incompatible >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> However, when I do >>>>>>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud] >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> locally, the google-auth package is not installed at all. >>>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming >>>>>>>>>>>>>>>>> from? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>
