Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Tue, Nov 14, 2017 at 7:58 AM, Thomas Caswell wrote: > I am in very supportive of this plan. > > For Matplotlib the intention is to do a mpl2.2LTS early 2018 and a mpl3.0 > (no major API breaks other than dropping py2 support) summer 2018 with the > same meaning of LTS. > > I also had thought about bumping the minimum numpy version of Matplotlib > to the first py3 only version when it is out. There is no technical > reason, but it seems nicely symmetric. > > In general we all need to get better about dropping support for old > versions of dependencies (I am throwing stones from inside my glass > house). The prolonged support of py2 has warped our idea of how long old > versions of things need to be supported and it imposes real costs up and > down the stack. > My $2c: dropping support for all-but-the-latest numpy is not a great idea. There's no need to support numpy versions that are >3 years old, but supporting 2-4 versions back is something most projects have consistently done, and it has real value. Both in terms of not forcing users to upgrade multiple packages in lock-step, and for things like debugging (is it a new numpy or an mpl bug? --> check if the failure disappears with older numpy). Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy grant update
On Thu, Oct 26, 2017 at 12:40 PM, Nathaniel Smith wrote: > On Wed, Oct 18, 2017 at 10:24 PM, Nathaniel Smith wrote: >> I'll also be giving a lunch talk at BIDS tomorrow to let folks locally >> know about what's going on, which I think will be recorded – I'll send >> around a link after in case others are interested. > > Here's that link: https://www.youtube.com/watch?v=fowHwlpGb34 Still no update on that job ad (though we're learning interesting things about Berkeley's HR system!), but we did make a little scratch repo to start brainstorming. This is mostly for getting our own thoughts in order, but if anyone's curious then here it is: https://github.com/njsmith/numpy-grant-planning/ -n -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
> Just to check that I am not misunderstanding: the version of pip should > not be more than a year old; "decades old" is just French hyperbola? Do I > understand right? Yes, sorry if you can't hear my french accent in writing, I can hear yours :-) There is also a "softer" requirement on setuptools which needs to be recent enough to 1) understand requires_python on machine that will _create_ the sdist/wheel. or 2) accept requires_python as a kwarg (even if does nothing), for linux system that will install from sdist. But by end of 2018 that will be a 3 or 4 years old setuptools. > Right, the requirement is pip 9, which is currently one year old and will be > >2 years old by the time this matters for numpy. > It does turn out that there's a bimodal distribution in the wild, where > people tend to either use an up to date pip, or else use some truly ancient > pip that some Linux LTS distro shipped 5 years ago. Numpy isn't the only > project that > will be forcing people to upgrade, though, so I think this > will work itself out. Especially since in the broken case what happens is > that users end up running our setup.py on an unsupported version of python, > so we'll be able to > detect that and print some loud and informative message. Correct, we did that for IPython, got a large spike of sdist-download from Py2+old_pip when we released a Py3 only, the spike disappeared after a few days. We still had a handful of bug reports from people thinking the "You must upgrade pip" message was not relevant, and we realised people pinned ipython with IPython==5.0.0 instead of IPython<6. So the "Loud informative message" should also like tell user how to pin numpy if they can't upgrade pip. -- Matthias On Mon, Nov 13, 2017 at 1:33 PM, Nathaniel Smith wrote: > On Nov 13, 2017 12:03, "Gael Varoquaux" > wrote: > > On Mon, Nov 13, 2017 at 10:26:31AM -0800, Matthias Bussonnier wrote: >> This behavior is "new" (Nov/Dec 2016). [snip] >> It _does_ require to have a version of pip which is not decades old > > Just to check that I am not misunderstanding: the version of pip should > not be more than a year old; "decades old" is just French hyperbola? Do I > understand right? > > > Right, the requirement is pip 9, which is currently one year old and will be >>2 years old by the time this matters for numpy. > > It does turn out that there's a bimodal distribution in the wild, where > people tend to either use an up to date pip, or else use some truly ancient > pip that some Linux LTS distro shipped 5 years ago. Numpy isn't the only > project that will be forcing people to upgrade, though, so I think this will > work itself out. Especially since in the broken case what happens is that > users end up running our setup.py on an unsupported version of python, so > we'll be able to detect that and print some loud and informative message. > > -n > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Nov 13, 2017 12:03, "Gael Varoquaux" wrote: On Mon, Nov 13, 2017 at 10:26:31AM -0800, Matthias Bussonnier wrote: > This behavior is "new" (Nov/Dec 2016). [snip] > It _does_ require to have a version of pip which is not decades old Just to check that I am not misunderstanding: the version of pip should not be more than a year old; "decades old" is just French hyperbola? Do I understand right? Right, the requirement is pip 9, which is currently one year old and will be >2 years old by the time this matters for numpy. It does turn out that there's a bimodal distribution in the wild, where people tend to either use an up to date pip, or else use some truly ancient pip that some Linux LTS distro shipped 5 years ago. Numpy isn't the only project that will be forcing people to upgrade, though, so I think this will work itself out. Especially since in the broken case what happens is that users end up running our setup.py on an unsupported version of python, so we'll be able to detect that and print some loud and informative message. -n ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Mon, Nov 13, 2017 at 10:26:31AM -0800, Matthias Bussonnier wrote: > This behavior is "new" (Nov/Dec 2016). [snip] > It _does_ require to have a version of pip which is not decades old Just to check that I am not misunderstanding: the version of pip should not be more than a year old; "decades old" is just French hyperbola? Do I understand right? Gaël ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
I am in very supportive of this plan. For Matplotlib the intention is to do a mpl2.2LTS early 2018 and a mpl3.0 (no major API breaks other than dropping py2 support) summer 2018 with the same meaning of LTS. I also had thought about bumping the minimum numpy version of Matplotlib to the first py3 only version when it is out. There is no technical reason, but it seems nicely symmetric. In general we all need to get better about dropping support for old versions of dependencies (I am throwing stones from inside my glass house). The prolonged support of py2 has warped our idea of how long old versions of things need to be supported and it imposes real costs up and down the stack. Tom On Mon, Nov 13, 2017 at 1:26 PM Matthias Bussonnier < bussonniermatth...@gmail.com> wrote: > > If a wheel is not available for the client platform, pip will try to > install the latest version of the source distribution (.tar.gz or .zip) > which I think is the cause of the problem here. > > Unless the sdist is tagged with require_python and users have > recent-enough pip. Which is what was referred to earlier as > "Automagically". > This behavior is "new" (Nov/Dec 2016). The upstream patches were > written (in part) by the IPython/Jupyter team, for this exact purpose, > to not install an incompatible sdists. > (Works great I can share download graphs for IPython[0]) > > It _does_ require to have a version of pip which is not decades old > though, and may not work if you use a pypi proxy which is not pep 503 > compliant (which happens, we got bug report, users then complained to > IT who fixed it). > -- > M > > On Mon, Nov 13, 2017 at 10:14 AM, Olivier Grisel > wrote: > > If a wheel is not available for the client platform, pip will try to > install > > the latest version of the source distribution (.tar.gz or .zip) which I > > think is the cause of the problem here. > > > > -- > > Olivier > > > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] PyArray_GETITEM and PyArray_SETITEM
Hello All, I need to make use of the limited numpy API access Pybind11 gives, in order to add a feature to it. It seems to give access to functions from numpy_api.py [1]. I need to use PyArray_GETITEM and PyArray_SETITEM in order to get and set array elements [2], these functions / macros are not exposed via numpy_api.py, but are in `numpy/ndarraytypes.h`. We were wondering why aren't PyArray_GETITEM and PyArray_SETITEM exposed like the rest of numpy API? Is it possible to replicate the behavior using the members exposed in numpy_api.py ? Any help would be appreciated. Mmanu [1] https://github.com/numpy/numpy/blob/1368cbb696ae27b849eed67b4fd31c 550a55dad5/numpy/core/code_generators/numpy_api.py [2] https://github.com/pybind/pybind11/pull/1152/files#diff- 52f1945d779be1e60903590907bb9326R241 ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Mon, Nov 13, 2017 at 10:31 AM, Stefan van der Walt wrote: > > Is this documented anywhere? I couldn't find it via Google, and suspect > it may be widely useful in the next few months. Everything you need to know is on the Python3Statement practicality page: http://www.python3statement.org/practicalities/ (If it's not , or is unclear, complain to me or TK, yes we should make it more visible) M Pacer and I also gave a talk at Pycon https://www.youtube.com/watch?v=2DkfPzWWC2Q, slides https://carreau.github.io/pycon2017/#/ and Pybay https://www.youtube.com/watch?v=3i6n1RwqQCo, slides http://carreau.github.io/talks/2017-08-13-pybay/docs/index.html#/ Raw data for the graphs https://github.com/Carreau/talks/blob/master/2017-08-13-pybay/IPython-dls_2.ipynb -- M > > Stéfan > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Mon, Nov 13, 2017, at 10:26, Matthias Bussonnier wrote: > Unless the sdist is tagged with require_python and users have > recent-enough pip. Is this documented anywhere? I couldn't find it via Google, and suspect it may be widely useful in the next few months. Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
> If a wheel is not available for the client platform, pip will try to install > the latest version of the source distribution (.tar.gz or .zip) which I think > is the cause of the problem here. Unless the sdist is tagged with require_python and users have recent-enough pip. Which is what was referred to earlier as "Automagically". This behavior is "new" (Nov/Dec 2016). The upstream patches were written (in part) by the IPython/Jupyter team, for this exact purpose, to not install an incompatible sdists. (Works great I can share download graphs for IPython[0]) It _does_ require to have a version of pip which is not decades old though, and may not work if you use a pypi proxy which is not pep 503 compliant (which happens, we got bug report, users then complained to IT who fixed it). -- M On Mon, Nov 13, 2017 at 10:14 AM, Olivier Grisel wrote: > If a wheel is not available for the client platform, pip will try to install > the latest version of the source distribution (.tar.gz or .zip) which I > think is the cause of the problem here. > > -- > Olivier > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
If a wheel is not available for the client platform, pip will try to install the latest version of the source distribution (.tar.gz or .zip) which I think is the cause of the problem here. -- Olivier ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
> The trouble is PyPi doesn't allow multiple branches. So if you upload NumPy > 2.0 wheels, then you cannot turn around and upload 1.18.X bug-fix patches. > At least, this is my understanding of PyPi. That's perfectly feasible. We've been maintaining a 6.x (Python 3 only) and 5.x (Python2+3) of IPython for about a year now. -- M On Mon, Nov 13, 2017 at 10:08 AM, Robert McLeod wrote: > > > On Mon, Nov 13, 2017 at 7:47 AM, Matthias Bussonnier > wrote: >> >> > For this to be efficient, it should be done soon enough to allow >> > downstream projects to adapt their requirements.txt. >> > Release managers: how much more effort would it be to upload current >> > numpy to both numpy and numpylts? >> >> I'm not quite sure I see the point. you would ask downstream to change >> `numpy` to `numpylts` instead of `numpy` to `numpy<2` ? >> >> Also I think then you have the risk of having for example pandas >> saying `numpy<2` and scipy saying `numpylts` and now the pasckages are >> incompatible ? > > > The trouble is PyPi doesn't allow multiple branches. So if you upload > NumPy 2.0 wheels, then you cannot turn around and upload 1.18.X bug-fix > patches. At least, this is my understanding of PyPi. > > > > -- > Robert McLeod, Ph.D. > robbmcl...@gmail.com > robbmcl...@protonmail.com > www.entropyreduction.al > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Mon, Nov 13, 2017 at 7:47 AM, Matthias Bussonnier < bussonniermatth...@gmail.com> wrote: > > For this to be efficient, it should be done soon enough to allow > downstream projects to adapt their requirements.txt. > > Release managers: how much more effort would it be to upload current > numpy to both numpy and numpylts? > > I'm not quite sure I see the point. you would ask downstream to change > `numpy` to `numpylts` instead of `numpy` to `numpy<2` ? > > Also I think then you have the risk of having for example pandas > saying `numpy<2` and scipy saying `numpylts` and now the pasckages are > incompatible ? The trouble is PyPi doesn't allow multiple branches. So if you upload NumPy 2.0 wheels, then you cannot turn around and upload 1.18.X bug-fix patches. At least, this is my understanding of PyPi. -- Robert McLeod, Ph.D. robbmcl...@gmail.com robbmcl...@protonmail.com www.entropyreduction.al ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On Fri, Nov 10, 2017 at 2:03 PM, Robert McLeod wrote: > Pip repo names and actual module names don't have to be the same. One > potential work-around would be to make a 'numpylts' repo on PyPi which is > the 1.17 version with support for Python 2.7 and bug-fix releases as > required. This will still cause regressions but it's a matter of modifying > `requirements.txt` in downstream Python 2.7 packages and not much else. > > E.g. in `requirements.txt`: > > numpy;python_version>"3.0" > numpylts; python_version<"3.0" > Can't we handle this with numpy versioning? IIUC, numpy (py3 only) and numpy (LTS) will not only support different platforms, but also be different versions. So if you have py2 or py2+3 code that uses numpy, it will have to specify a <= version number anyway. Also -- I think Nathaniel's point was that wheels have the python version baked in, so pip, when run from py2, should find the latest py2 compatible numpy automagically. And thanks for writing this up -- LGTM -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
> For this to be efficient, it should be done soon enough to allow downstream > projects to adapt their requirements.txt. > Release managers: how much more effort would it be to upload current numpy to > both numpy and numpylts? I'm not quite sure I see the point. you would ask downstream to change `numpy` to `numpylts` instead of `numpy` to `numpy<2` ? Also I think then you have the risk of having for example pandas saying `numpy<2` and scipy saying `numpylts` and now the pasckages are incompatible ? -- M On Mon, Nov 13, 2017 at 12:11 AM, Daπid wrote: > On 10 November 2017 at 23:03, Robert McLeod wrote: >> >> E.g. in `requirements.txt`: >> >> numpy;python_version>"3.0" >> numpylts; python_version<"3.0" >> >> In both cases you still call `import numpy` in the code. > > > For this to be efficient, it should be done soon enough to allow downstream > projects to adapt their requirements.txt. > > Release managers: how much more effort would it be to upload current numpy > to both numpy and numpylts? > > /David. > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal of timeline for dropping Python 2.7 support
On 10 November 2017 at 23:03, Robert McLeod wrote: > E.g. in `requirements.txt`: > > numpy;python_version>"3.0" > numpylts; python_version<"3.0" > > In both cases you still call `import numpy` in the code. > For this to be efficient, it should be done soon enough to allow downstream projects to adapt their requirements.txt. Release managers: how much more effort would it be to upload current numpy to both numpy and numpylts? /David. ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
