I believe we can do a release that is just focused on the Python artifacts, yes.
On Mon, Dec 7, 2020 at 6:52 AM Joris Van den Bossche <jorisvandenboss...@gmail.com> wrote: > > On Fri, 4 Dec 2020 at 21:11, Uwe L. Korn <m...@uwekorn.com> wrote: > > > Hello all, > > > > Today the Karotothek CI turned quite red in > > https://github.com/JDASoftwareGroup/kartothek/pull/383 / > > https://github.com/JDASoftwareGroup/kartothek/pull/383/checks?check_run_id=1497941813 > > as the new NumPy 1.20rc1 was pulled in. It simply broke all pyarrow<->NumPy > > interop as now dtypes returned by numpy are actual subclasses not directly > > numpy.dtype instances anymore. I reported the issue over at > > https://github.com/numpy/numpy/issues/17913. We are running into that as > > we build our wheels and conda packages with an older release of NumPy that > > has a faulty implementation of PyArray_DescrCheck. > > > > (a) For upcoming releases, we can either move our minimal supported NumPy > > to 1.16.6 or merge the PR over at > > https://github.com/apache/arrow/pull/8834 > > (b) Existing conda(-forge) packages can get a repodata patch that adds a > > numpy<1.20 constraint to them > > (c) I'll rebuild the latest but still frequently used pyarrow releases on > > conda-forge using numpy 1.16.6 > > (d) Old pyarrow wheels (Python<3.8) though won't be easily fixed and > > instead will return the confusing "ArrowTypeError: Did not pass numpy.dtype > > object" error message. Personally my approach would be here to not do > > anything and simply direct users to downgrade NumPy if they run into the > > issue. > > > > In addition to this last item (pip installs), doing a small 2.0.1 bugfix > release with this patch would also help a lot I think. It would at least > ensure that plain pip installs with latest versions will work (while it > doesn't solve it for older pyarrow releases of course, in case people > upgrade numpy in an existing environment, or install numpy with pyarrow > pinned to an older version). > > Does our project governance allow doing a python-only release? (meaning, a > release branch where the 2.0.1 tag compared to 2.0.0 tag only includes > changes to the python libraries) That would make it less burdensome to > resolve part of this situation. > > > > Is anyone objecting to this approach? > > > > Cheers > > Uwe > >
