[Python-Dev] Re: Difficulty of testing beta releases now available
Hi, On Wed, May 26, 2021 at 2:05 AM Neil Schemenauer wrote: > - Cython doesn't work because of _PyGen_Send change [1] My team fixed the Python 3.10 compatibility in the Cython 0.29.x branch, but the latest 0.29.x release (0.29.23, April 14, 2021) doesn't include these fixes yet. A new Cython release is needed. I'm not sure how to help Cython to release a new version. In Fedora, we have downstream patches until a new Cython is released: https://src.fedoraproject.org/rpms/Cython/pull-request/28 > - numpy cannot be installed because of _Py_HashDouble() change [2] "pip install numpy" also fails with an error about "incompatible tag". It's an issue in packaging which is vendored in wheel which is used (as a vendored copy?) by pip to build a local wheel package of numpy. numpy doesn't ship wheel packages for Python 3.10 beta releases which is a good idea, the ABI is not stable yet. > Can we do any things to improve the situation? Perhaps using the > pre-release functionality of PyPI would help. We would have to > somehow encourage 3rd party packages to upload pre-releases that are > compatible with our beta/RC releases. In Fedora, we use a "COPR build": we partially rebuild the OS with Python 3.10, and then we check for package build failures. We are working on switching to Python 3.10 by default: * https://fedoraproject.org/wiki/Changes/Python3.10 * https://bugzilla.redhat.com/show_bug.cgi?id=1890881 For example, my colleague Lumir submitted the trio bug report ;-) https://github.com/python-trio/trio/issues/1899 We are doing our best to report issues to projects. Sometimes we help to fix them, but we cannot fix all compatibility issues! In the past, I tried to work on a small Python script to run the test suite of some projects with the main branch of Python. I had practical issues: * There is no simple way to get install test dependencies of a Python project, especially non-Python dependencies. * When a dependency was not compatible with the next Python, I had to maintain a patch until my fix is accepted upstream. * Sometimes, I had to pull the code from Git rather than using a tarball/ZIP (to not have to wait for a new release). At the end, all these practical issues need solutions which look very closely to a recipe to build a package for a Linux distribution: * Recipe to pull the source. * Recipe to pull build dependencies. * Downstream patches (until a fix is merged upstream). * Recipe to run tests: we try to always run tests when we build a package in Fedora. Advantages of using Fedora COPR: * Reuse Fedora infrastructure (building packages need physical servers to build packages and run tests) * Reuse the public Fedora bug tracker to coordinate the work. * Reuse Fedora "specfiles" (recipe to build a RPM package). * Prepare Fedora to be updated to the next Python. Sadly, Fedora COPR might be a little bit too Fedora specific: require to understand specfiles, how to submit a PR on a specfile, use the Fedora bug tracker, etc. Fedora Rawhide was just updated to switch Python 3.10 by default! /usr/bin/python3 is now Python 3.10 beta2. In COPR we could skip tests to build dependencies and discover more compatibiliy issues. In Rawhide, tests cannot be skipped and so some packages cannot be built yet. For example, astropy has a few failing tests: https://github.com/astropy/astropy/issues/11821 Again, the practical problem is that numpy cannot easily be installed on Python 3.10 (pip compatibiltiy tag issue), and so astropy may wait until Python 3.10 final is released to look into these tests failures. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6YRNA2F2G3RSXWG72WFEFOEAARIA57E5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Difficulty of testing beta releases now available
For "pip install numpy", a new numpy release should fix the issue. The issue was already fixed in packaging 20.8: * https://github.com/pypa/packaging/commit/611982be44d7d91453a97082f58d8ea349f89d00 * https://github.com/pypa/packaging/pull/355 The fix is already included in wheel 0.36.2 (which includes a vendored copy of packaging): * https://github.com/pypa/wheel/commit/f6fd2472c7a4a836a3ebe12eab96a1ed8a31c061 And numpy pyproject.toml was updated to use wheel 0.36.2 which contains the fix: * https://github.com/numpy/numpy/commit/9569f40917c80b9506f0969b1e89e2fc119d4ae3 The pyproject.toml fix is not part of 1.20.3: https://github.com/numpy/numpy/blob/v1.20.3/pyproject.toml -- I didn't even know that it was possible to require a specific wheel version in pyproject.toml to build a binary wheel package in pip! Current pyproject.toml: https://github.com/numpy/numpy/blob/main/pyproject.toml [build-system] # Minimum requirements for the build system to execute. requires = [ "packaging==20.5; platform_machine=='arm64'", # macos M1 "setuptools<49.2.0", "wheel==0.36.2", "Cython>=0.29.21,<3.0", # Note: keep in sync with tools/cythonize.py ] Victor On Fri, Jun 11, 2021 at 4:30 PM Victor Stinner wrote: > > Hi, > > On Wed, May 26, 2021 at 2:05 AM Neil Schemenauer > wrote: > > - Cython doesn't work because of _PyGen_Send change [1] > > My team fixed the Python 3.10 compatibility in the Cython 0.29.x > branch, but the latest 0.29.x release (0.29.23, April 14, 2021) > doesn't include these fixes yet. A new Cython release is needed. I'm > not sure how to help Cython to release a new version. > > In Fedora, we have downstream patches until a new Cython is released: > https://src.fedoraproject.org/rpms/Cython/pull-request/28 > > > > - numpy cannot be installed because of _Py_HashDouble() change [2] > > "pip install numpy" also fails with an error about "incompatible tag". > It's an issue in packaging which is vendored in wheel which is used > (as a vendored copy?) by pip to build a local wheel package of numpy. > numpy doesn't ship wheel packages for Python 3.10 beta releases which > is a good idea, the ABI is not stable yet. > > > > Can we do any things to improve the situation? Perhaps using the > > pre-release functionality of PyPI would help. We would have to > > somehow encourage 3rd party packages to upload pre-releases that are > > compatible with our beta/RC releases. > > In Fedora, we use a "COPR build": we partially rebuild the OS with > Python 3.10, and then we check for package build failures. We are > working on switching to Python 3.10 by default: > > * https://fedoraproject.org/wiki/Changes/Python3.10 > * https://bugzilla.redhat.com/show_bug.cgi?id=1890881 > > For example, my colleague Lumir submitted the trio bug report ;-) > https://github.com/python-trio/trio/issues/1899 > > We are doing our best to report issues to projects. Sometimes we help > to fix them, but we cannot fix all compatibility issues! > > In the past, I tried to work on a small Python script to run the test > suite of some projects with the main branch of Python. I had practical > issues: > > * There is no simple way to get install test dependencies of a Python > project, especially non-Python dependencies. > * When a dependency was not compatible with the next Python, I had to > maintain a patch until my fix is accepted upstream. > * Sometimes, I had to pull the code from Git rather than using a > tarball/ZIP (to not have to wait for a new release). > > At the end, all these practical issues need solutions which look very > closely to a recipe to build a package for a Linux distribution: > > * Recipe to pull the source. > * Recipe to pull build dependencies. > * Downstream patches (until a fix is merged upstream). > * Recipe to run tests: we try to always run tests when we build a > package in Fedora. > > Advantages of using Fedora COPR: > > * Reuse Fedora infrastructure (building packages need physical servers > to build packages and run tests) > * Reuse the public Fedora bug tracker to coordinate the work. > * Reuse Fedora "specfiles" (recipe to build a RPM package). > * Prepare Fedora to be updated to the next Python. > > Sadly, Fedora COPR might be a little bit too Fedora specific: require > to understand specfiles, how to submit a PR on a specfile, use the > Fedora bug tracker, etc. > > Fedora Rawhide was just updated to switch Python 3.10 by default! > /usr/bin/python3 is now Python 3.10 beta2. In COPR we could skip tests > to build dependencies and discover more compatibiliy issues. In > Rawhide, tests cannot be skipped and so some packages cannot be built > yet. For example, astropy has a few failing tests: > https://github.com/astropy/astropy/issues/11821 > > Again, the practical problem is that numpy cannot easily be installed > on Python 3.10 (pip compatibiltiy tag issue), and so astropy may wait > until Python 3.10 final is released to look into these tests failures. > > Victor > -- > Night gathers, and now m
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2021-06-04 - 2021-06-11) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open7457 (+31) closed 48676 (+53) total 56133 (+84) Open issues with patches: 2955 Issues opened (60) == #11105: Compiling recursive Python ASTs crash the interpreter https://bugs.python.org/issue11105 reopened by BTaskaya #44242: enum.IntFlag regression: missing values cause TypeError https://bugs.python.org/issue44242 reopened by ethan.furman #44246: 3.10 beta 1: breaking change in importlib.metadata entry point https://bugs.python.org/issue44246 reopened by Anthony Sottile #44314: [doc] SSLContext.set_ciphers() link to OpenSSL cipher list for https://bugs.python.org/issue44314 opened by cjmayo #44316: Support preserving path meaning in os.path.normpath() and absp https://bugs.python.org/issue44316 opened by barneygale #44317: Suggestion for better syntax errors in tokenizer errors https://bugs.python.org/issue44317 opened by wyz23x2 #44318: Asyncio classes missing __slots__ https://bugs.python.org/issue44318 opened by Bluenix2 #44319: setup openssl faild on linux (ubuntu 20.04) https://bugs.python.org/issue44319 opened by Battant #44321: os.EX_OK for Windows https://bugs.python.org/issue44321 opened by samuelmarks #44323: install module fail on windows 10 https://bugs.python.org/issue44323 opened by Battant #44324: add a "expected expression" syntax error https://bugs.python.org/issue44324 opened by CCLDArjun #44325: IDLE: Fix shell comment anomalies https://bugs.python.org/issue44325 opened by terry.reedy #44328: time.monotonic() should use a different clock source on Window https://bugs.python.org/issue44328 opened by lunixbochs2 #44330: IDLE: Colorizer and output tests hang on macOS https://bugs.python.org/issue44330 opened by terry.reedy #44331: Generate static PyCodeObjects for faster startup https://bugs.python.org/issue44331 opened by nascheme #44334: Use bytearray in urllib.unquote_to_bytes https://bugs.python.org/issue44334 opened by eng.mustafaelagamey #44336: Windows buildbots hang after fatal exit https://bugs.python.org/issue44336 opened by jkloth #44338: Port LOAD_GLOBAL to adaptive interpreter https://bugs.python.org/issue44338 opened by Mark.Shannon #44339: Discrepancy between math.pow(0.0, -inf) and 0.0**-inf https://bugs.python.org/issue44339 opened by mark.dickinson #44340: Add support for building cpython with clang thin lto https://bugs.python.org/issue44340 opened by holmanb #44342: enum with inherited type won't pickle https://bugs.python.org/issue44342 opened by Tom.Brown #44343: Adding the "with" statement support to ContextVar https://bugs.python.org/issue44343 opened by achimnol #44344: Documentation for pow() should include the possibility of comp https://bugs.python.org/issue44344 opened by eyadams #44346: Fraction constructor may accept spaces around '/' https://bugs.python.org/issue44346 opened by Sergey.Kirpichev #44347: Unclear documentation for shutil.copytree() https://bugs.python.org/issue44347 opened by tilman.vogel #44348: test_exceptions.ExceptionTests.test_recursion_in_except_handle https://bugs.python.org/issue44348 opened by kj #44350: IDLE: support Command-click on window title on macOS https://bugs.python.org/issue44350 opened by Aivar.Annamaa #44351: distutils.sysconfig.parse_makefile() regression in Python 3.10 https://bugs.python.org/issue44351 opened by hroncok #44353: PEP 604 NewType https://bugs.python.org/issue44353 opened by joperez #44354: ssl deprecation warnings erganomics https://bugs.python.org/issue44354 opened by graingert #44355: Allow spaces in format strings https://bugs.python.org/issue44355 opened by steven.daprano #44358: AMD64 RHEL8 LTO + PGO 3.x build failed with: /usr/bin/ld: Dwar https://bugs.python.org/issue44358 opened by vstinner #44359: test_ftplib fails as "env changes" if a socket operation times https://bugs.python.org/issue44359 opened by vstinner #44361: test_smtpnet failed with SMTPServerDisconnected on x86 Gentoo https://bugs.python.org/issue44361 opened by vstinner #44362: improve documentation of SSL deprecations https://bugs.python.org/issue44362 opened by graingert #44365: Bad dataclass post-init example https://bugs.python.org/issue44365 opened by MicaelJarniac #44368: Invalid mapping patterns give confusing SyntaxErrors https://bugs.python.org/issue44368 opened by brandtbucher #44369: Improve syntax error for wrongly closed strings https://bugs.python.org/issue44369 opened by pablogsal #44370: Inconsistent results for min() and max() with math.nan as argu https://bugs.python.org/issue44370 opened by joel.larose #44371: asyncio.wait_for does not cancel running tasks in the correct https://bugs.python.org/issue44371 opened by ofekkir #44372: Can't install Python3.8, 3.9, 3.10 various errors i
