Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
Hi,
Le ven. 1 mars 2019 à 02:12, Neil Schemenauer a écrit :
> I believe the correct fix is to use PEP 3121 per-interpreter module
> state. I created a new issue:
>
> https://github.com/psycopg/psycopg2/issues/854
>
> I think the fix is not trival as the psycopgmodule.c source code has
> change a fair bit to use the PEP 3121 APIs.
The problem is this function:
/* Return nonzero if the current one is the main interpreter */
static int
psyco_is_main_interp(void)
{
static PyInterpreterState *main_interp = NULL; /* Cached reference */
PyInterpreterState *interp;
if (main_interp) {
return (main_interp == PyThreadState_Get()->interp);
}
/* No cached value: cache the proper value and try again. */
interp = PyInterpreterState_Head();
while (interp->next)
interp = interp->next;
main_interp = interp;
assert (main_interp);
return psyco_is_main_interp();
}
https://github.com/psycopg/psycopg2/blob/599432552aae4941c2b282e9251330f1357b2a45/psycopg/utils.c#L407
I'm not sure that this code is safe. In CPython, iterating on
interp->next is protected by a lock:
HEAD_LOCK();
...
HEAD_UNLOCK();
We already expose the main interpreter since Python 3.7:
PyInterpreterState_Main(). psycopg can be modified to use directly
this function rather than playing black magic with CPython internals.
IMHO it's a good thing that the compilation failed: that such bug is found :-)
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 2019-02-26, Stephane Wirtel wrote:
> I also filled an issue [2] for brotlipy (used by httpbin and requests).
> The problem is with PyInterpreterState.
I tried compiling psycopg2 today and it has a similar problem:
psycopg/psycopgmodule.c: In function ‘psyco_is_main_interp’:
psycopg/psycopgmodule.c:689:18: error: dereferencing pointer to incomplete
type ‘PyInterpreterState’ {aka ‘struct _is’}
while (interp->next)
That code is inside a function:
/* Return nonzero if the current one is the main interpreter */
static int
psyco_is_main_interp(void)
...
I believe the correct fix is to use PEP 3121 per-interpreter module
state. I created a new issue:
https://github.com/psycopg/psycopg2/issues/854
I think the fix is not trival as the psycopgmodule.c source code has
change a fair bit to use the PEP 3121 APIs.
Regards,
Neil
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
Armin Rigo released https://pypi.org/project/cffi/1.12.2/ which is compatible with Python 3.8.0a2. The issue was related to the PyInterpreterState change: https://bugs.python.org/issue35886#msg336501 Note: "[RELEASE] Python 3.8.0a1 is now available for testing" the correct version is 3.8.0a2 :-) Victor Le mar. 26 févr. 2019 à 14:02, Stephane Wirtel a écrit : > > Hi Łukasz, > > Thank you for your job. > > I have created a Merge Request for the docker image of Barry [1]. > > I also filled an issue [2] for brotlipy (used by httpbin and requests). > The problem is with PyInterpreterState. > > Via Twitter, I have proposed to the community to fix the issue [2]. > > [1]: https://gitlab.com/python-devs/ci-images/merge_requests/7 > [2]: https://github.com/python-hyper/brotlipy/issues/147 > > Thanks again for your job. > > Cheers, > > Stéphane > > -- > Stéphane Wirtel - https://wirtel.be - @matrixise > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
Hi Łukasz, Thank you for your job. I have created a Merge Request for the docker image of Barry [1]. I also filled an issue [2] for brotlipy (used by httpbin and requests). The problem is with PyInterpreterState. Via Twitter, I have proposed to the community to fix the issue [2]. [1]: https://gitlab.com/python-devs/ci-images/merge_requests/7 [2]: https://github.com/python-hyper/brotlipy/issues/147 Thanks again for your job. Cheers, Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
I packaged another release. Go get it here: https://www.python.org/downloads/release/python-380a2/ Python 3.8.0a2 is the second of four planned alpha releases of Python 3.8, the next feature release of Python. During the alpha phase, Python 3.8 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next preview release, 3.8.0a3, is planned for 2019-03-25. This time around the stable buildbots were a bit less green than they should have. This early in the cycle, I didn't postpone the release and I didn't use the revert hammer. But soon enough, I will. Let's make sure future changes keep the buildbots happy. - Ł signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 2/7/19 5:16 PM, Stephane Wirtel wrote: On 02/06, Petr Viktorin wrote: On 2/6/19 8:43 AM, Stephane Wirtel wrote: On 02/05, Barry Warsaw wrote: On Feb 5, 2019, at 02:24, Stephane Wirtel wrote: You’re welcome! I just pushed an update to add 3.8.0a1 to the set of Python’s (including git head). Do you think there’s a better way to publicize these images? I know that Julien Palard wanted a docker image with all the versions of Python, see: https://github.com/docker-library/python/issues/373 For my part, I wanted to propose a docker image with the last version of Python and try to use it for the detection of bugs in the main python projects (django, numpy, flask, pandas, etc...) with a CI (example: Gitlab-CI) First issue: pytest uses the ast module of python and since 3.8.0a1, the tests do not pass -> new issue for pytest FWIW, we're preparing to rebuild all Fedora packages with the 3.8 alphas/betas, so everything's tested when 3.8.0 is released: https://fedoraproject.org/wiki/Changes/Python3.8 Hi Petr, Will you execute the tests of these packages? It's best practice to include the test suite in Fedora packages. Sometimes it's not – e.g. if the tests need network access, or all extra testing dependencies aren't available, or most frequently, the maintainer is just lazy. If you have a specific package in mind, I can check. Currently django & numpy get tested; flask & pandas don't. For 3.7, we did the rebuild much later in the cycle. The builds themselves caught async/await SyntaxErrors, and tests caught a lot of StopIteration leaking. At the time it felt like no one really knew what porting to 3.7.0 would look like – similar to how people didn't think "unicode" would be a big problem in py3k. That's what we're trying to avoid for 3.8.0. I have a small discussion with Julien Palard and I wanted to create a small CI where I will execute the tests of the updated packages from the RSS feed of PyPI. The first one was pytest That sounds exciting! Something like that is on my "interesting possible projects" list, but alas, not at the top :( ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
Sorry Petr, I didn't see this message with the test suites. -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 02/06, Petr Viktorin wrote: On 2/6/19 8:43 AM, Stephane Wirtel wrote: On 02/05, Barry Warsaw wrote: On Feb 5, 2019, at 02:24, Stephane Wirtel wrote: You’re welcome! I just pushed an update to add 3.8.0a1 to the set of Python’s (including git head). Do you think there’s a better way to publicize these images? I know that Julien Palard wanted a docker image with all the versions of Python, see: https://github.com/docker-library/python/issues/373 For my part, I wanted to propose a docker image with the last version of Python and try to use it for the detection of bugs in the main python projects (django, numpy, flask, pandas, etc...) with a CI (example: Gitlab-CI) First issue: pytest uses the ast module of python and since 3.8.0a1, the tests do not pass -> new issue for pytest FWIW, we're preparing to rebuild all Fedora packages with the 3.8 alphas/betas, so everything's tested when 3.8.0 is released: https://fedoraproject.org/wiki/Changes/Python3.8 Hi Petr, Will you execute the tests of these packages? I have a small discussion with Julien Palard and I wanted to create a small CI where I will execute the tests of the updated packages from the RSS feed of PyPI. The first one was pytest -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 2/6/19 2:26 PM, Matthias Klose wrote: On 06.02.19 13:23, Petr Viktorin wrote: FWIW, we're preparing to rebuild all Fedora packages with the 3.8 alphas/betas, so everything's tested when 3.8.0 is released: https://fedoraproject.org/wiki/Changes/Python3.8 That should cover the main Python projects, too. well, the real challenge is that all test suites of third party packages still pass on all architectures. From past transitions, I know that this costs the most time and resources. Same experience here. In Fedora, tests are generally run as part of the build. (Sorry, that was definitely not obvious from my message!) But yes, targeting 3.8 for Ubuntu 20.04 LTS as well. \o/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 06.02.19 13:23, Petr Viktorin wrote: > FWIW, we're preparing to rebuild all Fedora packages with the 3.8 > alphas/betas, > so everything's tested when 3.8.0 is released: > https://fedoraproject.org/wiki/Changes/Python3.8 > > That should cover the main Python projects, too. well, the real challenge is that all test suites of third party packages still pass on all architectures. From past transitions, I know that this costs the most time and resources. But yes, targeting 3.8 for Ubuntu 20.04 LTS as well. Matthias ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 2/6/19 8:43 AM, Stephane Wirtel wrote: On 02/05, Barry Warsaw wrote: On Feb 5, 2019, at 02:24, Stephane Wirtel wrote: You’re welcome! I just pushed an update to add 3.8.0a1 to the set of Python’s (including git head). Do you think there’s a better way to publicize these images? I know that Julien Palard wanted a docker image with all the versions of Python, see: https://github.com/docker-library/python/issues/373 For my part, I wanted to propose a docker image with the last version of Python and try to use it for the detection of bugs in the main python projects (django, numpy, flask, pandas, etc...) with a CI (example: Gitlab-CI) First issue: pytest uses the ast module of python and since 3.8.0a1, the tests do not pass -> new issue for pytest FWIW, we're preparing to rebuild all Fedora packages with the 3.8 alphas/betas, so everything's tested when 3.8.0 is released: https://fedoraproject.org/wiki/Changes/Python3.8 That should cover the main Python projects, too. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 02/05, Barry Warsaw wrote: On Feb 5, 2019, at 02:24, Stephane Wirtel wrote: You’re welcome! I just pushed an update to add 3.8.0a1 to the set of Python’s (including git head). Do you think there’s a better way to publicize these images? I know that Julien Palard wanted a docker image with all the versions of Python, see: https://github.com/docker-library/python/issues/373 For my part, I wanted to propose a docker image with the last version of Python and try to use it for the detection of bugs in the main python projects (django, numpy, flask, pandas, etc...) with a CI (example: Gitlab-CI) First issue: pytest uses the ast module of python and since 3.8.0a1, the tests do not pass -> new issue for pytest Cheers, Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On Feb 5, 2019, at 02:24, Stephane Wirtel wrote: > > I was not aware of this image. So it's true that it's very useful. > > Thank you very much, You’re welcome! I just pushed an update to add 3.8.0a1 to the set of Python’s (including git head). Do you think there’s a better way to publicize these images? Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
Hi Barry, I was not aware of this image. So it's true that it's very useful. Thank you very much, Stéphane On 02/04, Barry Warsaw wrote: On Feb 4, 2019, at 05:02, Stephane Wirtel wrote: Just one idea, we could create a Docker image with this alpha version. This Docker image could be used with the CI of the main projects and the test suites of these projects. If we have some issues, we should create an issue for python 3.8.0a1. The time machine strikes again! https://gitlab.com/python-devs/ci-images/tree/master We call these “semi-official”! The current image takes a slightly different approach, by including all the latest Python versions from 2.7, and 3.4-3.8, plus git head. I just pushed an update for the latest Python 3.8 alpha and 3.7.2. It’s building now, but the image should be published on quay.io as soon as that’s done. Contributions most welcome! -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/stephane%40wirtel.be -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On Feb 4, 2019, at 05:02, Stephane Wirtel wrote: > > Just one idea, we could create a Docker image with this alpha version. > > This Docker image could be used with the CI of the main projects and the > test suites of these projects. > > If we have some issues, we should create an issue for python 3.8.0a1. The time machine strikes again! https://gitlab.com/python-devs/ci-images/tree/master We call these “semi-official”! The current image takes a slightly different approach, by including all the latest Python versions from 2.7, and 3.4-3.8, plus git head. I just pushed an update for the latest Python 3.8 alpha and 3.7.2. It’s building now, but the image should be published on quay.io as soon as that’s done. Contributions most welcome! -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
Hi Łukasz, I have some issues with pytest and this release, you can see this BPO https://bugs.python.org/issue35895 Have a nice day and thank you for your job. Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 02/04, Stephane Wirtel wrote: It's unofficial but I used the Dockerfile for 3.7 and created this Docker image: https://cloud.docker.com/u/matrixise/repository/docker/matrixise/python Sorry: here is the right link https://hub.docker.com/r/matrixise/python -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
It's unofficial but I used the Dockerfile for 3.7 and created this Docker image: https://cloud.docker.com/u/matrixise/repository/docker/matrixise/python docker pull matrixise/python:3.8.0a1 I am not an expert about the releasing of a Docker image but we could work with that and try to improve it. If one person use Gitlab-CI, this person can add a new test for this version and use this image (matrixise/python:3.8.0a1) or an official image, just for the tests... Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
On 02/04, Łukasz Langa wrote: I packaged my first release. *wipes sweat off of face* Go get it here: https://www.python.org/downloads/release/python-380a1/ Python 3.8.0a1 is the first of four planned alpha releases of Python 3.8, the next feature release of Python. During the alpha phase, Python 3.8 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next preview release, 3.8.0a2, is planned for 2019-02-24. Apart from building the Mac installers, Ned helped me a lot with the process, thank you! Ernest was super quick providing me with all required access and fixing a Unicode problem I found in Salt, thank you! Finally, this release was made on a train to Düsseldorf. There's a PyPy sprint there. The train is pretty cool, makes this "Wasm! Wasm!" sound. - Ł Hi Lukasz, Just one idea, we could create a Docker image with this alpha version. This Docker image could be used with the CI of the main projects and the test suites of these projects. If we have some issues, we should create an issue for python 3.8.0a1. Good idea? Have a nice day, Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing
I packaged my first release. *wipes sweat off of face* Go get it here: https://www.python.org/downloads/release/python-380a1/ Python 3.8.0a1 is the first of four planned alpha releases of Python 3.8, the next feature release of Python. During the alpha phase, Python 3.8 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next preview release, 3.8.0a2, is planned for 2019-02-24. Apart from building the Mac installers, Ned helped me a lot with the process, thank you! Ernest was super quick providing me with all required access and fixing a Unicode problem I found in Salt, thank you! Finally, this release was made on a train to Düsseldorf. There's a PyPy sprint there. The train is pretty cool, makes this "Wasm! Wasm!" sound. - Ł signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
