[Python-Dev] Re: Latest PEP 554 updates.
On 06/05/2020 21:52, Eric Snow wrote: On Wed, May 6, 2020 at 2:25 PM Jeff Allen wrote: ... My reason for worrying about this is that, while the C-API has been there for some time, it has not had heavy use in taxing cases AFAIK, and I think there is room for it to be incorrect. I am thinking more about Jython than CPython, but ideally they are the same structures. When I put the structures to taxing use cases on paper, they don't seem quite to work. Jython has been used in environments with thread-pools, concurrency, and multiple interpreters, and this aspect has had to be "fixed" several times. That insight would be super helpful and much appreciated. :) Is that all on the docs you've linked? As far as it goes. I intended to (will eventually) elaborate the more complex cases, such as concurrency and application server, where I think a Thread may have "history" in a runtime that should be ignored. There's more on my local repo, but not about this yet. I have linked you into one page of a large and rambling (at times) account of experiments I'm doing. Outside be dragons. The other thing I might point to would be Jython bugs that may be clues something is still wrong conceptually, or at least justify getting those concepts clear (https://bugs.jython.org issues 2642, 2507, 2513, 2846, 2465, 2107 to name a few). This is great stuff, Jeff! Thanks for sharing it. I was able to skim through but don't have time to dig in at the moment. I'll reply in detail as soon as I can. Thanks. I hope it's a positive contribution. Isn't PlantUML awesome? The key argument (or where I'm mistaken) is that, once you start sharing objects, only the function you call knows the right Interpreter (import context) to use, so in principle, it is different in every frame. You can't get to it from the current thread. Jeff ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OGMJULX5RIVP2GFIX3G2TAUZAYQKAA5D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
On 05/05/2020 16:45, Eric Snow wrote: On Mon, May 4, 2020 at 11:30 AM Eric Snow wrote: Further feedback is welcome, though I feel like the PR is ready (or very close to ready) for pronouncement. Thanks again to all. FYI, after consulting with the steering council I've decided to change the target release to 3.10, when we expect to have per-interpreter GIL landed. That will help maximize the impact of the module and avoid any confusion. I'm undecided on releasing a 3.9-only module on PyPI. If I do it will only be for folks to try it out early and I probably won't advertise it much. -eric Eric: Many thanks for working on this so carefully for so long. I'm happy to see the per-interpreter GIL will now be studied fully before final commitment to subinterpreters in the stdlib. I would have chipped in in those terms to the review, but others succesfully argued for "provisional" inclusion, and I was content with that. My reason for worrying about this is that, while the C-API has been there for some time, it has not had heavy use in taxing cases AFAIK, and I think there is room for it to be incorrect. I am thinking more about Jython than CPython, but ideally they are the same structures. When I put the structures to taxing use cases on paper, they don't seem quite to work. Jython has been used in environments with thread-pools, concurrency, and multiple interpreters, and this aspect has had to be "fixed" several times. My use cases include sharing objects between interpreters, which I know the PEP doesn't. The C-API docs acknowledge that object sharing can't be prevented, but do their best to discourage it because of the hazards around allocation. Trouble is, I think it can happen unawares. The fact that Java takes on lifecycle management suggests it shouldn't be a fundamental problem in Jython. I know from other discussion it's where many would like to end up, even in CPython. This is all theory: I don't have even a model implementation, so I won't pontificate. However, I do have pictures, without which I find it impossible to think about this subject. I couldn't find your pictures, so share mine here (WiP): https://the-very-slow-jython-project.readthedocs.io/en/latest/architecture/interpreter-structure.html#runtime-thread-and-interpreter-cpython I would be interested in how you solve the problem of finding the current interpreter, discussed in the article. My preferred answer is: https://the-very-slow-jython-project.readthedocs.io/en/latest/architecture/interpreter-structure.html#critical-structures-revisited That's the API change I think is needed. It might not have a visible effect on the PEP, but it's worth bearing in mind the risk of exposing a thing you might shortly find you want to change. Jeff Allen ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E2BMM2IVKMDJGWOWQWCSDZCNPZOKEJMJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
On Wed, May 6, 2020 at 2:25 PM Jeff Allen wrote: > Many thanks for working on this so carefully for so long. I'm happy to see > the per-interpreter GIL will now be studied fully before final commitment to > subinterpreters in the stdlib. I would have chipped in in those terms to the > review, but others succesfully argued for "provisional" inclusion, and I was > content with that. No problem. :) > My reason for worrying about this is that, while the C-API has been there for > some time, it has not had heavy use in taxing cases AFAIK, and I think there > is room for it to be incorrect. I am thinking more about Jython than CPython, > but ideally they are the same structures. When I put the structures to taxing > use cases on paper, they don't seem quite to work. Jython has been used in > environments with thread-pools, concurrency, and multiple interpreters, and > this aspect has had to be "fixed" several times. That insight would be super helpful and much appreciated. :) Is that all on the docs you've linked? > My use cases include sharing objects between interpreters, which I know the > PEP doesn't. The C-API docs acknowledge that object sharing can't be > prevented, but do their best to discourage it because of the hazards around > allocation. Trouble is, I think it can happen unawares. The fact that Java > takes on lifecycle management suggests it shouldn't be a fundamental problem > in Jython. I know from other discussion it's where many would like to end up, > even in CPython. Yeah, for now we will strictly disallow sharing actual objects between interpreters in Python Code. It would be an interesting project to try loosening that at some point (especially with immutable type), but we're going to start from the safer position. We have no plans to add any similar restrictions to the C-API, where by you're typically much more free to shoot your own foot. :) > This is all theory: I don't have even a model implementation, so I won't > pontificate. However, I do have pictures, without which I find it impossible > to think about this subject. I couldn't find your pictures, so share mine > here (WiP): > > https://the-very-slow-jython-project.readthedocs.io/en/latest/architecture/interpreter-structure.html#runtime-thread-and-interpreter-cpython > > I would be interested in how you solve the problem of finding the current > interpreter, discussed in the article. My preferred answer is: > > https://the-very-slow-jython-project.readthedocs.io/en/latest/architecture/interpreter-structure.html#critical-structures-revisited > > That's the API change I think is needed. It might not have a visible effect > on the PEP, but it's worth bearing in mind the risk of exposing a thing you > might shortly find you want to change. This is great stuff, Jeff! Thanks for sharing it. I was able to skim through but don't have time to dig in at the moment. I'll reply in detail as soon as I can. In the meantime, the implementation of PEP 554 exposes a single part of PyInterpreterState: the ID (an int). The only other internal-ish info we expose is whether or not an interpreter (by ID) is currently running. The only functionality we provide is: create, destroy, and run_string(). -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7RZCIKVRIKXTNFT7IRNLA3OQ5CX2AIJ6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
On Mon, May 4, 2020 at 11:30 AM Eric Snow wrote: > Further feedback is welcome, though I feel like the PR is ready (or > very close to ready) for pronouncement. Thanks again to all. FYI, after consulting with the steering council I've decided to change the target release to 3.10, when we expect to have per-interpreter GIL landed. That will help maximize the impact of the module and avoid any confusion. I'm undecided on releasing a 3.9-only module on PyPI. If I do it will only be for folks to try it out early and I probably won't advertise it much. -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PZO7ZQB7OAOEJ7AXMJNMDKZC3B3UVDZA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
> On May 4, 2020, at 10:30 AM, Eric Snow wrote: > > Further feedback is welcome, though I feel like the PR is ready (or > very close to ready) for pronouncement. Thanks again to all. Congratulations. Regardless of the outcome, you've certainly earned top marks for vision, tenacity, team play, and overcoming adversity. May your sub-interpreters be plentiful, Raymond ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TD26ZW2EKO2Q4OFHRHEEF2MQPLXAGHL6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
On Mon, May 4, 2020 at 1:22 PM Paul Moore wrote: > One thing I would like to see is a comment confirming that as part of > the implementation, all stdlib modules will be made > subinterpreter-safe. Yeah, I'd meant to put a note. I'll add one. Thanks! -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CVGSOKANMSIGPZKRL6IQDOYJYZRZ3NE2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
On Mon, 4 May 2020 at 19:19, Eric Snow wrote: > > On Mon, May 4, 2020 at 11:30 AM Eric Snow wrote: > > Further feedback is welcome, though I feel like the PR is ready (or > > very close to ready) for pronouncement. Thanks again to all. > > oops > > s/the PR is ready/the PEP is ready/ One thing I would like to see is a comment confirming that as part of the implementation, all stdlib modules will be made subinterpreter-safe. I know we talked about this, and you seemed OK with the idea, so if it's already in the document I apologise. (I'm reading the raw rst in a proportional font - not the best way of making sure I don't miss things! :-() Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/23RWSNSFVRTC4MCCRWSRZVTAHW4YFIZX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Latest PEP 554 updates.
On Mon, May 4, 2020 at 11:30 AM Eric Snow wrote: > Further feedback is welcome, though I feel like the PR is ready (or > very close to ready) for pronouncement. Thanks again to all. oops s/the PR is ready/the PEP is ready/ -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/D36JNZMMA2O746KLWBBWMGN2F7MQHVWM/ Code of Conduct: http://python.org/psf/codeofconduct/
