Hi Eric, Le ven. 17 avr. 2020 à 20:56, Eric Snow <ericsnowcurren...@gmail.com> a écrit : > With the 3.9 feature freeze coming up I'm considering options for PEP > 554. I'm hopeful to have a pronouncement from Antoine in the near > future. If that comes in time for 3.9, we will have the > implementation ready to go. It is already nearly complete and the > remaining work should be done in the next week or so.
I have concerns about shared data. The current implementation of CPython and subinterpreters still shares singletons. Moreover, I understand that the PEP proposes to first directly share PyObject between subinterpreters, like bytes objects. Isolating subinterpreters is a big task and so all issues cannot be solved at once. So I am not with making some compromises for a first milestone. The problem is to define which semantics can be exposed in the public API for the first milestone. -- Sharing directly singletons like None can become a performance kill once subinterpreters will run in parallel: https://bugs.python.org/issue39511 Mark Shannon summarized: "Having two CPUs write to the same cache line is a well known performance problem. There's nothing special about CPython here. The proper name for it seems to be "cache line ping-pong", but a search for "false sharing" might be more informative." The problem is that PyObject.ob_refcnt should be protected by atomic operations or locks if these objects are shared directly. I proposed to have singletons per interpreter, some people would prefer immortal singletons. -- For bytes or buffer objects, I understand that you propose to share the exact same PyObject objects between interpreters, at least in the first implementation. It may be better to have one proxy object in each interpreter which would control which interpreters can read and which interpreters can write into the object. It's a similar but more complex issue than singletons. -- What is the plan for Python 3.9? What semantics would be exposed in Python 3.9? Should we wait until these questions are answered before making the API public? Victor > I'm looking for feedback on a nagging worry I have with merging PEP > 554 into 3.9 (if it is accepted) without having the per-interpreter > GIL. See below. I'll update the PEP later with relevant info from > this discussion. > > wishing-you-were-all-here-ly yours, > > -eric > > +++++++++++++++++++++++++++++++++++++++++++++++++ (++++++++++ > > Dilemma > ============ > > Many folks have conflated PEP 554 with having a per-interpreter GIL. > In fact, I was careful to avoid any mention of parallelism or the GIL > in the PEP. Nonetheless some are expecting that when PEP 554 lands we > will reach multi-core nirvana. > > While PEP 554 might be accepted and the implementation ready in time > for 3.9, the separate effort toward a per-interpreter GIL is unlikely > to be sufficiently done in time. That will likely happen in the next > couple months (for 3.10). > > So...would it be sufficiently problematic for users if we land PEP 554 > in 3.9 without per-interpreter GIL? > > Options > ============ > > Here are the options as I see them (if the PEP is accepted in time for 3.9): > > 1. merge PEP 554 into 3.9 even if per-interpreter GIL doesn't get into > 3.9 (they get parallelism for free in 3.10) > 2. like 1, but mark the module as provisional until per-interpreter GIL lands > 3. do not merge PEP 554 until per-interpreter GIL is merged > 4. like 3, but publish a 3.9-only module to PyPI in the meantime > > Context > ============ > > Like I said, PEP 554 purposefully does not talk about parallelism or > the GIL. Instead it focuses strictly on the following (with an > emphasis on minimalism): > > * exposing the existing C-API > * supporting a new concurrency model (CSP) > > Regarding that first one, the value is in exposing the existing > functionality to a broader group of people. keep in mind that > subinterpreters currently have various other limitations (aside from > sharing the GIL): > > * nothing about them has been optimized (e.g. interpreter startup, > data sharing, data passing) > * extension modules with process-global state may break > * some bugs still remain in the corners > > Currently, there is a chicken-and-egg problem. It is unlikely that > there will be sufficient motivation to address those limitations until > the community starts really using subinterpreters. Hence PEP 554. > > My Position > ============ > > At this point I think we should go with option #1 (or possibly #2). > IMHO, we would be fine merging PEP 554 without per-interpreter GIL. > Here's why: > > * the two objectives of the PEP have value for the community (as > explained in the PEP) > * the sooner we can get the functionality of subinterpreters out to > the broader world, the better > * I don't think the lack of parallelism in 3.9 will trip anyone up > > My only concern is that folks try them out in 3.9, get frustrated by > the limitations, and then there is a mental barrier to trying them > again in the future when the limitations have been reduced or > eliminated. However, I don't think that will be a problem for people > that would benefit from serious use of subinterpreters. > > Furthermore, if we're clear that "for now" subinterpreters still share > the GIL in 3.9 then I'm not so worried about folks getting confused. > In fact, as long as we're clear about all the current limitations then > there isn't much downside to option #1. > > Marking the module as "provisional" may help communicate that there > are limitations that impact use. > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/3HVRFWHDMWPNR367GXBILZ4JJAUQ2STZ/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DBKP7XEG4TI5ATQOZ2HDD7XJWVOSETZ6/ Code of Conduct: http://python.org/psf/codeofconduct/
