On 7 September 2017 at 15:48, Nathaniel Smith <n...@pobox.com> wrote: > I've actually argued with the PyPy devs to try to convince them to add > subinterpreter support as part of their experiments with GIL-removal, > because I think the semantics would genuinely be nicer to work with > than raw threads, but they're convinced that it's impossible to make > this work. Or more precisely, they think you could make it work in > theory, but that it would be impossible to make it meaningfully more > efficient than using multiple processes. I want them to be wrong, but > I have to admit I can't see a way to make it work either...
The gist of the idea is that with subinterpreters, your starting point is multiprocessing-style isolation (i.e. you have to use pickle to transfer data between subinterpreters), but you're actually running in a shared-memory threading context from the operating system's perspective, so you don't need to rely on mmap to share memory over a non-streaming interface. It's also definitely the case that to make this viable, we'd need to provide fast subinterpreter friendly alternatives to C globals for use by extension modules (otherwise adding subinterpreter compatibility will be excessively painful), and PEP 550 is likely to be helpful there. Personally, I think it would make sense to add the module under PEP 411 provisional status, and make it's continued existence as a public API contingent on actually delivering on the "lower overhead multi-core support than multiprocessing" goal (even if it only delivers on that front on Windows, where process creation is more expensive and there's no fork() equivalent). However, I'd also be entirely happy with our adding it as a private "_subinterpreters" API for testing & experimentation purposes (see https://bugs.python.org/issue30439 ), and reconsidering introducing it as a public API after there's more concrete evidence as to what can actually be achieved based on it. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
