[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-07 Thread Paul Moore
On Thu, 7 May 2020 at 01:34, Cameron Simpson wrote: > Maybe I'm missing something, but the example that comes to my mind is > embedding a Python interpreter in an existing nonPython programme. > > My pet one-day-in-the-future example is mutt, whose macro language is... > crude. And mutt is singl

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-07 Thread Emily Bowman
On Wed, May 6, 2020 at 12:36 PM Nathaniel Smith wrote: > > Sure, zero cost is always better than some cost, I'm not denying that > :-). What I'm trying to understand is whether the difference is > meaningful enough to justify subinterpreters' increased complexity, > fragility, and ecosystem break

[Python-Dev] for Glenn Linderman Re: Re: Improvement to SimpleNamespace

2020-05-07 Thread Cameron Simpson
Apologies to other list members. Glenn, we were having a conversation off list and there's no evidence my replies reached you. Could you have a glance in your spam (if you have such a thing) to see if my messages are lying there idle? From the 15th and 20th of April. GMail certainly seems to

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-07 Thread Eric Snow
On Thu, May 7, 2020 at 2:50 AM Emily Bowman wrote: > While large object copies are fairly fast -- I wouldn't say trivial, a > gigabyte copy will introduce noticeable lag when processing enough of them -- > the flip side of having large objects is that you want to avoid having so > many copies t

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-07 Thread Cody Piersall
On Tue, May 5, 2020 at 6:44 PM Joseph Jenne via Python-Dev wrote: > > I'm seeing a drop in performance of both multiprocess and subinterpreter > based runs in the 8-CPU case, where performance drops by about half > despite having enough logical CPUs, while the other cases scale quite > well. Is th

[Python-Dev] Re: Issues with import_fresh_module

2020-05-07 Thread Brett Cannon
Maybe an initialization/import side-effect bug which is triggered if the module is imported twice? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/pyt

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-07 Thread Gregory P. Smith
On Wed, May 6, 2020 at 1:14 PM Serhiy Storchaka wrote: > 06.05.20 00:46, Victor Stinner пише: > > Subinterpreters and multiprocessing have basically the same speed on > > this benchmark. > > It does not look like there are some advantages of subinterpreters > against multiprocessing. > There is

[Python-Dev] Deprecate os.removedirs() and os.renames()

2020-05-07 Thread Serhiy Storchaka
It seems to me that os.removedirs() and os.renames() was added just for symmetry with os.makedirs(). All three functions have similar structure and was added in the same commit. Seems they were initially code examples of using some os.path and os functions. Unlike to quite popular os.makedirs(

[Python-Dev] Re: Deprecate os.removedirs() and os.renames()

2020-05-07 Thread Kyle Stanley
Serhiy Storchaka wrote: > I propose to deprecate these functions and remove them in future Python versions. +1, assuming the deprecation lasts for at least two versions and the available alternatives are explicitly mentioned in the What's New entry (for both the version they're initially deprecate

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-07 Thread Victor Stinner
Le mer. 6 mai 2020 à 22:10, Serhiy Storchaka a écrit : > I am wondering how much 3.9 will be slower than 3.8 in single-thread > single-interpreter mode after getting rid of all process-wide singletons > and caches (Py_None, Py_True, Py_NonImplemented. small integers, > strings, tuples, _Py_IDENTIF

[Python-Dev] Re: Issues with import_fresh_module

2020-05-07 Thread Chris Jerdonek
To expand on my earlier comment about changing the module under test to make your testing easier, asyncio is one library that has lots of tests of different combinations of its C and Python implementations being used together. As far as I know, it doesn't use import_fresh_module or similar hackery

[Python-Dev] Re: Latest PEP 554 updates.

2020-05-07 Thread Jeff Allen
On 06/05/2020 21:52, Eric Snow wrote: On Wed, May 6, 2020 at 2:25 PM Jeff Allen wrote: ... My reason for worrying about this is that, while the C-API has been there for some time, it has not had heavy use in taxing cases AFAIK, and I think there is room for it to be incorrect. I am thinking m