On 10/21/20 1:40 PM, Mark Shannon wrote:
Hi Petr,
On 21/10/2020 11:49 am, Petr Viktorin wrote:
Let me explain an impression I'm getting. It is *just one aspect* of
my opinion, one that doesn't make sense to me. Please tell me where it
is wrong.
In the C API, there's a somewhat controversial refactoring going on,
which involves passing around tstate arguments. I'm not saying [the
first discussion] was perfect, and there are still issues, but,
however flawed the "do-ocracy" process is, it is the best way we found
to move forward. No one who can/wants to do the work has a better
solution.
Later, Mark says there is an even better way – or at least, a less
intrusive one! In [the second discussion], he hints at it vaguely
(from that limited info I have, it involves switching to C11 and/or
using compiler-specific extensions -- not an easy change to do). But
frustratingly, Mark doesn't reveal any actual details, and a lot of
the complaints are about churn and merge conflicts.
And now, there's news -- the better solution won't be revealed unless
the PSF pays for it!
There's no secret. C thread locals are well documented.
I even provided a code example last time we discussed it.
You reminded me of it yesterday ;)
https://godbolt.org/z/dpSo-Q
At the risk of going off topic: That's for GCC. As far as I know, MSVC
uses something like __declspec( thread ).
What are the options for generic C99 compilers, other than staying slow?
The "even faster" solution I mentioned yesterday, is as I stated
yesterday to use an aligned stack.
If you wanted more info, you could have asked :)
First, you ensure that the stack is in a 2**N aligned block.
Assuming that the C stack grows down from the top, then the threadstate
struct goes at the bottom. It's probably a good idea to put a guard page
between the C stack and the threadstate struct.
The struct's address can then be found by masking off the bottom N bits
from the stack pointer.
This approach uses 0 registers and cost 1 ALU instruction. Can't get
cheaper than that :)
It's not portable and probably a pain to implement, but it is fast.
But it doesn't matter how it's implemented. The implementation is hidden
behind `PyThreadState_GET()`, it can be changed to use a thread local,
or to some fancy aligned stack, without the rest of the codebase changing.
Not portable and hard to implement is a pain for maintenance –
especially porting CPython to new compilers/platforms/situations.
The alternative is changing the codebase, which (it seems from the
discussions) would give us comparable performance, everywhere, and the
result can be maintained by many more people.
That's a very bad situation to be in for having discussions:
basically, either we disregard Mark and go with the not-ideal
solution, or virtually all work on changing the C API and internal
structures is blocked.
The existence of multiple interpreters should be orthogonal to speeding
up those interpreters, provided the separation is clean and well designed.
But it should be clean and well designed anyway, IMO.
+1
I sense a similar thing happening here:
https://github.com/ericsnowcurrently/multi-core-python/issues/69 --
The title of that issue is 'Clarify what is a "sub-interpreter" and what
is an "interpreter"'?
there's a vague proposal to do things very differently, but I find it
This?
https://github.com/ericsnowcurrently/multi-core-python/issues/69#issuecomment-712837899
I'll continue there.
hard to find anything actionable. I would like to change my plans to
align with Mark's fork, or to better explain some of the
non-performance reasons for recent/planned changes. But I can't,
because details are behind a paywall.
Let's make this very clear.
My objections to the way multiple interpreters is being implemented has
very little to do speeding up the interpreter and entirely to do with
long term maintenance and ultimate success of the project.
Obviously, I would like it if multiple interpreters didn't slowdown
CPython.
But that has always been the case.
Thank you for clearing my doubts!
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/W5JF77HVMIJ3Q5RSL3R2TOJGZ4JEWJRS/
Code of Conduct: http://python.org/psf/codeofconduct/
