On 5 October 2017 at 18:45, Eric Snow <ericsnowcurren...@gmail.com> wrote:
> After we move to not sharing the GIL between interpreters: > > Channel.send(obj): # in interp A > incref(obj) > if type(obj).tp_share == NULL: > raise ValueError("not a shareable type") > set_owner(obj) # obj.owner or add an obj -> interp entry to global > table > ch.objects.append(obj) > > Channel.recv(): # in interp B > orig = ch.objects.pop(0) > obj = orig.tp_share() > set_shared(obj, orig) # add to a global table > return obj > This would be hard to get to work reliably, because "orig.tp_share()" would be running in the receiving interpreter, but all the attributes of "orig" would have been allocated by the sending interpreter. It gets more reliable if it's *Channel.send* that calls tp_share() though, but moving the call to the sending side makes it clear that a tp_share protocol would still need to rely on a more primitive set of "shareable objects" that were the permitted return values from the tp_share call. And that's the real pay-off that comes from defining this in terms of the memoryview protocol: Py_buffer structs *aren't* Python objects, so it's only a regular C struct that gets passed across the interpreter boundary (the reference to the original objects gets carried along passively as part of the CIV - it never gets *used* in the receiving interpreter). > bytes.tp_share(): > obj = blank_bytes(len(self)) > obj.ob_sval = self.ob_sval # hand-wavy memory sharing > return obj > This is effectively reinventing memoryview, while trying to pretend it's an ordinary bytes object. Don't reinvent memoryview :) > bytes.tp_free(): # under no-shared-GIL: > # most of this could be pulled into a macro for re-use > orig = lookup_shared(self) > if orig != NULL: > current = release_LIL() > interp = lookup_owner(orig) > acquire_LIL(interp) > decref(orig) > release_LIL(interp) > acquire_LIL(current) > # clear shared/owner tables > # clear/release self.ob_sval > free(self) > I don't think we should be touching the behaviour of core builtins solely to enable message passing to subinterpreters without a shared GIL. The simplest possible variant of CIVs that I can think of would be able to avoid that outcome by being a memoryview subclass, since they just need to hold the extra reference to the original interpreter, and include some logic to swtich interpreters at the appropriate time. That said, I think there's definitely a useful design question to ask in this area, not about bytes (which can be readily represented by a memoryview variant in the receiving interpreter), but about *strings*: they have a more complex internal layout than bytes objects, but as long as the receiving interpreter can make sure that the original string continues to exist, then you could usefully implement a "strview" type to avoid having to go through an encode/decode cycle just to pass a string to another subinterpreter. That would provide a reasonable compelling argument that CIVs *shouldn't* be implemented as memoryview subclasses, but instead defined as *containing* a managed view of an object owned by a different interpreter. That way, even if the initial implementation only supported CIVs that contained a memoryview instance, we'd have the freedom to define other kinds of views later (such as strview), while being able to reuse the same CIV machinery. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com