Thanks Tim. That's one unambiguous answer. I did consider posting to python-list, but this seemed somewhat python-devish. Any appetite for documenting which foundational functions are const-safe in the sense I described? Perhaps we could help. What kind of analysis, demand or momentum is needed for the Python dev team to consider introducing such a contract?
To be honest, I did do some CPython source code staring already. And at least for the functions we care about, it wasn't all that painful (PyDict_GetItem being the trickiest). Doing this wholesale might be a superhuman task, but I'm thinking a practical subset could be relatively straightforward while still useful, 80/20. On Tue, Jul 17, 2018 at 7:15 AM, Tim Peters <tim.pet...@gmail.com> wrote: > [Radim Řehůřek <ra...@rare-technologies.com>] > >> one of our Python projects calls for pretty heavy, low-level >> optimizations. > > >> We went down the rabbit hole and determined that having access to >> PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict, unicode) >> on Python objects **outside of GIL** might be a good-enough solution. The >> Python objects in question are guaranteed to live and not be mutated >> externally in any way. They're "frozen" and read-only. >> >> Under what conditions is it OK to call these 3 functions on such objects? >> > > From the "initialization, Finalization, and Threads" section of the > Python/C API Reference Manual: > > Therefore, the rule exists that only the thread that has acquired the GIL >> may operate on Python objects or call Python/C API functions. > > > Under protection of the GIL is the only documented - or intended - way to > do anything with Python objects, or to call virtually any Python/C API > function. > > > >> More generally, what is the CPython 2.7/3.5 contract regarding (lack of) >> object mutation, and the need for reference counting and synchronization >> via GIL? >> > > There is no intention to support GIL-free access to any Python objects. > So that's the contract: "All warranties are null & void if you do just > about anything while not holding the GIL". > > >> Which C API functions are safe to call on "const" objects? >> > > None. If you find some combinations of CPython versions, functions, and > objects, that happen to work, that's fine, but there's no guarantee they'll > continue to work, not even across maintenance releases. Although they'll > _probably_ continue to work. > > Obviously releasing GIL and then calling C API is hacky, but from initial >> experiments, it seems to work (see https://stackoverflow.com/ >> questions/51351609/can-i-const-access-cpython-objects-without-gil). But >> I'm wondering if there's a more formal contract around this behaviour. >> > > Nope! You're on your own here. > > Which doesn't mean you can't do it. Just that if things blow up, you're > still on your own - since you're far outside what the documentation says is > required (which I suspect you knew before you asked ;-) ), the project > won't even accept a bug report. > > If you want more help trying to guess which functions _may_ work outside > the law, that's fine too, but "python-dev" (this mailing list) is for > development _of_ Python itself, not for questions about _using_ Python. > The more general python-list would be more appropriate. But, since you're > trying to use the C API in unsupported ways it wasn't intended to be used, > you'll likely have a hard time finding people with significant experience > doing the same thing. Since it's not at all an intended use, there are no > "general principles" at work to ease the pain of staring at the CPython > source code to try to guess what might go wrong across all code paths. >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
