On 25.10.2021 21:40, byk...@gmail.com wrote: > Due to https://www.python.org/dev/peps/pep-0554/ multi-interpreters > implementation going really slow, I had the audicity to try an alternative > route > towards the same objective of implementing multicore support of python: > instead of sharing the memory by running multiple threads, I employed > an interprocess shared memory with multiple processes. > > I know there are multiprocessing.sharedctypes and > multiprocessing.shared_memory, > but I went much deeper into it by solving two problems they failed to solve: > sharing of complex dynamic objects and synchronization of data access. > > I have a working prototype to show: > https://github.com/byko3y/python-shared-objects > It's a kind of software transactional memory within shared memory. It has a > basic support > for fundamental python types (bool, int, str, bytes, tuple, list, dict, > object), > for both atomic and non-atomic transactions via fine-grained RW-locks, has a > built-in > protection against deadlock and starvation. > > Most of my code can also be used for cross-interpreter communication after > PEP 554 > is successfully implemented, since cross-interpreter communication is still > an open question.
This looks interesting. The 32-bit limitation is a bit of a bummer, but I suppose that can be lifted, right ? Some additional pointers for inspiration: - Here's an old project trying to do more or less the same: http://poshmodule.sourceforge.net/ - Another newer one, which is specific to numpy arrays: https://pypi.org/project/SharedArray/ - For more general purpose types, there's Apache Arrow's Plasma store: https://arrow.apache.org/docs/python/plasma.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Oct 27 2021) >>> Python Projects, Coaching and Support ... https://www.egenix.com/ >>> Python Product Development ... https://consulting.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/ _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XTDWVI4LHCBYP6LIRRINVXOZZOGI4OCY/ Code of Conduct: http://python.org/psf/codeofconduct/
