Hi, Thanks for replying. > One thing that is worth thinking about is the safety of the API that > is put together. A memory segment plus a separate detached semaphore > or mutex can be used to build a safe API, but is not itself a safe > API.
Agreed. That’s why I am more inclined to the second solution that I mentioned. > For instance, we could have an object representing a memory range that > doesn't offer read/write at all, but allows: > - either one process write access over the range > - or any number of readers read access over the range > - allows subdividing the range (so that you can e.g. end one write > lock and keep another) Where will this memory object be stored ? Locking a particular range instead of the whole memory segment will be relatively efficient because processes using different ranges can write simultaneously. Since, this object will also be shared across multiple processes, there must be a safe way to update it. Any thoughts on that ? > On 27-Jul-2020, at 3:50 PM, Robert Collins <robe...@robertcollins.net> wrote: > > On Sun, 26 Jul 2020 at 19:11, Vinay Sharma via Python-ideas > <python-ideas@python.org> wrote: >> >> Problem: >> Currently, let’s say I create a shared_memory segment using >> mulitprocessing.shared_memory.SharedMemory in Process 1 and open the same in >> Process 2. >> Then, I try to write some data to the shared memory segment using both the >> processes, so for me to prevent any race condition (data corruption), either >> these operations must be atomic, or I should be able to lock / unlock shared >> memory segment, which I cannot at the moment. >> >> I earlier posted a solution to this problem, which received positive >> response, but there weren’t many responses to it, despite the fact this >> problem makes shared_memory practically unusable if there are simultaneous >> writes. >> So, the purpose of this post is to have discussion about the solution of the >> same. > > One thing that is worth thinking about is the safety of the API that > is put together. A memory segment plus a separate detached semaphore > or mutex can be used to build a safe API, but is not itself a safe > API. > > A safe API shouldn't allow writes to the memory segment while the > mutex is unlocked, rather than allowing one to build a safe API from > the various pieces. (There may / will be lower level primitives that > are unsafe). > > We can look at a lot of the APIs in the Rust community for examples of > this sort of thing. > > Python doesn't have the borrow checker to enforce usage, but we could > still work from the same basic principle - given there are multiple > processes involved that make it easier to have safe outcomes. > > For instance, we could have an object representing a memory range that > doesn't offer read/write at all, but allows: > - either one process write access over the range > - or any number of readers read access over the range > - allows subdividing the range (so that you can e.g. end one write > lock and keep another) > > For instance, > https://doc.rust-lang.org/std/vec/struct.Vec.html#method.split_at_mut > is an in-process API that is very similar. > > -Rob _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZD4XWNVI3QAG22MK7473B65FEQBLF4LM/ Code of Conduct: http://python.org/psf/codeofconduct/
