- self.cond.wait() + await self.cond.wait() I've no tests for this :P
On 26 June 2017 at 21:37, Dima Tisnek <dim...@gmail.com> wrote: > Chris, here's a simple RWLock implementation and analysis: > > ``` > import asyncio > > > class RWLock: > def __init__(self): > self.cond = asyncio.Condition() > self.readers = 0 > self.writer = False > > async def lock(self, write=False): > async with self.cond: > # write requested: there cannot be readers or writers > # read requested: there can be other readers but not writers > while self.readers and write or self.writer: > self.cond.wait() > if write: self.writer = True > else: self.readers += 1 > # self.cond.notifyAll() would be good taste > # however no waiters can be unblocked by this state change > > async def unlock(self, write=False): > async with self.cond: > if write: self.writer = False > else: self.readers -= 1 > self.cond.notifyAll() # notify (one) could be used `if not > write:` > ``` > > Note that `.unlock` cannot validate that it's called by same coroutine > as `.lock` was. > That's because there's no concept for "current_thread" for coroutines > -- there can be many waiting on each other in the stack. > > Obv., this code could be nicer: > * separate context managers for read and write cases > * .unlock can be automatic (if self.writer: unlock_for_write()) at the > cost of opening doors wide open to bugs > * policy can be introduced if `.lock` identified itself (by an > object(), since there's no thread id) in shared state > * notifyAll() makes real life use O(N^2) for N being number of > simultaneous write lock requests > > Feel free to use it :) > > > > On 26 June 2017 at 20:21, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >> On Mon, Jun 26, 2017 at 10:02 AM, Dima Tisnek <dim...@gmail.com> wrote: >>> Chris, coming back to your use-case. >>> Do you want to synchronise side-effect creation/deletion for the >>> sanity of side-effects only? >>> Or do you imply that callers' actions are synchronised too? >>> In other words, do your callers use those directories out of band? >> >> If I understand your question, the former. The callers aren't / need >> not be synchronized, and they aren't aware of the underlying >> synchronization happening inside the higher-level create() and >> delete() functions they would be using. (These are the two >> higher-level functions described in my pseudocode.) >> >> The synchronization is needed inside these create() and delete() >> functions since the low-level directory operations occur in different >> threads (because they are wrapped by run_in_executor()). >> >> --Chris >> >>> >>> >>> P.S./O.T. when it comes to directories, you probably want hierarchical >>> locks rather than RW. >>> >>> >>> On 26 June 2017 at 11:28, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >>>> On Mon, Jun 26, 2017 at 1:43 AM, Dima Tisnek <dim...@gmail.com> wrote: >>>>> Perhaps you can share your use-case, both as pseudo-code and a link to >>>>> real code. >>>>> >>>>> I'm specifically interested to see why/where you'd like to use a >>>>> read-write async lock, to evaluate if this is something common or >>>>> specific, and if, perhaps, some other paradigm (like queue, worker >>>>> pool, ...) may be more useful in general case. >>>>> >>>>> I'm also curious if a full set of async sync primitives may one day >>>>> lead to async monitors. Granted, simple use of async monitor is really >>>>> a future/promise, but perhaps there are complex use cases in the >>>>> UI/react domain with its promise/stream dichotomy. >>>> >>>> Thank you, Dima. In my last email I shared pseudo-code for an approach >>>> to read-write synchronization that is independent of use case. [1] >>>> >>>> For the use case, my original purpose in mind was to synchronize many >>>> small file operations on disk like creating and removing directories >>>> that possibly share intermediate segments. The real code isn't public. >>>> But these would be operations like os.makedirs() and os.removedirs() >>>> that would be wrapped by loop.run_in_executor() to be non-blocking. >>>> The directory removal using os.removedirs() is the operation I thought >>>> should require exclusive access, so as not to interfere with directory >>>> creations in progress. >>>> >>>> Perhaps a simpler, dirtier approach would be not to synchronize at all >>>> and simply retry directory creations that fail until they succeed. >>>> That could be enough to handle rare cases where simultaneous creation >>>> and removal causes an error. You could view this an EAFP approach. >>>> >>>> Either way, I think the process of thinking through patterns for >>>> read-write synchronization is helpful for getting a better general >>>> feel and understanding of async. >>>> >>>> --Chris >>>> >>>> >>>>> >>>>> Cheers, >>>>> d. >>>>> >>>>> On 25 June 2017 at 23:13, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >>>>>> I'm relatively new to async programming in Python and am thinking >>>>>> through possibilities for doing "read-write" synchronization. >>>>>> >>>>>> I'm using asyncio, and the synchronization primitives that asyncio >>>>>> exposes are relatively simple [1]. Have options for async read-write >>>>>> synchronization already been discussed in any detail? >>>>>> >>>>>> I'm interested in designs where "readers" don't need to acquire a lock >>>>>> -- only writers. It seems like one way to deal with the main race >>>>>> condition I see that comes up would be to use loop.time(). Does that >>>>>> ring a bell, or might there be a much simpler way? >>>>>> >>>>>> Thanks, >>>>>> --Chris >>>>>> >>>>>> >>>>>> [1] https://docs.python.org/3/library/asyncio-sync.html >>>>>> _______________________________________________ >>>>>> Async-sig mailing list >>>>>> Async-sig@python.org >>>>>> https://mail.python.org/mailman/listinfo/async-sig >>>>>> Code of Conduct: https://www.python.org/psf/codeofconduct/ _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
