Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

2018-04-17 Thread Roberto Martínez
Hi, I don't know if there is a third party solution for this. I think the closest you can get today using the standard library is using a multiprocessing.manager().Lock (which can be shared among processes) and call the lock.acquire() function with asyncio.run_in_executor(), using a ThreadedPoolE

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

2018-04-17 Thread Chris Jerdonek
If you're already using PostgreSQL, you might also look at "advisory locks": https://www.postgresql.org/docs/current/static/explicit-locking.html#ADVISORY-LOCKS --Chris On Tue, Apr 17, 2018 at 4:34 AM, Ludovic Gasc wrote: > Hi Nickolai, > > Thanks for your suggestions, especially for the file sy

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

2018-04-17 Thread Ludovic Gasc
Hi Nickolai, Thanks for your suggestions, especially for the file system lock: We don't have often locks, but we must be sure it's locked. For 1) and 4) suggestions, in fact we have several systems to sync and also a PostgreSQL transaction, the request must be treated by the same worker from begi

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

2018-04-17 Thread Nickolai Novik
Hi, redis lock has own limitations and depending on your use case it may or may not be suitable [1]. If possible I would redefine problem and also considered: 1) create worker per specific resource type to avoid locking 2) optimistic locking 3) File system lock like in twisted, but not sure about p

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

2018-04-17 Thread Ludovic Gasc
Hi Roberto, Thanks for the pointer, it's exactly the type of feedbacks I'm looking for: Ideas that are out-of-box of my confort zone. However, in our use case, we are using gunicorn, that uses forks instead of multiprocessing to my knowledge, I can't use multiprocessing without to remove gunicorn.