Hi Nickolai, Thanks for your suggestions, especially for the file system lock: We don't have often locks, but we must be sure it's locked.
For 1) and 4) suggestions, in fact we have several systems to sync and also a PostgreSQL transaction, the request must be treated by the same worker from beginning to end and the other systems aren't idempotent at all, it's "old-school" proprietary systems, good luck to change that ;-) Regards. -- Ludovic Gasc (GMLudo) 2018-04-17 12:46 GMT+02:00 Nickolai Novik <nickolaino...@gmail.com>: > Hi, redis lock has own limitations and depending on your use case it may > or may not be suitable [1]. If possible I would redefine problem and also > considered: > 1) create worker per specific resource type to avoid locking > 2) optimistic locking > 3) File system lock like in twisted, but not sure about performance and > edge cases there > 4) make operation on resource idempotent > > [1] http://martin.kleppmann.com/2016/02/08/how-to-do- > distributed-locking.html > [2] https://github.com/twisted/twisted/blob/e38cc25a67747899c6984d6ebaa8d3 > d134799415/src/twisted/python/lockfile.py > > On Tue, 17 Apr 2018 at 13:01 Ludovic Gasc <gml...@gmail.com> wrote: > >> Hi Roberto, >> >> Thanks for the pointer, it's exactly the type of feedbacks I'm looking >> for: Ideas that are out-of-box of my confort zone. >> However, in our use case, we are using gunicorn, that uses forks instead >> of multiprocessing to my knowledge, I can't use multiprocessing without to >> remove gunicorn. >> >> If somebody is using aioredlock in his project, I'm interested by >> feedbacks. >> >> Have a nice week. >> >> >> -- >> Ludovic Gasc (GMLudo) >> >> 2018-04-17 7:19 GMT+02:00 Roberto Martínez <robertomartin...@gmail.com>: >> >>> >>> Hi, >>> >>> I don't know if there is a third party solution for this. >>> >>> I think the closest you can get today using the standard library is >>> using a multiprocessing.manager().Lock (which can be shared among >>> processes) and call the lock.acquire() function with >>> asyncio.run_in_executor(), using a ThreadedPoolExecutor to avoid blocking >>> the asyncio event loop. >>> >>> Best regards, >>> Roberto >>> >>> >>> El mar., 17 abr. 2018 a las 0:05, Ludovic Gasc (<gml...@gmail.com>) >>> escribió: >>> >>>> Hi, >>>> >>>> I'm looking for a equivalent of asyncio.Lock ( >>>> https://docs.python.org/3/library/asyncio-sync.html#asyncio.Lock) but >>>> shared between several processes on the same server, because I'm migrating >>>> a daemon from mono-worker to multi-worker pattern. >>>> >>>> For now, the closest solution in term of API seems aioredlock: >>>> https://github.com/joanvila/aioredlock#aioredlock >>>> But I'm not a big fan to use polling nor with a timeout because the >>>> lock I need is very critical, I prefer to block the code than unlock with >>>> timeout. >>>> >>>> Do I miss a new awesome library or do you have an easier approach ? >>>> >>>> Thanks for your responses. >>>> -- >>>> Ludovic Gasc (GMLudo) >>>> >>> >>
_______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
