If you're already using PostgreSQL, you might also look at "advisory locks": https://www.postgresql.org/docs/current/static/explicit-locking.html#ADVISORY-LOCKS
--Chris On Tue, Apr 17, 2018 at 4:34 AM, Ludovic Gasc <gml...@gmail.com> wrote: > Hi Nickolai, > > Thanks for your suggestions, especially for the file system lock: We don't > have often locks, but we must be sure it's locked. > > For 1) and 4) suggestions, in fact we have several systems to sync and also > a PostgreSQL transaction, the request must be treated by the same worker > from beginning to end and the other systems aren't idempotent at all, it's > "old-school" proprietary systems, good luck to change that ;-) > > Regards. > -- > Ludovic Gasc (GMLudo) > > 2018-04-17 12:46 GMT+02:00 Nickolai Novik <nickolaino...@gmail.com>: >> >> Hi, redis lock has own limitations and depending on your use case it may >> or may not be suitable [1]. If possible I would redefine problem and also >> considered: >> 1) create worker per specific resource type to avoid locking >> 2) optimistic locking >> 3) File system lock like in twisted, but not sure about performance and >> edge cases there >> 4) make operation on resource idempotent >> >> [1] >> http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html >> [2] >> https://github.com/twisted/twisted/blob/e38cc25a67747899c6984d6ebaa8d3d134799415/src/twisted/python/lockfile.py >> >> On Tue, 17 Apr 2018 at 13:01 Ludovic Gasc <gml...@gmail.com> wrote: >>> >>> Hi Roberto, >>> >>> Thanks for the pointer, it's exactly the type of feedbacks I'm looking >>> for: Ideas that are out-of-box of my confort zone. >>> However, in our use case, we are using gunicorn, that uses forks instead >>> of multiprocessing to my knowledge, I can't use multiprocessing without to >>> remove gunicorn. >>> >>> If somebody is using aioredlock in his project, I'm interested by >>> feedbacks. >>> >>> Have a nice week. >>> >>> >>> -- >>> Ludovic Gasc (GMLudo) >>> >>> 2018-04-17 7:19 GMT+02:00 Roberto Martínez <robertomartin...@gmail.com>: >>>> >>>> >>>> Hi, >>>> >>>> I don't know if there is a third party solution for this. >>>> >>>> I think the closest you can get today using the standard library is >>>> using a multiprocessing.manager().Lock (which can be shared among >>>> processes) >>>> and call the lock.acquire() function with asyncio.run_in_executor(), using >>>> a >>>> ThreadedPoolExecutor to avoid blocking the asyncio event loop. >>>> >>>> Best regards, >>>> Roberto >>>> >>>> >>>> El mar., 17 abr. 2018 a las 0:05, Ludovic Gasc (<gml...@gmail.com>) >>>> escribió: >>>>> >>>>> Hi, >>>>> >>>>> I'm looking for a equivalent of asyncio.Lock >>>>> (https://docs.python.org/3/library/asyncio-sync.html#asyncio.Lock) but >>>>> shared between several processes on the same server, because I'm >>>>> migrating a >>>>> daemon from mono-worker to multi-worker pattern. >>>>> >>>>> For now, the closest solution in term of API seems aioredlock: >>>>> https://github.com/joanvila/aioredlock#aioredlock >>>>> But I'm not a big fan to use polling nor with a timeout because the >>>>> lock I need is very critical, I prefer to block the code than unlock with >>>>> timeout. >>>>> >>>>> Do I miss a new awesome library or do you have an easier approach ? >>>>> >>>>> Thanks for your responses. >>>>> -- >>>>> Ludovic Gasc (GMLudo) >>> >>> > > > _______________________________________________ > Async-sig mailing list > Async-sig@python.org > https://mail.python.org/mailman/listinfo/async-sig > Code of Conduct: https://www.python.org/psf/codeofconduct/ > _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/