I’d like to write a circuit breaker wrapper to use with services that I call from inside a Django app, and I’d like to get some pointers on the following questions:
1. What’s the most global, persisting, shared-across-requests type of data I can read & write to memory? 2. However I can do this, does this require using `threading.Lock()`? (looking around `dispatch.Signal` I see some use of it) 3. If this data is shared per-process, what’s a good way to see how many processes it would be (for uwsgi is that just the # of worker processes)? The term “circuit breaker” is referring to service clients that cut themselves off from making future requests to a service after the service has failed to connect or read/write after a certain error threshold. They can employ retries to eventually right themselves. If a service continues to fail, even despite strict timeouts, it can significantly slow down a site or even cause the webserver to run out of threads and it will effectively reach a DoS scenario. The clients are per-webserver (or maybe in this case, per-process) and all independently can open or close their “circuits” based on what they’re experiencing. To effectively track the effectiveness of service calls without having a performance impact on the webserver, it really needs to store this info in memory—which is the source of my original questions—and I assume this means I’m stuck with only info per-process? I’m OK with the info disappearing after server restarts, but might there be some gotchas around other configurations, like the `--harakiri` option for uwsgi? As stated above, I’m also a little unclear on when using threading locks would be required vs not, given that python uses the GIL—but maybe certain configurations do allow for non-thread-safe things to happen? I found 2 barebones approaches on github, but the correctness/effectiveness of each was quite unclear: - https://github.com/cuker/django-patchboard/blob/master/patchboard/circuitbreaker.py - https://github.com/globocom/memcached_memoize/blob/master/memcached_memoize/decorators/circuit_breaker.py Any help would be appreciated—whatever I build for this would definitely be open-sourced to the community! -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/35f791f2-d857-44de-a9e7-e52e346728b9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
