Dear all, I thought you'd find this entertaining or possibly even useful as a fun-with-0MQ example: I hacked support for dispatching 0MQ PUB messages from a Redis server whenever a Redis key (string only for now) expires.
It's quite limited functionality, should probably be changed to support all Redis types, and likely stands no chance of being incorporated upstream, though. It does what I need it to do (extremely high read/write load session cache which expires old sessions into a cold storage, a fat mysql, from where they can be promoted back into the hot cache if necessary). https://github.com/tsee/redis/tree/msg_on_expire One detail that I am still wondering about is the type of socket to use: - There will be many Redis instances on many servers (it's single-threaded and did I say there's extremely high read/write load?). - Eventhough the hot cache is not write-through (yay!), I expect a rather high insert load on mysql, so expect to require more than one worker that inserts expired sessions into the database. - All the Redis instances have slaves for resiliency. - This needs to be able to fail over elegantly without losing too much data. Thus => I will probably switch to PUSH in the expiry hook with multiple insert workers PULLing. => I could have a worker on the same machine as Redis, one worker per Redis instance, but while that would provide more than enough insert throughput, it doesn't allow batching of inserts as much for improved database load. => How can the 0MQ mesh of connections recover if a Redis instance fails over to the slave? ==> Redis should reconnect. But of a worker is added/removed, the worker needs to reconnect. Who should be binding instead of connecting? What's the general strategy for resilience and failover in such setups with 0MQ? BTW: Did I miss a bit of the 0MQ API that allows checking for dropped connections to trigger reconnects? --Steffen _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
