Hi Joel, > I'm planning to write a prototype using zeromq. I see in the > documentation that zeromq provides > a publisher/subscriber mode. I want to use this mode in my application, > but i'm asking how zeromq handles > slow consumer. > > Is there some documentation about this point ?
The documentation is lacking in this area :( However, the principle is pretty simple: You can think of the network as series of tubes. Each tube has some max capacity. For example, when communicating using 2 0MQ sockets there's senders 0MQ queue, kernel SNDBUF, queue in the NIC, queue in the switch, queue in the receiver's NIC, kernel RCVBUF, receiver's 0MQ queue... When consumer is slow these queues start growing. The process starts at the place of the bottleneck then propagates up back to the sender. Once all the queues are full, meaning that sender cannot send another message without exceeding the limits, some measure must be taken. The exact measure depends on the messaging pattern. For example for REQ/REP send function blocks in such a situation. For PUB/SUB the message is accepted and dropped. While the latter looks scary, it makes perfect sense because blocking would cause whole system to stop. Even innocent subscribers would not get messages. If you want to reasonably guarantee the delivery, the only way is to make the queues very large. We've seen queues set to max. 200GB to be able to hold whole trading day of market data. Martin _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
