So are you saying I can't have this now? :) This sounds like a generalization of the approach I have been thinking about, using ZK for coordination.
On Feb 23, 2011, at 3:39 PM, Pieter Hintjens wrote: > Hi Michael, > > You're absolutely right to say that sequencing won't work if the > subscriber uses a filter, which most will. > > Your concept of channel matches the stream concept I've explained > here: http://zeromq.org/osdp, that may give you some more ideas. It's > a product we hope to build over time, with input from clients and > users. > > Regards > - > Pieter Hintjens > iMatix > > On Wed, Feb 23, 2011 at 10:24 PM, Michael Kogan > <[email protected]> wrote: >> I don't need really guaranteed delivery, what I need is to be able to know >> if I am missing messages. The messages in my case represent data that is >> durably stored elsewhere, so I don't need a durable message store ala AMQP. >> The per-publisher sequence numbers are reasonable in fact that is the best I >> could come up with as well. >> Still, if subscribers are not subscribing to all messages, but using the >> first message of multipart messages to route (as in the example in the >> guide, which I am using, thank you), the messages received will not be free >> of sequence gaps. Of course, if we know that we have a reasonably >> constrained set of theses discrete routing messages (I called them >> channels), we could number along them too. I think it may be a neat feature >> for 0mq to have that kind of scheme on top of regular pub/sub so that the >> publisher socket would use its id and routing "header" information to >> maintain sequences and subscriber socket would know if it is missing >> messages. >> Mike. >> P.S. Thanks to all that are so responsive on the list and for developing a >> really neat piece of software. >> >> On Feb 23, 2011, at 11:13 AM, John Flanagan wrote: >> >> >> On Wed, Feb 23, 2011 at 12:09 PM, Chuck Remes <[email protected]> >> wrote: >>> >>> On Feb 23, 2011, at 11:31 AM, Pieter Hintjens wrote: >>> >>>> On Wed, Feb 23, 2011 at 6:25 PM, Chuck Remes <[email protected]> >>>> wrote: >>>> >>>>> If you really need reliable delivery of each published message, then >>>>> the Pub/Sub pattern is not appropriate. You should use the REQ/REP pattern >>>>> because you'll need to acknowledge each message. >>>> >>>> Even that won't guarantee delivery :-) What if your acknowledgment gets >>>> lost? >>> >>> True. Then you need timeouts and retransmissions. Definitely complicated >>> stuff... >>> >>>> Plus it adds extraordinary latency to traffic. Depending on your >>>> application, you would want some kind of asynchronous negative >>>> acknowledgment, i.e. "I'm missing data X, please (re)send". Possibly >>>> out of band so the normal pubsub traffic can scale to many clients >>>> over multicast. >>> >>> Batching up the missing sequence numbers can be difficult. I like using a >>> "sliding windows" algo for acknowledgement. You essentially end up >>> recreating the sliding windows logic from zmodem, tcp, etc. >> >> This, this, this. >> >> By the time you layer on all the things you need to implement to achieve >> reasonable reliability guarantees, you will have IMPLEMENTED tcp... poorly. >> >> If you need reliable messaging, you need tcp. The only way around this is >> to NOT need reliable messaging. If there's no possible way to avoid >> reliable messaging, then get used to using tcp. Seriously. You will save >> yourself endless heartache. >> >> The interesting discussion then becomes figuring out ways to avoid or relax >> the reliability requirement. That's a complex topic and the tradeoff >> choices tend to get made for domain-specific reasons so the solutions >> frequently fail to be useful in the generic case. >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
