Hi Michael, You're absolutely right to say that sequencing won't work if the subscriber uses a filter, which most will.
Your concept of channel matches the stream concept I've explained here: http://zeromq.org/osdp, that may give you some more ideas. It's a product we hope to build over time, with input from clients and users. Regards - Pieter Hintjens iMatix On Wed, Feb 23, 2011 at 10:24 PM, Michael Kogan <[email protected]> wrote: > I don't need really guaranteed delivery, what I need is to be able to know > if I am missing messages. The messages in my case represent data that is > durably stored elsewhere, so I don't need a durable message store ala AMQP. > The per-publisher sequence numbers are reasonable in fact that is the best I > could come up with as well. > Still, if subscribers are not subscribing to all messages, but using the > first message of multipart messages to route (as in the example in the > guide, which I am using, thank you), the messages received will not be free > of sequence gaps. Of course, if we know that we have a reasonably > constrained set of theses discrete routing messages (I called them > channels), we could number along them too. I think it may be a neat feature > for 0mq to have that kind of scheme on top of regular pub/sub so that the > publisher socket would use its id and routing "header" information to > maintain sequences and subscriber socket would know if it is missing > messages. > Mike. > P.S. Thanks to all that are so responsive on the list and for developing a > really neat piece of software. > > On Feb 23, 2011, at 11:13 AM, John Flanagan wrote: > > > On Wed, Feb 23, 2011 at 12:09 PM, Chuck Remes <[email protected]> > wrote: >> >> On Feb 23, 2011, at 11:31 AM, Pieter Hintjens wrote: >> >> > On Wed, Feb 23, 2011 at 6:25 PM, Chuck Remes <[email protected]> >> > wrote: >> > >> >> If you really need reliable delivery of each published message, then >> >> the Pub/Sub pattern is not appropriate. You should use the REQ/REP pattern >> >> because you'll need to acknowledge each message. >> > >> > Even that won't guarantee delivery :-) What if your acknowledgment gets >> > lost? >> >> True. Then you need timeouts and retransmissions. Definitely complicated >> stuff... >> >> > Plus it adds extraordinary latency to traffic. Depending on your >> > application, you would want some kind of asynchronous negative >> > acknowledgment, i.e. "I'm missing data X, please (re)send". Possibly >> > out of band so the normal pubsub traffic can scale to many clients >> > over multicast. >> >> Batching up the missing sequence numbers can be difficult. I like using a >> "sliding windows" algo for acknowledgement. You essentially end up >> recreating the sliding windows logic from zmodem, tcp, etc. > > This, this, this. > > By the time you layer on all the things you need to implement to achieve > reasonable reliability guarantees, you will have IMPLEMENTED tcp... poorly. > > If you need reliable messaging, you need tcp. The only way around this is > to NOT need reliable messaging. If there's no possible way to avoid > reliable messaging, then get used to using tcp. Seriously. You will save > yourself endless heartache. > > The interesting discussion then becomes figuring out ways to avoid or relax > the reliability requirement. That's a complex topic and the tradeoff > choices tend to get made for domain-specific reasons so the solutions > frequently fail to be useful in the generic case. > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
