Ooops, finger slipped. Pieter, what is the difference between the "stream" you describe in your document and forwarder, other than use of a registration server? Do the streams in your estimation have a location? Or are they a collection of connected forwarders (one per box)?
Thank you, Mike. On Feb 23, 2011, at 4:03 PM, Michael Kogan wrote: > So are you saying I can't have this now? :) This sounds like a > generalization of the approach I have been thinking about, using ZK for > coordination. > > On Feb 23, 2011, at 3:39 PM, Pieter Hintjens wrote: > >> Hi Michael, >> >> You're absolutely right to say that sequencing won't work if the >> subscriber uses a filter, which most will. >> >> Your concept of channel matches the stream concept I've explained >> here: http://zeromq.org/osdp, that may give you some more ideas. It's >> a product we hope to build over time, with input from clients and >> users. >> >> Regards >> - >> Pieter Hintjens >> iMatix >> >> On Wed, Feb 23, 2011 at 10:24 PM, Michael Kogan >> <[email protected]> wrote: >>> I don't need really guaranteed delivery, what I need is to be able to know >>> if I am missing messages. The messages in my case represent data that is >>> durably stored elsewhere, so I don't need a durable message store ala AMQP. >>> The per-publisher sequence numbers are reasonable in fact that is the best I >>> could come up with as well. >>> Still, if subscribers are not subscribing to all messages, but using the >>> first message of multipart messages to route (as in the example in the >>> guide, which I am using, thank you), the messages received will not be free >>> of sequence gaps. Of course, if we know that we have a reasonably >>> constrained set of theses discrete routing messages (I called them >>> channels), we could number along them too. I think it may be a neat feature >>> for 0mq to have that kind of scheme on top of regular pub/sub so that the >>> publisher socket would use its id and routing "header" information to >>> maintain sequences and subscriber socket would know if it is missing >>> messages. >>> Mike. >>> P.S. Thanks to all that are so responsive on the list and for developing a >>> really neat piece of software. >>> >>> On Feb 23, 2011, at 11:13 AM, John Flanagan wrote: >>> >>> >>> On Wed, Feb 23, 2011 at 12:09 PM, Chuck Remes <[email protected]> >>> wrote: >>>> >>>> On Feb 23, 2011, at 11:31 AM, Pieter Hintjens wrote: >>>> >>>>> On Wed, Feb 23, 2011 at 6:25 PM, Chuck Remes <[email protected]> >>>>> wrote: >>>>> >>>>>> If you really need reliable delivery of each published message, then >>>>>> the Pub/Sub pattern is not appropriate. You should use the REQ/REP >>>>>> pattern >>>>>> because you'll need to acknowledge each message. >>>>> >>>>> Even that won't guarantee delivery :-) What if your acknowledgment gets >>>>> lost? >>>> >>>> True. Then you need timeouts and retransmissions. Definitely complicated >>>> stuff... >>>> >>>>> Plus it adds extraordinary latency to traffic. Depending on your >>>>> application, you would want some kind of asynchronous negative >>>>> acknowledgment, i.e. "I'm missing data X, please (re)send". Possibly >>>>> out of band so the normal pubsub traffic can scale to many clients >>>>> over multicast. >>>> >>>> Batching up the missing sequence numbers can be difficult. I like using a >>>> "sliding windows" algo for acknowledgement. You essentially end up >>>> recreating the sliding windows logic from zmodem, tcp, etc. >>> >>> This, this, this. >>> >>> By the time you layer on all the things you need to implement to achieve >>> reasonable reliability guarantees, you will have IMPLEMENTED tcp... poorly. >>> >>> If you need reliable messaging, you need tcp. The only way around this is >>> to NOT need reliable messaging. If there's no possible way to avoid >>> reliable messaging, then get used to using tcp. Seriously. You will save >>> yourself endless heartache. >>> >>> The interesting discussion then becomes figuring out ways to avoid or relax >>> the reliability requirement. That's a complex topic and the tradeoff >>> choices tend to get made for domain-specific reasons so the solutions >>> frequently fail to be useful in the generic case. >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
