Holger, Here is a whitepaper for a reliable pub-sub product we're now building:
https://github.com/Malamute/malamute-core/blob/master/MALAMUTE.md If you'd like to be involved in this project (on a commercial basis), let me know. -Pieter On Fri, Oct 17, 2014 at 11:38 AM, Holger Joukl <[email protected]> wrote: > > Hi, > > I realize I'm sometimes confused by terms used by myself and others. > > So, in an attempt to clarify my thoughts, first and foremost to myself, by > writing them down here's how I tend to think about pub-sub messaging: > > (0) Publish-subscribe (pub-sub) messaging: > - one or many sender(s) publish data on one or more "channels" > - one or many listener(s) subscribe to one or more channels > - communication is asynchronous > - listeners might not receive all messages even if up-and-running when > the messages are being sent (due to network glitches or whatever) > > (1) Reliable pub-sub messaging: > - one or many sender(s) publish data on one or more "channels" > - one or many listener(s) subscribe to one or more channels > - communication is asynchronous > - as long as a listener is up-and-running when the messages are being sent > it will > receive all messages from all senders publishing on the channels they've > subscribed to, > in order (order per sender) > > (2) Guaranteed/Certified pub-sub messaging: > - one or many sender(s) publish data on one or more "channels" > - one or many listener(s) subscribe to one or more channels > - communication is asynchronous > - as long as a listener is up-and-running when the messages are being sent > it will > receive all messages from all senders publishing on the channels they've > subscribed to, > in order (order per sender) > - even if a listener is *not* up-and-running when messages are being sent a > listener > will still get all the messages, i.e. the missed messages will get > re-delivered, > in order (order per sender); the listener will not receive new messages > until it > has received all missed message > - as a consequence, messages need to get persisted: > - either each listeners' subscriptions need to get registered when > opening a > subscription, or by predefined configuration, so persisted messages can be > safely > deleted from persistent storage when all registered listeners have received > them > an explicitly acknowledged that fact > - or all sent messages need to get persisted for a period of time so a > listener > can request missed messages for retransmission > > Sometimes the "reliability" distinction is called different "qualities of > service" > (QoS). > > Note that I've deliberately ignored any problems that might arise in > pub-sub > communications, e.g. slow consumers in (unreliable) high frequency > scenarios > or the like. > > Pub-sub messaging can be implemented over a variety of transports and > protocols: > (UDP) broadcast, multicast, TCP, ... > The transport + protocol used determines the properties of > pub-sub-messaging, e.g.: > - plain UDP broadcast is unreliable > - PGM or NORM multicast is reliable > - a protocol on top of TCP/UDP/reliable multicast is ususally necessary for > guaranteed/certified messaging > > Design approaches you encounter in the wild: > > - central broker with queues, optionally persistent (e.g. AMQP brokers, JMS > providers, IBM MQ, ...) > - senders do not know anything about listeners > - listeners do not know anything about senders > - the broker is "the rendezvous point" for communication, often called > "message bus" > - senders connect to the broker (usually TCP) > - listeners connect to the broker (usually TCP) > - channels are usually called "topics" (basically a special case of queue > that allows > for many listeners to receive the topic messages) > - broker knows registered guaranteed/certified listeners: When all known > listeners > have retrieved and acknowledged a certain message on a topic this > message will get > deleted from the queue > - queue persistence to make queued messages survive broker failure > - broker might be distributed, i.e. multiple broker working in > cooperation, e.g. for reasons > of throughput scaling, partitioning of data channels, replication > - broker is a single point of failure so will normally get clustered and > replicated for > some notion of cold/hot standby for mission critical communication > > - central broker with a persistent commit log or journal (e.g. Apache > Kafka, ZPER, ...) > - senders do not know anything about listeners > - listeners do not know anything about senders > - the broker is "the rendezvous point" for communication > - senders connect to the broker (usually TCP) > - listeners connect to the broker (usually TCP) > - messages sent on the channels are persisted by the broker as sequenced > continous message > "commit logs" for a configurable period of time > - listeners can get retransmission of any historic message within the > configured period of > time > - listeners are responsible for their state, i.e. which messages they > have already > processed > - listener state is basically just a pointer to its current position in > the commit log > - broker might be distributed, i.e. multiple broker working in > cooperation, e.g. for reasons > of throughput scaling, partitioning of data channels, replication > - broker is a single point of failure so will normally get clustered and > replicated for > some notion of cold/hot standby for mission critical communication > > - distributed message bus (e.g. TIB/Rendezvous, http://iris.karalabe.com/ > , ...) > - consists of a network of "micro brokers", sometimes called daemons or > agent or relay process, > ususally one per host > - senders connect to a (local) broker (usually IPC or TCP) > - listeners connect to a (local) broker (usually IPC or TCP) > - the "network of brokers" is "the rendezvous point" for communication, > i.e. the brokers > collectively form a "message bus" > - senders do not know anything about listeners in case of reliable > pub-sub > - listeners do not know anything about senders in case of reliable > pub-sub > - senders know about registered listeners in case of guaranteed/certified > pub-sub > - message persistence is local to the sending clients (called "ledger" > file e.g. in TIB/Rv > terms); might also be possible local to the local broker but I haven't > seen such an approach > yet > - registered listeners need to acknowledge messages with their senders so > they can remove them > from persistent storage > - brokers are not single points of failure > - brokers do not need separate clustering/replication apart from what > needs to be done > for the mission critical applications running on their host systems > anyway; they are > basically guarded by the same safety measures taken for the host > system, ranging from none > to whatever > > - somewhat hybrid forms, e.g. brokers that scale out elastically like > http://roq-messaging.org/ > > Hope no one's annoyed by the > lengthy-not-really-a-question-nor-problem-description > kind of post. Glad to get anyone's thoughts or totally different views or > hints on the > glaringly obvious that I missed. > > Best regards > Holger > > Landesbank Baden-Wuerttemberg > Anstalt des oeffentlichen Rechts > Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz > HRA 12704 > Amtsgericht Stuttgart > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
