lucperkins commented on a change in pull request #1507: Using Pulsar as a
message queue (cookbook)
URL: https://github.com/apache/incubator-pulsar/pull/1507#discussion_r180510283
##########
File path: site/docs/latest/cookbooks/message-queue.md
##########
@@ -0,0 +1,89 @@
+---
+title: Using Pulsar as a message queue
+lead: Although Pulsar is typically known as a real-time messaging system, it's
also an excellent choice for a queuing system
+tags: [clients, java, python, message queue, cookbook]
+---
+
+Message queues are essential components of many large-scale data
architectures. If every single work object that passes through your system
absolutely *must* be processed in spite of the slowness or downright failure of
this or that system component, there's a good chance that you'll need a message
queue to step in and ensure that unprocessed data is retained---with correct
ordering---until the required actions are taken.
+
+Pulsar is a great choice for a message queue because:
+
+* it was built with [persistent message
storage](../../getting-started/ConceptsAndArchitecture#persistent-storage) in
mind
+* it offers automatic load balancing across {% popover consumers %} for
messages on a topic (or custom load balancing if you wish)
+
+{% include admonition.html type="success" content="You can use the same Pulsar
installation to act as a real-time message bus and as a message queue if you
wish (or just one or the other). You can set aside some topics for real-time
purposes and other topics for message queue purposes (or use specific
namespaces for either purpose if you wish)." %}
+
+## Client configuration changes
+
+To use a Pulsar {% popover topic %} as a message queue, you should distribute
the receiver load on that topic across several {% popover consumers %}. Each
consumer must:
+
+* Have a [shared
subscription](../../getting-started/ConceptsAndArchitecture#shared) and use the
same subscription name as the other consumers (otherwise the subscription is
not shared)
+* Have the receiver queue size set to zero (or very low). Each Pulsar {%
popover consumer %} has a **receiver queue** that determines how many messages
the consumer will attempt to fetch at a time. A receiver queue of 1000 (the
default), for example, means that the consumer will attempt to process 1000
messages from the topic's backlog upon connection. Setting the receiver queue
to zero essentially means ensuring that each consumer is only doing one thing
at a time.
Review comment:
@merlimat Hmmm. I'm a bit confused. I was under the impression that
drastically reducing the receiver queue size was *the* recommendation for using
Pulsar as an MQ. Without that, it seems to me that this doc might be
superfluous (I'm fine with eliminating it if so).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
