Github user rdhabalia commented on a diff in the pull request:

    https://github.com/apache/incubator-pulsar/pull/532#discussion_r124362085
  
    --- Diff: docs/NonPersistentTopic.md ---
    @@ -0,0 +1,96 @@
    +
    +# Non-persistent topic
    +
    +<!-- TOC depthFrom:2 depthTo:4 withLinks:1 updateOnSave:1 orderedList:0 -->
    +
    +- [Topic](#topic)
    +- [Non-persistent topic](#non-persistent-topic)
    +   - [Performance](#performance)
    +- [Client API](#client-api)
    +   - [Consumer API](#consumer-api)
    +   - [Producer API](#producer-api)
    +- [Non-persistent configuration](#non-persistent-configuration)
    +   - [Namespace configuration](#namespace-configuration)
    +   - [Broker configuration](#broker-configuration)
    +
    +<!-- /TOC -->
    +
    +## Topic
    +
    +In pulsar, topic is a key entity that used to connect producers and 
consumers. Pulsar supports **persistent** and **non-persistent** topics which 
can be chosen by application based on its usecases and requirements. Producers 
and consumers can use the same client-api to connect to persistent and 
non-persistent topic. 
    +
    +## Non-persistent topic
    +
    +As name suggests, non-persist topic does not persist messages into any 
durable storage disk unlike persistent topic where messages are durably 
persisted on multiple disks. 
    +
    +Therefore, if you are using persistent delivery, messages are persisted to 
disk/database so that they will survive a broker restart or subscriber 
failover. While using non-persistent delivery, if you kill a broker or 
subscriber is disconnected then subscriber will lose all in-transit messages. 
So, client may see message loss with non-persistent topic.
    +
    +- In non-persistent topic, as soon as broker receives published message, 
it immediately delivers this message to all connected subscribers without 
persisting them into any storage. so, if subscriber gets disconnected with 
broker then broker will not be able to deliver those in-transit messages and 
subscribers will never be able to receive those messages again.
    +- Broker only allows configured number of in-flight messages per client 
connection. So, if producer tries to publish messages higher than this rate, 
then broker silently drops those new incoming messages without processing and 
delivering them to the subscribers. However, broker acknowledges with special 
message-id (`msg-id: -1:-1`) for those dropped messages to signal producer 
about the message drop.
    +
    +### Performance
    +
    +Non-persistent messaging is usually faster than persistent messaging 
because broker does not persist messages and immediately sends ack back to 
producer as soon as that message received by the broker. Therefore, producer 
sees comparatively low publish latency with non-persistent topic.
    +
    +
    +## Client API
    +
    +
    +A topic name will look like:
    +
    +```
    +non-persistent://my-property/us-west/my-namespace/my-topic
    +```
    +
    +Producer and consumer can connect to non-persistent topic in a similar way 
as persistent topic except topic name must start with `non-persistent`.
    +
    +Non-persistent topic supports all 3 different subscription-modes : 
**Exclusive**, **Shared**, **Failover** which are already explained in details 
at [GettingStarted](GettingStarted.md). 
    +
    +
    +### Consumer API
    +
    +```
    +PulsarClient client = PulsarClient.create("pulsar://localhost:6650");
    +
    +Consumer consumer = client.subscribe(
    +            "non-persistent://sample/standalone/ns1/my-topic",
    +            "my-subscribtion-name");
    +```
    +
    +### Producer API
    +
    +```
    +PulsarClient client = PulsarClient.create("pulsar://localhost:6650");
    +
    +Producer producer = client.createProducer(
    +            "non-persistent://sample/standalone/ns1/my-topic");
    +```
    +
    +## Non-persistent configuration
    +
    +### Namespace configuration
    +
    +
    +Boker creates non-persistent topic only under a namespace which is 
configured with non-persistency. Client can set non-persistency of a namespace 
using admin API and tools:
    +
    +```
    +bin/pulsar-admin namespaces set-non-persistency test-property/cl1/ns1 
--non-persistent
    --- End diff --
    
    > What happens in case the namespace is already in use and have persistent 
topics on it?
    
    Before setting non-persistency, it should verify that namespace should not 
have any persistent-topic under it else it should fail the request. and it can 
flip from non-persistent to persistent if there is no np-topic is being loaded 
under that namespace.
    
    > For example, let say I'm doing a lookup for non-persistent://.... and the 
namespace is not owned by any broker, then the load balancer will assign this 
namespace to a broker that supports non-persistent topics.
    
    Actually, idea behind marking namespace as non-persistent to just isolate 
namespace for non-persistent topics and configured-broker (allow to own only 
np-topics) owns only non-persistent topics at any time. 
    If we don't have have namespace level visibility and if broker assigns 
topic at runtime on lookup eg: `non-persistent://..` then on first 
`non-persistent (np)` lookup configured-broker (which should only owns np 
topic) owns that namespace and if the same namespace has persistent topic also 
then configured broker ends up owning mix of persistent and non-persistent 
topics. Therefore, team decided to take namespace level approach to avoid mix 
of topics under a configured broker.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---

Reply via email to