[GitHub] [pulsar] codelipenghui commented on issue #4533: Add subscription modes for java client doc.
codelipenghui commented on issue #4533: Add subscription modes for java client doc. URL: https://github.com/apache/pulsar/pull/4533#issuecomment-502975493 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat closed issue #4551: Failed to build c++ client at first time in linux
merlimat closed issue #4551: Failed to build c++ client at first time in linux URL: https://github.com/apache/pulsar/issues/4551 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4549: [pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles LongSerializer
codelipenghui commented on issue #4549: [pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles LongSerializer URL: https://github.com/apache/pulsar/pull/4549#issuecomment-502975892 run java8 tests run cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] merlimat merged pull request #4552: [Issue 4551] Fix build error of pulsar c++ client
merlimat merged pull request #4552: [Issue 4551] Fix build error of pulsar c++ client URL: https://github.com/apache/pulsar/pull/4552 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated (e9425c5 -> f349ef3)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git. from e9425c5 [pulsar-broker] Make non-tls web/broker-service optional (#3501) add f349ef3 Fix build error of pulsar c++ client. (#4552) No new revisions were added by this update. Summary of changes: site2/docs/developing-cpp.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[GitHub] [pulsar] codelipenghui commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source
codelipenghui commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source URL: https://github.com/apache/pulsar/pull/3786#issuecomment-502976200 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on issue #4546: Introduce multi host in pulsar doc
zymap commented on issue #4546: Introduce multi host in pulsar doc URL: https://github.com/apache/pulsar/pull/4546#issuecomment-502977732 @Anonymitaet Thanks for your review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on a change in pull request #4546: Introduce multi host in pulsar doc
zymap commented on a change in pull request #4546: Introduce multi host in
pulsar doc
URL: https://github.com/apache/pulsar/pull/4546#discussion_r294637980
##
File path: site2/docs/admin-api-overview.md
##
@@ -70,3 +70,23 @@ config.setTlsTrustCertsFilePath(tlsTrustCertsFilePath);
PulsarAdmin admin = new PulsarAdmin(url, config);
```
+
+If you have mutilple brokers to using, you can use multi host like pulsar
service, like this:
+```java
+URL url = new URL("http://localhost:8080,localhost:8081,localhost:8082";);
+// Pass auth-plugin class fully-qualified name if Pulsar-security enabled
Review comment:
I think not.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie opened a new pull request #4556: [tests] Scope the dependencies for spark and storm tests to test scope
sijie opened a new pull request #4556: [tests] Scope the dependencies for spark and storm tests to test scope URL: https://github.com/apache/pulsar/pull/4556 *Motivation* In gitsha b7a5677324a79708fd288fee6e8852f16a08ab6c , it introduced a provided dependency for pulsar-storm-test, which it will fail release process. Since it will attempt to transfer the pom file for `com.twitter:carbonite:pom:1.5.0` which is a dependency comes from storm-server. *Changes* Make all the dependencies scoped at test scope for two test modules. So maven release will not attempt to release them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4544: [Doc] Fixed a typo in Cookbook / Topic compaction
sijie commented on issue #4544: [Doc] Fixed a typo in Cookbook / Topic compaction URL: https://github.com/apache/pulsar/pull/4544#issuecomment-502988600 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie merged pull request #4529: [docs] Update site2/docs/functions-quickstart.md
sijie merged pull request #4529: [docs] Update site2/docs/functions-quickstart.md URL: https://github.com/apache/pulsar/pull/4529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [docs] Update site2/docs/functions-quickstart.md (#4529)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2f53fb6 [docs] Update site2/docs/functions-quickstart.md (#4529)
2f53fb6 is described below
commit 2f53fb6f9350643e47f5abeb433d7fea72d0832b
Author: Jennifer Huang <47805623+jennifer88hu...@users.noreply.github.com>
AuthorDate: Tue Jun 18 15:47:29 2019 +0800
[docs] Update site2/docs/functions-quickstart.md (#4529)
### Motivation
While add "Write and run a Go function" section, improve doc on Pulsar
Functions: quick start throughout.
### Modifications
1. Add a section of "Write and run a Go function".
2. Update the the first-person pronoun as the second-person pronoun.
3. Correct the usage of "Pulsar Functions" and "a Pulsar function". For the
usage discussion, see #4507 .
4. Change the future tense into present tense.
5. Refine part of structure (heading level).
6. Refine sentences to be more clear and concise.
7. Refine some typos.
### Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? ( docs)
---
site2/docs/functions-quickstart.md | 181 -
1 file changed, 117 insertions(+), 64 deletions(-)
diff --git a/site2/docs/functions-quickstart.md
b/site2/docs/functions-quickstart.md
index 6b7a026..2e1265c 100644
--- a/site2/docs/functions-quickstart.md
+++ b/site2/docs/functions-quickstart.md
@@ -1,20 +1,20 @@
---
-id: functions-quickstart
-title: Get started with Pulsar Functions
-sidebar_label: Get started
+id: functions-quickstart
+title: Get started with Pulsar Functions
+sidebar_label: Get started
---
-This tutorial will walk you through running a
[standalone](reference-terminology.md#standalone) Pulsar
[cluster](reference-terminology.md#cluster) on your machine and then running
your first Pulsar Functions using that cluster. The first function will run in
local run mode (outside your Pulsar
[cluster](reference-terminology.md#cluster)), while the second will run in
cluster mode (inside your cluster).
+This tutorial walks you through running a
[standalone](reference-terminology.md#standalone) Pulsar
[cluster](reference-terminology.md#cluster) on your machine, and then running
your first Pulsar Function using that cluster. The first Pulsar Function runs
in local run mode (outside your Pulsar
[cluster](reference-terminology.md#cluster)), while the second runs in cluster
mode (inside your cluster).
-> In local run mode, your Pulsar Function will communicate with your Pulsar
cluster but will run outside of the cluster.
+> In local run mode, Pulsar Functions communicate with Pulsar cluster, but run
outside of the cluster.
## Prerequisites
-In order to follow along with this tutorial, you'll need to have
[Maven](https://maven.apache.org/download.cgi) installed on your machine.
+Install [Maven](https://maven.apache.org/download.cgi) on your machine.
## Run a standalone Pulsar cluster
-In order to run our Pulsar Functions, we'll need to run a Pulsar cluster
locally first. The easiest way to do that is to run Pulsar in
[standalone](reference-terminology.md#standalone) mode. Follow these steps to
start up a standalone cluster:
+In order to run Pulsar Functions, you need to run a Pulsar cluster locally
first. The easiest way is to run Pulsar in
[standalone](reference-terminology.md#standalone) mode. Follow these steps to
start up a standalone cluster.
```bash
$ wget pulsar:binary_release_url
@@ -24,11 +24,11 @@ $ bin/pulsar standalone \
--advertised-address 127.0.0.1
```
-When running Pulsar in standalone mode, the `public` tenant and `default`
namespace will be created automatically for you. That tenant and namespace will
be used throughout this tutorial.
+When running Pulsar in standalone mode, the `public` tenant and the `default`
namespace are created automatically. The tenant and namespace are used
throughout this tutorial.
## Run a Pulsar Function in local run mode
-Let's start with a simple function that takes a string as input from a Pulsar
topic, adds an exclamation point to the end of the string, and then publishes
that new string to another Pulsar topic. Here's the code for the function:
+You can start with a simple function that takes a string as input from a
Pulsar topic, adds an exclamation point to the end of the string, and then
publishes the new string to another Pulsar topic. The following is the code for
the function.
```java
package org.apache.pulsar.functions.api.examples;
@@ -43,7 +43,7 @@ public class ExclamationFunction implements Function {
}
```
-A JAR file containing this and several other functions (written in Java) is
included with the binary distribution you downloaded above (in the `exa
[GitHub] [pulsar] sijie commented on issue #4507: [Docs] Update site2/docs/functions-overview.md
sijie commented on issue #4507: [Docs] Update site2/docs/functions-overview.md URL: https://github.com/apache/pulsar/pull/4507#issuecomment-502989161 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #4533: Add subscription modes for java client doc.
sijie commented on a change in pull request #4533: Add subscription modes for
java client doc.
URL: https://github.com/apache/pulsar/pull/4533#discussion_r294653927
##
File path: site2/docs/client-libraries-java.md
##
@@ -306,6 +306,185 @@ consumerBuilder
});
```
+### Subscription modes
+
+Pulsar has various [subscription modes](concepts-messaging#subscription-modes)
to match different scenarios. A topic can have multiple subscriptions with
different subscription modes. However, a subscription can only have one
subscription mode at a time.
+
+A subscription is identified with the subscription name, and a subscription
name can specify only one subscription mode at a time. You can change the
subscription mode, yet you have to let all existing consumers of this
subscription offline first.
+
+Different subscription modes have different message distribution modes. This
section describes the differences of subscription modes and how to use them.
+
+In order to better describe their differences, assuming you have a topic named
"my-topic", and the producer has published 10 messages.
+
+```java
+Producer producer = client.newProducer(Schema.STRING)
+.topic("my-topic")
+.enableBatch(false)
+.create();
+// 3 messages with "key-1", 3 messages with "key-2", 2 messages with "key-3"
and 2 messages with "key-4"
+producer.newMessage().key("key-1").value("message-1-1").send();
+producer.newMessage().key("key-1").value("message-1-2").send();
+producer.newMessage().key("key-1").value("message-1-3").send();
+producer.newMessage().key("key-2").value("message-2-1").send();
+producer.newMessage().key("key-2").value("message-2-2").send();
+producer.newMessage().key("key-2").value("message-2-3").send();
+producer.newMessage().key("key-3").value("message-3-1").send();
+producer.newMessage().key("key-3").value("message-3-2").send();
+producer.newMessage().key("key-4").value("message-4-1").send();
+producer.newMessage().key("key-4").value("message-4-2").send();
+```
+
+ Exclusive
+
+Create a new consumer and subscribe with the `Exclusive` subscription mode.
+
+```java
+Consumer consumer = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Exclusive)
+.subscribe()
+```
+
+Only the first consumer is allowed to the subscription, other consumers
receive an error. The first consumer receives all 10 messages, and the
consuming order is the same as the producing order.
+
+> Note:
+>
+> If topic is a partitioned topic, the first consumer subscribes to all
partitioned topics, other consumers are not assigned with partitions and
receive an error.
+
+ Failover
+
+Create new consumers and subscribe with the`Failover` subscription mode.
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+//conumser1 is the active consumer, consumer2 is the standby consumer.
+//consumer1 receives 5 messages and then crashes, consumer2 takes over as an
active consumer.
+
+
+```
+
+Multiple consumers can attach to the same subscription, yet only the first
consumer is active, and others are standby. When the active consumer is
disconnected, messages will be dispatched to one of standby consumers, and the
standby consumer becomes active consumer.
+
+If the first active consumer receives 5 messages and is disconnected, the
standby consumer becomes active consumer. Consumer1 will receive:
+
+```
+("key-1", "message-1-1")
+("key-1", "message-1-2")
+("key-1", "message-1-3")
+("key-2", "message-2-1")
+("key-2", "message-2-2")
+```
+
+consumer2 will receive:
+
+```
+("key-2", "message-2-3")
+("key-3", "message-3-1")
+("key-3", "message-3-2")
+("key-4", "message-4-1")
+("key-4", "message-4-2")
+```
+
+> Note:
+>
+> If a topic is a partitioned topic, the partitioned topic only has one active
consumer, and messages of the partition are distributed to multiple consumers.
Review comment:
hmm I think the description here is misleading.
I think it should contain two points:
- each partition only has one active consumer
- messages of one partition only distributed to one consumer
- multiple partitions are distributed to multiple consumers
@codelipenghui right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper
sijie commented on a change in pull request #4555: [Issue 4422] Support pulsar
scripts to work with bookkeeper
URL: https://github.com/apache/pulsar/pull/4555#discussion_r294655437
##
File path: conf/pulsar_env.sh
##
@@ -58,3 +58,16 @@ PULSAR_EXTRA_OPTS=${PULSAR_EXTRA_OPTS:-"
-Dpulsar.allocator.exit_on_oom=true -Di
#Wait time before forcefully kill the pulser server instance, if the stop is
not successful
#PULSAR_STOP_TIMEOUT=
+
+# Set BOOKIE options here to ensure that all pulsar scripts can work seamless
with bookkeeper
+
+# default settings for starting bookkeeper
+
+# Memory size options
+BOOKIE_MEM=" -Xms2g -Xmx2g -XX:MaxDirectMemorySize=2g"
+
+# Garbage collection options
+BOOKIE_GC=" -XX:+UseG1GC -XX:MaxGCPauseMillis=10 -XX:+ParallelRefProcEnabled
-XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis
-XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50
-XX:+DisableExplicitGC -XX:-ResizePLAB"
+
+# Extra options to be passed to the jvm
+BOOKIE_EXTRA_OPTS="${BOOKIE_EXTRA_OPTS} ${BOOKIE_MEM} ${BOOKIE_GC}
-Dio.netty.leakDetectionLevel=disabled
-Dio.netty.recycler.maxCapacity.default=1000
-Dio.netty.recycler.linkCapacity=1024"
Review comment:
maybe remove BOOKIE_MEM and BOOKIE_GC and just use PULSAR_MEM and PULSAR_GC?
Because the only requirement is for people to add extra bookie opts (for
kerberos), right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4553: [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints
sijie commented on issue #4553: [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints URL: https://github.com/apache/pulsar/pull/4553#issuecomment-502991905 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #4533: Add subscription modes for java client doc.
codelipenghui commented on a change in pull request #4533: Add subscription
modes for java client doc.
URL: https://github.com/apache/pulsar/pull/4533#discussion_r294660790
##
File path: site2/docs/client-libraries-java.md
##
@@ -306,6 +306,185 @@ consumerBuilder
});
```
+### Subscription modes
+
+Pulsar has various [subscription modes](concepts-messaging#subscription-modes)
to match different scenarios. A topic can have multiple subscriptions with
different subscription modes. However, a subscription can only have one
subscription mode at a time.
+
+A subscription is identified with the subscription name, and a subscription
name can specify only one subscription mode at a time. You can change the
subscription mode, yet you have to let all existing consumers of this
subscription offline first.
+
+Different subscription modes have different message distribution modes. This
section describes the differences of subscription modes and how to use them.
+
+In order to better describe their differences, assuming you have a topic named
"my-topic", and the producer has published 10 messages.
+
+```java
+Producer producer = client.newProducer(Schema.STRING)
+.topic("my-topic")
+.enableBatch(false)
+.create();
+// 3 messages with "key-1", 3 messages with "key-2", 2 messages with "key-3"
and 2 messages with "key-4"
+producer.newMessage().key("key-1").value("message-1-1").send();
+producer.newMessage().key("key-1").value("message-1-2").send();
+producer.newMessage().key("key-1").value("message-1-3").send();
+producer.newMessage().key("key-2").value("message-2-1").send();
+producer.newMessage().key("key-2").value("message-2-2").send();
+producer.newMessage().key("key-2").value("message-2-3").send();
+producer.newMessage().key("key-3").value("message-3-1").send();
+producer.newMessage().key("key-3").value("message-3-2").send();
+producer.newMessage().key("key-4").value("message-4-1").send();
+producer.newMessage().key("key-4").value("message-4-2").send();
+```
+
+ Exclusive
+
+Create a new consumer and subscribe with the `Exclusive` subscription mode.
+
+```java
+Consumer consumer = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Exclusive)
+.subscribe()
+```
+
+Only the first consumer is allowed to the subscription, other consumers
receive an error. The first consumer receives all 10 messages, and the
consuming order is the same as the producing order.
+
+> Note:
+>
+> If topic is a partitioned topic, the first consumer subscribes to all
partitioned topics, other consumers are not assigned with partitions and
receive an error.
+
+ Failover
+
+Create new consumers and subscribe with the`Failover` subscription mode.
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+//conumser1 is the active consumer, consumer2 is the standby consumer.
+//consumer1 receives 5 messages and then crashes, consumer2 takes over as an
active consumer.
+
+
+```
+
+Multiple consumers can attach to the same subscription, yet only the first
consumer is active, and others are standby. When the active consumer is
disconnected, messages will be dispatched to one of standby consumers, and the
standby consumer becomes active consumer.
+
+If the first active consumer receives 5 messages and is disconnected, the
standby consumer becomes active consumer. Consumer1 will receive:
+
+```
+("key-1", "message-1-1")
+("key-1", "message-1-2")
+("key-1", "message-1-3")
+("key-2", "message-2-1")
+("key-2", "message-2-2")
+```
+
+consumer2 will receive:
+
+```
+("key-2", "message-2-3")
+("key-3", "message-3-1")
+("key-3", "message-3-2")
+("key-4", "message-4-1")
+("key-4", "message-4-2")
+```
+
+> Note:
+>
+> If a topic is a partitioned topic, the partitioned topic only has one active
consumer, and messages of the partition are distributed to multiple consumers.
Review comment:
yes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #4533: Add subscription modes for java client doc.
codelipenghui commented on a change in pull request #4533: Add subscription
modes for java client doc.
URL: https://github.com/apache/pulsar/pull/4533#discussion_r294660790
##
File path: site2/docs/client-libraries-java.md
##
@@ -306,6 +306,185 @@ consumerBuilder
});
```
+### Subscription modes
+
+Pulsar has various [subscription modes](concepts-messaging#subscription-modes)
to match different scenarios. A topic can have multiple subscriptions with
different subscription modes. However, a subscription can only have one
subscription mode at a time.
+
+A subscription is identified with the subscription name, and a subscription
name can specify only one subscription mode at a time. You can change the
subscription mode, yet you have to let all existing consumers of this
subscription offline first.
+
+Different subscription modes have different message distribution modes. This
section describes the differences of subscription modes and how to use them.
+
+In order to better describe their differences, assuming you have a topic named
"my-topic", and the producer has published 10 messages.
+
+```java
+Producer producer = client.newProducer(Schema.STRING)
+.topic("my-topic")
+.enableBatch(false)
+.create();
+// 3 messages with "key-1", 3 messages with "key-2", 2 messages with "key-3"
and 2 messages with "key-4"
+producer.newMessage().key("key-1").value("message-1-1").send();
+producer.newMessage().key("key-1").value("message-1-2").send();
+producer.newMessage().key("key-1").value("message-1-3").send();
+producer.newMessage().key("key-2").value("message-2-1").send();
+producer.newMessage().key("key-2").value("message-2-2").send();
+producer.newMessage().key("key-2").value("message-2-3").send();
+producer.newMessage().key("key-3").value("message-3-1").send();
+producer.newMessage().key("key-3").value("message-3-2").send();
+producer.newMessage().key("key-4").value("message-4-1").send();
+producer.newMessage().key("key-4").value("message-4-2").send();
+```
+
+ Exclusive
+
+Create a new consumer and subscribe with the `Exclusive` subscription mode.
+
+```java
+Consumer consumer = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Exclusive)
+.subscribe()
+```
+
+Only the first consumer is allowed to the subscription, other consumers
receive an error. The first consumer receives all 10 messages, and the
consuming order is the same as the producing order.
+
+> Note:
+>
+> If topic is a partitioned topic, the first consumer subscribes to all
partitioned topics, other consumers are not assigned with partitions and
receive an error.
+
+ Failover
+
+Create new consumers and subscribe with the`Failover` subscription mode.
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+//conumser1 is the active consumer, consumer2 is the standby consumer.
+//consumer1 receives 5 messages and then crashes, consumer2 takes over as an
active consumer.
+
+
+```
+
+Multiple consumers can attach to the same subscription, yet only the first
consumer is active, and others are standby. When the active consumer is
disconnected, messages will be dispatched to one of standby consumers, and the
standby consumer becomes active consumer.
+
+If the first active consumer receives 5 messages and is disconnected, the
standby consumer becomes active consumer. Consumer1 will receive:
+
+```
+("key-1", "message-1-1")
+("key-1", "message-1-2")
+("key-1", "message-1-3")
+("key-2", "message-2-1")
+("key-2", "message-2-2")
+```
+
+consumer2 will receive:
+
+```
+("key-2", "message-2-3")
+("key-3", "message-3-1")
+("key-3", "message-3-2")
+("key-4", "message-4-1")
+("key-4", "message-4-2")
+```
+
+> Note:
+>
+> If a topic is a partitioned topic, the partitioned topic only has one active
consumer, and messages of the partition are distributed to multiple consumers.
Review comment:
yes, updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #4533: Add subscription modes for java client doc.
codelipenghui commented on a change in pull request #4533: Add subscription
modes for java client doc.
URL: https://github.com/apache/pulsar/pull/4533#discussion_r294660790
##
File path: site2/docs/client-libraries-java.md
##
@@ -306,6 +306,185 @@ consumerBuilder
});
```
+### Subscription modes
+
+Pulsar has various [subscription modes](concepts-messaging#subscription-modes)
to match different scenarios. A topic can have multiple subscriptions with
different subscription modes. However, a subscription can only have one
subscription mode at a time.
+
+A subscription is identified with the subscription name, and a subscription
name can specify only one subscription mode at a time. You can change the
subscription mode, yet you have to let all existing consumers of this
subscription offline first.
+
+Different subscription modes have different message distribution modes. This
section describes the differences of subscription modes and how to use them.
+
+In order to better describe their differences, assuming you have a topic named
"my-topic", and the producer has published 10 messages.
+
+```java
+Producer producer = client.newProducer(Schema.STRING)
+.topic("my-topic")
+.enableBatch(false)
+.create();
+// 3 messages with "key-1", 3 messages with "key-2", 2 messages with "key-3"
and 2 messages with "key-4"
+producer.newMessage().key("key-1").value("message-1-1").send();
+producer.newMessage().key("key-1").value("message-1-2").send();
+producer.newMessage().key("key-1").value("message-1-3").send();
+producer.newMessage().key("key-2").value("message-2-1").send();
+producer.newMessage().key("key-2").value("message-2-2").send();
+producer.newMessage().key("key-2").value("message-2-3").send();
+producer.newMessage().key("key-3").value("message-3-1").send();
+producer.newMessage().key("key-3").value("message-3-2").send();
+producer.newMessage().key("key-4").value("message-4-1").send();
+producer.newMessage().key("key-4").value("message-4-2").send();
+```
+
+ Exclusive
+
+Create a new consumer and subscribe with the `Exclusive` subscription mode.
+
+```java
+Consumer consumer = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Exclusive)
+.subscribe()
+```
+
+Only the first consumer is allowed to the subscription, other consumers
receive an error. The first consumer receives all 10 messages, and the
consuming order is the same as the producing order.
+
+> Note:
+>
+> If topic is a partitioned topic, the first consumer subscribes to all
partitioned topics, other consumers are not assigned with partitions and
receive an error.
+
+ Failover
+
+Create new consumers and subscribe with the`Failover` subscription mode.
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+//conumser1 is the active consumer, consumer2 is the standby consumer.
+//consumer1 receives 5 messages and then crashes, consumer2 takes over as an
active consumer.
+
+
+```
+
+Multiple consumers can attach to the same subscription, yet only the first
consumer is active, and others are standby. When the active consumer is
disconnected, messages will be dispatched to one of standby consumers, and the
standby consumer becomes active consumer.
+
+If the first active consumer receives 5 messages and is disconnected, the
standby consumer becomes active consumer. Consumer1 will receive:
+
+```
+("key-1", "message-1-1")
+("key-1", "message-1-2")
+("key-1", "message-1-3")
+("key-2", "message-2-1")
+("key-2", "message-2-2")
+```
+
+consumer2 will receive:
+
+```
+("key-2", "message-2-3")
+("key-3", "message-3-1")
+("key-3", "message-3-2")
+("key-4", "message-4-1")
+("key-4", "message-4-2")
+```
+
+> Note:
+>
+> If a topic is a partitioned topic, the partitioned topic only has one active
consumer, and messages of the partition are distributed to multiple consumers.
Review comment:
yes. updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] murong00 commented on a change in pull request #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper
murong00 commented on a change in pull request #4555: [Issue 4422] Support
pulsar scripts to work with bookkeeper
URL: https://github.com/apache/pulsar/pull/4555#discussion_r294669631
##
File path: conf/pulsar_env.sh
##
@@ -58,3 +58,16 @@ PULSAR_EXTRA_OPTS=${PULSAR_EXTRA_OPTS:-"
-Dpulsar.allocator.exit_on_oom=true -Di
#Wait time before forcefully kill the pulser server instance, if the stop is
not successful
#PULSAR_STOP_TIMEOUT=
+
+# Set BOOKIE options here to ensure that all pulsar scripts can work seamless
with bookkeeper
+
+# default settings for starting bookkeeper
+
+# Memory size options
+BOOKIE_MEM=" -Xms2g -Xmx2g -XX:MaxDirectMemorySize=2g"
+
+# Garbage collection options
+BOOKIE_GC=" -XX:+UseG1GC -XX:MaxGCPauseMillis=10 -XX:+ParallelRefProcEnabled
-XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis
-XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50
-XX:+DisableExplicitGC -XX:-ResizePLAB"
+
+# Extra options to be passed to the jvm
+BOOKIE_EXTRA_OPTS="${BOOKIE_EXTRA_OPTS} ${BOOKIE_MEM} ${BOOKIE_GC}
-Dio.netty.leakDetectionLevel=disabled
-Dio.netty.recycler.maxCapacity.default=1000
-Dio.netty.recycler.linkCapacity=1024"
Review comment:
@sijie Yeah, I also found the jvm settings are the same and can be reused.
Already changed, PTAL.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[pulsar] branch master updated: Do not strip ExecutionException from the stack trace (#4493)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 571b684 Do not strip ExecutionException from the stack trace (#4493)
571b684 is described below
commit 571b68464e25e8e71d3d278f58ba3a3df0c67a5b
Author: Matteo Merli
AuthorDate: Tue Jun 18 01:55:43 2019 -0700
Do not strip ExecutionException from the stack trace (#4493)
* Do not strip ExecutionException from the stack trace
---
.../pulsar/client/api/PulsarClientException.java | 85 ++
.../apache/pulsar/client/impl/ConsumerBase.java| 48 ++--
.../pulsar/client/impl/ConsumerBuilderImpl.java| 13 +---
.../apache/pulsar/client/impl/ConsumerImpl.java| 19 ++---
.../client/impl/MultiTopicsConsumerImpl.java | 22 ++
.../apache/pulsar/client/impl/ProducerBase.java| 37 ++
.../pulsar/client/impl/ProducerBuilderImpl.java| 13 +---
.../pulsar/client/impl/PulsarClientImpl.java | 15 +---
.../pulsar/client/impl/ReaderBuilderImpl.java | 13 +---
.../pulsar/client/impl/ZeroQueueConsumerImpl.java | 5 +-
.../client/impl/auth/AuthenticationBasic.java | 2 +-
.../pulsar/functions/instance/ContextImpl.java | 12 +--
12 files changed, 130 insertions(+), 154 deletions(-)
diff --git
a/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/PulsarClientException.java
b/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/PulsarClientException.java
index 7aa72e1..bb681ad 100644
---
a/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/PulsarClientException.java
+++
b/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/PulsarClientException.java
@@ -19,6 +19,7 @@
package org.apache.pulsar.client.api;
import java.io.IOException;
+import java.util.concurrent.ExecutionException;
/**
* Base type of exception thrown by Pulsar client
@@ -62,12 +63,20 @@ public class PulsarClientException extends IOException {
}
public static class TimeoutException extends PulsarClientException {
+public TimeoutException(Throwable t) {
+super(t);
+}
+
public TimeoutException(String msg) {
super(msg);
}
}
public static class IncompatibleSchemaException extends
PulsarClientException {
+public IncompatibleSchemaException(Throwable t) {
+super(t);
+}
+
public IncompatibleSchemaException(String msg) {
super(msg);
}
@@ -86,6 +95,10 @@ public class PulsarClientException extends IOException {
}
public static class ConnectException extends PulsarClientException {
+public ConnectException(Throwable t) {
+super(t);
+}
+
public ConnectException(String msg) {
super(msg);
}
@@ -212,4 +225,76 @@ public class PulsarClientException extends IOException {
super(msg);
}
}
+
+public static PulsarClientException unwrap(Throwable t) {
+if (t instanceof PulsarClientException) {
+return (PulsarClientException) t;
+} else if (t instanceof RuntimeException) {
+throw (RuntimeException) t;
+} else if (!(t instanceof ExecutionException)) {
+// Generic exception
+return new PulsarClientException(t);
+} else if (t instanceof InterruptedException) {
+Thread.currentThread().interrupt();
+return new PulsarClientException(t);
+}
+
+// Unwrap the exception to keep the same exception type but a stack
trace that includes the application calling
+// site
+Throwable cause = t.getCause();
+String msg = cause.getMessage();
+if (cause instanceof TimeoutException) {
+return new TimeoutException(msg);
+} else if (cause instanceof InvalidConfigurationException) {
+return new InvalidConfigurationException(msg);
+} else if (cause instanceof AuthenticationException) {
+return new AuthenticationException(msg);
+} else if (cause instanceof IncompatibleSchemaException) {
+return new IncompatibleSchemaException(msg);
+} else if (cause instanceof TooManyRequestsException) {
+return new TooManyRequestsException(msg);
+} else if (cause instanceof LookupException) {
+return new LookupException(msg);
+} else if (cause instanceof ConnectException) {
+return new ConnectException(msg);
+} else if (cause instanceof AlreadyClosedException) {
+return new AlreadyClosedException(msg);
+} else if (cause instanceof TopicTerminatedException) {
+return new TopicTerminatedException(msg);
+} else if (cause instanceof AuthorizationException) {
+return
[GitHub] [pulsar] codelipenghui merged pull request #4493: Do not strip ExecutionException from the stack trace
codelipenghui merged pull request #4493: Do not strip ExecutionException from the stack trace URL: https://github.com/apache/pulsar/pull/4493 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source
codelipenghui commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source URL: https://github.com/apache/pulsar/pull/3786#issuecomment-503014523 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4556: [tests] Scope the dependencies for spark and storm tests to test scope
sijie commented on issue #4556: [tests] Scope the dependencies for spark and storm tests to test scope URL: https://github.com/apache/pulsar/pull/4556#issuecomment-503023353 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4543: FailedCreatePodSandBox for bookie pods
sijie commented on issue #4543: FailedCreatePodSandBox for bookie pods URL: https://github.com/apache/pulsar/issues/4543#issuecomment-503024187 @ne1000 from the error message, it doesn't seem to be a pulsar problem. It looks like one of your kuberenetes worker had problem? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] zymap commented on issue #4546: Introduce multi host in pulsar doc
zymap commented on issue #4546: Introduce multi host in pulsar doc URL: https://github.com/apache/pulsar/pull/4546#issuecomment-503024308 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
sijie commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503025168 @KannarFr pulsar broker will to read the messages from offloaded segments when a topic consumer wants the messages. so it will consume inbound bandwidth to load the messages from S3 to brokers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503025607 @sijie You mean outbound? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
sijie commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503027893 @KannarFr I mean it consumes both inbound and outbound. the inbound bandwidth for loading the messages from S3 to broker. the outbound bandwidth for dispatching the messages from brokers to consumers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles LongSerializer (#4549)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 4c352f0 [pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles
LongSerializer (#4549)
4c352f0 is described below
commit 4c352f0371445a1e696f7a096357675f701d6cba
Author: Sijie Guo
AuthorDate: Tue Jun 18 02:36:44 2019 -0700
[pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles LongSerializer
(#4549)
*Motivation*
KafkaProducerInterceptorWrapper uses a LongDeserializer for retrieve
deserializer
*Modifications*
Fix the bug
*Verify this change*
Add unit test to cover the convertion
---
.../compat/KafkaProducerInterceptorWrapper.java| 5 +-
.../KafkaProducerInterceptorWrapperTest.java | 66 +++---
2 files changed, 60 insertions(+), 11 deletions(-)
diff --git
a/pulsar-client-kafka-compat/pulsar-client-kafka/src/main/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapper.java
b/pulsar-client-kafka-compat/pulsar-client-kafka/src/main/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapper.java
index 99195ce..5cedef9 100644
---
a/pulsar-client-kafka-compat/pulsar-client-kafka/src/main/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapper.java
+++
b/pulsar-client-kafka-compat/pulsar-client-kafka/src/main/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapper.java
@@ -34,6 +34,7 @@ import org.apache.kafka.common.serialization.DoubleSerializer;
import org.apache.kafka.common.serialization.IntegerDeserializer;
import org.apache.kafka.common.serialization.IntegerSerializer;
import org.apache.kafka.common.serialization.LongDeserializer;
+import org.apache.kafka.common.serialization.LongSerializer;
import org.apache.kafka.common.serialization.Serializer;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
@@ -232,10 +233,10 @@ public class KafkaProducerInterceptorWrapper
implements ProducerIntercepto
.getValue();
}
-private Deserializer getDeserializer(Serializer serializer) {
+static Deserializer getDeserializer(Serializer serializer) {
if (serializer instanceof StringSerializer) {
return new StringDeserializer();
-} else if (serializer instanceof LongDeserializer) {
+} else if (serializer instanceof LongSerializer) {
return new LongDeserializer();
} else if (serializer instanceof IntegerSerializer) {
return new IntegerDeserializer();
diff --git
a/pulsar-client-kafka-compat/pulsar-client-kafka/src/test/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapperTest.java
b/pulsar-client-kafka-compat/pulsar-client-kafka/src/test/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapperTest.java
index 8da6339..aadfce8 100644
---
a/pulsar-client-kafka-compat/pulsar-client-kafka/src/test/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapperTest.java
+++
b/pulsar-client-kafka-compat/pulsar-client-kafka/src/test/java/org/apache/pulsar/client/kafka/compat/KafkaProducerInterceptorWrapperTest.java
@@ -19,7 +19,21 @@
package org.apache.pulsar.client.kafka.compat;
import org.apache.kafka.clients.producer.ProducerRecord;
+import org.apache.kafka.common.serialization.ByteArrayDeserializer;
import org.apache.kafka.common.serialization.ByteArraySerializer;
+import org.apache.kafka.common.serialization.ByteBufferDeserializer;
+import org.apache.kafka.common.serialization.ByteBufferSerializer;
+import org.apache.kafka.common.serialization.BytesDeserializer;
+import org.apache.kafka.common.serialization.BytesSerializer;
+import org.apache.kafka.common.serialization.Deserializer;
+import org.apache.kafka.common.serialization.DoubleDeserializer;
+import org.apache.kafka.common.serialization.DoubleSerializer;
+import org.apache.kafka.common.serialization.IntegerDeserializer;
+import org.apache.kafka.common.serialization.IntegerSerializer;
+import org.apache.kafka.common.serialization.LongDeserializer;
+import org.apache.kafka.common.serialization.LongSerializer;
+import org.apache.kafka.common.serialization.Serializer;
+import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
import org.apache.pulsar.client.api.ProducerInterceptor;
import org.apache.pulsar.client.impl.ProducerInterceptors;
@@ -27,7 +41,7 @@ import org.apache.pulsar.client.impl.TypedMessageBuilderImpl;
import org.apache.pulsar.client.impl.schema.BytesSchema;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.stubbing.Answer;
-import org.testng.Assert;
+import org.testng.annotations.DataProvid
[GitHub] [pulsar] sijie merged pull request #4549: [pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles LongSerializer
sijie merged pull request #4549: [pulsar-kafka] Fix KafkaProducerInterceptorWrapper handles LongSerializer URL: https://github.com/apache/pulsar/pull/4549 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4544: [Doc] Fixed a typo in Cookbook / Topic compaction
sijie commented on issue #4544: [Doc] Fixed a typo in Cookbook / Topic compaction URL: https://github.com/apache/pulsar/pull/4544#issuecomment-503028954 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503029073 Oh, but I was talking about bandwidth on bucket not on broker. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4548: [schema] key/value schema enhancement
sijie commented on issue #4548: [schema] key/value schema enhancement URL: https://github.com/apache/pulsar/pull/4548#issuecomment-503029181 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr edited a comment on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr edited a comment on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503029073 Oh, but I was talking about bandwidth of bucket not of broker. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503029236 Anyway, same idea. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr edited a comment on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr edited a comment on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503029448 But huge consumption of bucket's outbound bandwidth. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503029448 But huge consumption of bucket outbound. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4550: Dashboard shows data only after running pulsar-admin command (docker, standalone)
sijie commented on issue #4550: Dashboard shows data only after running pulsar-admin command (docker, standalone) URL: https://github.com/apache/pulsar/issues/4550#issuecomment-503030569 @langep the current dashboard is using stats for displaying the resources. so when you run some operations on the cluster, the stats will be showing up and the dashboard will be able to collect stats and display the dashboard correctly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
sijie commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503031496 @KannarFr yes it is outbound bandwidth of buckets. but if you are going to read data from offloaded segments, don't you need to read those buckets and once you are reading those buckets, it is going to consume the outbound bandwidth, no? Is there any other approach to avoid outbound bandwidth on the buckets? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr closed issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr closed issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic
KannarFr commented on issue #4545: offloaded: outbound traffic >>> inbound traffic URL: https://github.com/apache/pulsar/issues/4545#issuecomment-503038887 Ok, it was to be sure. You are right. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on a change in pull request #4548: [schema] key/value schema enhancement
codelipenghui commented on a change in pull request #4548: [schema] key/value
schema enhancement
URL: https://github.com/apache/pulsar/pull/4548#discussion_r294732351
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -36,7 +37,7 @@
/**
* Message schema definition
*/
-public interface Schema {
+public interface Schema extends SchemaWriter {
Review comment:
Why consider to inherit SchemaWriter?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] zymap commented on issue #4546: Introduce multi host in pulsar doc
zymap commented on issue #4546: Introduce multi host in pulsar doc URL: https://github.com/apache/pulsar/pull/4546#issuecomment-503056961 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source
codelipenghui commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source URL: https://github.com/apache/pulsar/pull/3786#issuecomment-503057830 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4556: [tests] Scope the dependencies for spark and storm tests to test scope
codelipenghui commented on issue #4556: [tests] Scope the dependencies for spark and storm tests to test scope URL: https://github.com/apache/pulsar/pull/4556#issuecomment-503069942 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [tests] Scope the dependencies for spark and storm tests to test scope (#4556)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 5f5b609 [tests] Scope the dependencies for spark and storm tests to
test scope (#4556)
5f5b609 is described below
commit 5f5b6096f83bdae37427c6483bfd7cd6cc58a5cd
Author: Sijie Guo
AuthorDate: Tue Jun 18 07:22:42 2019 -0700
[tests] Scope the dependencies for spark and storm tests to test scope
(#4556)
*Motivation*
In gitsha b7a5677324a79708fd288fee6e8852f16a08ab6c , it introduced a
provided dependency for pulsar-storm-test, which it will fail release process.
Since it will attempt to transfer the pom file for
`com.twitter:carbonite:pom:1.5.0` which is a dependency comes from storm-server.
*Changes*
Make all the dependencies scoped at test scope for two test modules. So
maven release will not attempt to release them.
---
tests/pulsar-spark-test/pom.xml | 18 +++---
tests/pulsar-storm-test/pom.xml | 8 +++-
2 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/tests/pulsar-spark-test/pom.xml b/tests/pulsar-spark-test/pom.xml
index 5147673..130aef7 100644
--- a/tests/pulsar-spark-test/pom.xml
+++ b/tests/pulsar-spark-test/pom.xml
@@ -36,9 +36,10 @@
-org.apache.pulsar
-pulsar-spark
-${project.version}
+ org.apache.pulsar
+ pulsar-spark
+ ${project.version}
+ test
@@ -46,6 +47,7 @@
integration
${project.version}
test-jar
+ test
@@ -55,13 +57,15 @@
-org.mockito
-mockito-core
+ org.mockito
+ mockito-core
+ test
-org.apache.spark
-spark-streaming_2.10
+ org.apache.spark
+ spark-streaming_2.10
+ test
diff --git a/tests/pulsar-storm-test/pom.xml b/tests/pulsar-storm-test/pom.xml
index 59af212..82d28c1 100644
--- a/tests/pulsar-storm-test/pom.xml
+++ b/tests/pulsar-storm-test/pom.xml
@@ -39,6 +39,7 @@
org.apache.pulsar
pulsar-storm
${project.version}
+ test
org.apache.pulsar
@@ -51,7 +52,7 @@
org.apache.storm
storm-server
${storm.version}
- provided
+ test
ch.qos.logback
@@ -68,6 +69,7 @@
org.apache.pulsar
pulsar-broker
${project.version}
+ test
@@ -75,6 +77,7 @@
pulsar-broker
${project.version}
test-jar
+ test
@@ -82,6 +85,7 @@
managed-ledger-original
${project.version}
test-jar
+ test
@@ -92,12 +96,14 @@
org.asynchttpclient
async-http-client
+ test
org.apache.storm
storm-core
${storm.version}
+ test
ch.qos.logback
[GitHub] [pulsar] sijie merged pull request #4556: [tests] Scope the dependencies for spark and storm tests to test scope
sijie merged pull request #4556: [tests] Scope the dependencies for spark and storm tests to test scope URL: https://github.com/apache/pulsar/pull/4556 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper
sijie commented on issue #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper URL: https://github.com/apache/pulsar/pull/4555#issuecomment-503169309 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints (#4553)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 628bdd7 [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints (#4553) 628bdd7 is described below commit 628bdd73eb32bf8579a969625319eb328f4a8459 Author: Yijie Shen AuthorDate: Tue Jun 18 22:47:37 2019 +0800 [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints (#4553) `Tenant`, `namespace` and `sourceName` already appear in `PathParam` and shouldn't be a part of `sourceConfigJson`. --- .../pulsar/broker/admin/impl/FunctionsBase.java| 38 -- .../pulsar/broker/admin/impl/SourcesBase.java | 12 --- 2 files changed, 14 insertions(+), 36 deletions(-) diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java index 8a0f99c..11e20de 100644 --- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java +++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/FunctionsBase.java @@ -91,16 +91,10 @@ public class FunctionsBase extends AdminResource implements Supplier
[GitHub] [pulsar] sijie merged pull request #4553: [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints
sijie merged pull request #4553: [hotfix][minor]Remove tenant/namespace/name from sources and functions REST endpoints URL: https://github.com/apache/pulsar/pull/4553 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on a change in pull request #4548: [schema] key/value schema enhancement
sijie commented on a change in pull request #4548: [schema] key/value schema
enhancement
URL: https://github.com/apache/pulsar/pull/4548#discussion_r294868822
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/common/schema/KeyValue.java
##
@@ -37,4 +41,85 @@ public K getKey() {
public V getValue() {
return value;
}
+
+@Override
+public int hashCode() {
+return Objects.hash(key, value);
+}
+
+@Override
+public boolean equals(Object obj) {
+if (!(obj instanceof KeyValue)) {
+return false;
+}
+KeyValue another = (KeyValue) obj;
+return Objects.equals(key, another.key)
+&& Objects.equals(value, another.value);
+}
+
+@Override
+public String toString() {
+StringBuilder sb = new StringBuilder();
+sb.append("(key = \"")
+ .append(key)
+ .append("\", value = \"")
+ .append(value)
+ .append("\")");
+return sb.toString();
+}
+
+/**
+ * Decoder to decode key/value bytes.
+ */
+@FunctionalInterface
+public interface KeyValueDecoder {
+
+/**
+ * Decode key and value bytes into a {@link KeyValue} pair.
+ *
+ * @param keyData key data
+ * @param valueData value data
+ * @return the decoded {@link KeyValue} pair
+ */
+KeyValue decode(byte[] keyData, byte[] valueData);
+
+}
+
+/**
+ * Encode a key and value pair into a bytes array.
+ *
+ * @param key key object to encode
+ * @param keyWriter a writer to encode key object
+ * @param value value object to encode
+ * @param valueWriter a writer to encode value object
+ * @return the encoded bytes array
+ */
+public static byte[] encode(K key, SchemaWriter keyWriter,
Review comment:
@codelipenghui since I only need a serialize method for serializing K or V
into bytes. so I use `SchemaWriter` as the interface in the util functions. To
use existing Schema interface, I extend Schema interface to `SchemaWriter`.
Maybe I can just rewrite it using a different approach to get rid of extending
Schema from SchemaWriter. Let me give it a shot.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4544: [Doc] Fixed a typo in Cookbook / Topic compaction
sijie commented on issue #4544: [Doc] Fixed a typo in Cookbook / Topic compaction URL: https://github.com/apache/pulsar/pull/4544#issuecomment-503173820 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4358: [pulsar-io][jdbc sink]Support delete and update event for JDBC Sink
sijie commented on issue #4358: [pulsar-io][jdbc sink]Support delete and update event for JDBC Sink URL: https://github.com/apache/pulsar/pull/4358#issuecomment-503174058 @tuteng any updates? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #4358: [pulsar-io][jdbc sink]Support delete and update event for JDBC Sink
tuteng commented on issue #4358: [pulsar-io][jdbc sink]Support delete and update event for JDBC Sink URL: https://github.com/apache/pulsar/pull/4358#issuecomment-503178008 > @tuteng any updates? I'm very sorry, I haven't had time to do this recently. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #4541: Refactoring Function Component implementation
jerrypeng commented on issue #4541: Refactoring Function Component implementation URL: https://github.com/apache/pulsar/pull/4541#issuecomment-503224212 rerun cpp tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [Doc] Fixed a typo in Cookbook / Topic compaction (#4544)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new d1545bd [Doc] Fixed a typo in Cookbook / Topic compaction (#4544)
d1545bd is described below
commit d1545bda987e676c8b95aea86436146453573e72
Author: Daniel Andersson
AuthorDate: Tue Jun 18 19:34:24 2019 +0200
[Doc] Fixed a typo in Cookbook / Topic compaction (#4544)
## Motivation
Improving the quality of the documentation.
---
site2/docs/cookbooks-compaction.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/site2/docs/cookbooks-compaction.md
b/site2/docs/cookbooks-compaction.md
index 0a8f8ac..006c08f 100644
--- a/site2/docs/cookbooks-compaction.md
+++ b/site2/docs/cookbooks-compaction.md
@@ -17,7 +17,7 @@ To use compaction:
## When should I use compacted topics? {#when}
-The classic example of a topic that could benefit from compaction would be a
stock ticker topic through which consumers can access up-to-date values for
specific stocks. Imagine a scneario in which messages carrying stock value data
use the stock symbol as the key (`GOOG`, `AAPL`, `TWTR`, etc.). Compacting this
topic would give consumers on the topic two options:
+The classic example of a topic that could benefit from compaction would be a
stock ticker topic through which consumers can access up-to-date values for
specific stocks. Imagine a scenario in which messages carrying stock value data
use the stock symbol as the key (`GOOG`, `AAPL`, `TWTR`, etc.). Compacting this
topic would give consumers on the topic two options:
* They can read from the "original," non-compacted topic in case they need
access to "historical" values, i.e. the entirety of the topic's messages.
* They can read from the compacted topic if they only want to see the most
up-to-date messages.
[GitHub] [pulsar] sijie merged pull request #4544: [Doc] Fixed a typo in Cookbook / Topic compaction
sijie merged pull request #4544: [Doc] Fixed a typo in Cookbook / Topic compaction URL: https://github.com/apache/pulsar/pull/4544 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper
sijie commented on issue #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper URL: https://github.com/apache/pulsar/pull/4555#issuecomment-503248313 run java8 tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source
sijie commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source URL: https://github.com/apache/pulsar/pull/3786#issuecomment-503249170 @rdhabalia it seems the test failure PulsarFunctionE2ETest is related to this pull request. Do you mind taking a look? https://builds.apache.org/job/pulsar_precommit_java8/9833/testReport/junit/org.apache.pulsar.io/PulsarFunctionE2ETest/testPulsarFunctionStatus/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4548: [schema] key/value schema enhancement
sijie commented on issue #4548: [schema] key/value schema enhancement URL: https://github.com/apache/pulsar/pull/4548#issuecomment-503249767 @codelipenghui I have addressed your comment. PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] rdhabalia commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source
rdhabalia commented on issue #3786: [pulsar-function] fix deadlock in pulsar-source URL: https://github.com/apache/pulsar/pull/3786#issuecomment-503252337 sure, I will take a look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #4507: [Docs] Update site2/docs/functions-overview.md
jerrypeng commented on issue #4507: [Docs] Update site2/docs/functions-overview.md URL: https://github.com/apache/pulsar/pull/4507#issuecomment-503278175 rerun integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] sijie commented on issue #4548: [schema] key/value schema enhancement
sijie commented on issue #4548: [schema] key/value schema enhancement URL: https://github.com/apache/pulsar/pull/4548#issuecomment-503309996 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on issue #4538: [docs]Improve document of non-persistent topic
Anonymitaet commented on issue #4538: [docs]Improve document of non-persistent topic URL: https://github.com/apache/pulsar/pull/4538#issuecomment-503366014 @sijie @Anonymitaet Glad to help. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #4507: [Docs] Update site2/docs/functions-overview.md
Jennifer88huang commented on issue #4507: [Docs] Update site2/docs/functions-overview.md URL: https://github.com/apache/pulsar/pull/4507#issuecomment-503366686 run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang commented on issue #4554: [docs] Improve the structure of Pulsar Functions
Jennifer88huang commented on issue #4554: [docs] Improve the structure of Pulsar Functions URL: https://github.com/apache/pulsar/issues/4554#issuecomment-503375133 Is there any suggestion on the proposal? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng commented on issue #4554: [docs] Improve the structure of Pulsar Functions
jerrypeng commented on issue #4554: [docs] Improve the structure of Pulsar Functions URL: https://github.com/apache/pulsar/issues/4554#issuecomment-503376811 @Jennifer88huang I would change: 1. Function Worker -> Deployment or Cluster Deployment? Since first timers would even know what "Function worker" is. 2. Deploy -> Function Deployment? For more info on how to run functions. While I am ok with having separate headings/pages for Java, Python, and Go functions, we can also have them together but just have different examples for different languages like: https://spark.apache.org/docs/latest/quick-start.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4548: [schema] key/value schema enhancement
codelipenghui commented on issue #4548: [schema] key/value schema enhancement URL: https://github.com/apache/pulsar/pull/4548#issuecomment-503378011 run java8 tests run integration tests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui commented on issue #4548: [schema] key/value schema enhancement
codelipenghui commented on issue #4548: [schema] key/value schema enhancement URL: https://github.com/apache/pulsar/pull/4548#issuecomment-503377972 @sijie 👍, LGTM. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #4538: [docs]Improve document of non-persistent topic
Anonymitaet commented on a change in pull request #4538: [docs]Improve document
of non-persistent topic
URL: https://github.com/apache/pulsar/pull/4538#discussion_r295090584
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/NonPersistentTopics.java
##
@@ -66,9 +67,22 @@
@GET
@Path("/{tenant}/{namespace}/{topic}/partitions")
@ApiOperation(value = "Get partitioned topic metadata.")
-@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have
admin permission") })
-public PartitionedTopicMetadata
getPartitionedMetadata(@PathParam("tenant") String tenant,
-@PathParam("namespace") String namespace, @PathParam("topic")
@Encoded String encodedTopic,
+@ApiResponses(value = {
+@ApiResponse(code = 401, message = "Don't have permission to
administrate resources on this tenant"),
+@ApiResponse(code = 403, message = "Don't have admin permission"),
+@ApiResponse(code = 404, message = "tenant/namespace/topic doesn't
exit does not exist"),
+@ApiResponse(code = 412, message = "Topic name is not valid"),
+@ApiResponse(code = 500, message = "Internal server error"),
+@ApiResponse(code = 503, message = "Failed to validate Cluster
configuration")
Review comment:
Why capitalize the first letter of "Cluster"?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #4538: [docs]Improve document of non-persistent topic
Anonymitaet commented on a change in pull request #4538: [docs]Improve document
of non-persistent topic
URL: https://github.com/apache/pulsar/pull/4538#discussion_r295090482
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/NonPersistentTopics.java
##
@@ -66,9 +67,22 @@
@GET
@Path("/{tenant}/{namespace}/{topic}/partitions")
@ApiOperation(value = "Get partitioned topic metadata.")
-@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have
admin permission") })
-public PartitionedTopicMetadata
getPartitionedMetadata(@PathParam("tenant") String tenant,
-@PathParam("namespace") String namespace, @PathParam("topic")
@Encoded String encodedTopic,
+@ApiResponses(value = {
+@ApiResponse(code = 401, message = "Don't have permission to
administrate resources on this tenant"),
+@ApiResponse(code = 403, message = "Don't have admin permission"),
+@ApiResponse(code = 404, message = "tenant/namespace/topic doesn't
exit does not exist"),
Review comment:
```suggestion
@ApiResponse(code = 404, message = "tenant/namespace/topic does
not exist"),
```
Please check all similar cases.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #4538: [docs]Improve document of non-persistent topic
Anonymitaet commented on a change in pull request #4538: [docs]Improve document
of non-persistent topic
URL: https://github.com/apache/pulsar/pull/4538#discussion_r295090088
##
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/NonPersistentTopics.java
##
@@ -66,9 +67,22 @@
@GET
@Path("/{tenant}/{namespace}/{topic}/partitions")
@ApiOperation(value = "Get partitioned topic metadata.")
-@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have
admin permission") })
-public PartitionedTopicMetadata
getPartitionedMetadata(@PathParam("tenant") String tenant,
-@PathParam("namespace") String namespace, @PathParam("topic")
@Encoded String encodedTopic,
+@ApiResponses(value = {
+@ApiResponse(code = 401, message = "Don't have permission to
administrate resources on this tenant"),
Review comment:
```suggestion
@ApiResponse(code = 401, message = "Don't have permission to
manage resources on this tenant"),
```
Want to express the meaning of "manage resources"? If so, please check all
similar cases.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai commented on a change in pull request #4548: [schema] key/value schema enhancement
jiazhai commented on a change in pull request #4548: [schema] key/value schema
enhancement
URL: https://github.com/apache/pulsar/pull/4548#discussion_r294661892
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -54,6 +55,13 @@ default void validate(byte[] message) {
decode(message);
}
+/**
+ * {@inheritDoc}
+ */
+default byte[] write(T message) {
Review comment:
what is the benefit to have a write method？
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai commented on a change in pull request #4548: [schema] key/value schema enhancement
jiazhai commented on a change in pull request #4548: [schema] key/value schema
enhancement
URL: https://github.com/apache/pulsar/pull/4548#discussion_r294661892
##
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/Schema.java
##
@@ -54,6 +55,13 @@ default void validate(byte[] message) {
decode(message);
}
+/**
+ * {@inheritDoc}
+ */
+default byte[] write(T message) {
Review comment:
what is the benefit to have a write method？
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] jiazhai merged pull request #4546: Introduce multi host in pulsar doc
jiazhai merged pull request #4546: Introduce multi host in pulsar doc URL: https://github.com/apache/pulsar/pull/4546 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: Introduce multi host in pulsar doc (#4546)
This is an automated email from the ASF dual-hosted git repository.
zhaijia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2e94523 Introduce multi host in pulsar doc (#4546)
2e94523 is described below
commit 2e9452363fd53ece33bddde43c5655bd63d09d8a
Author: Yong Zhang
AuthorDate: Wed Jun 19 10:44:48 2019 +0800
Introduce multi host in pulsar doc (#4546)
Pulsar already have been supported multi host. Introduce them in Pulsar
document.
---
site2/docs/admin-api-overview.md| 20
site2/docs/client-libraries-java.md | 12
site2/docs/deploy-bare-metal.md | 2 ++
3 files changed, 34 insertions(+)
diff --git a/site2/docs/admin-api-overview.md b/site2/docs/admin-api-overview.md
index 7b48c86..df12d3e 100644
--- a/site2/docs/admin-api-overview.md
+++ b/site2/docs/admin-api-overview.md
@@ -70,3 +70,23 @@ config.setTlsTrustCertsFilePath(tlsTrustCertsFilePath);
PulsarAdmin admin = new PulsarAdmin(url, config);
```
+
+If you have multiple brokers to use, you can use multi-host like Pulsar
service. For example,
+```java
+URL url = new URL("http://localhost:8080,localhost:8081,localhost:8082";);
+// Pass auth-plugin class fully-qualified name if Pulsar-security is enabled.
+String authPluginClassName = "com.org.MyAuthPluginClass";
+// Pass auth-param if auth-plugin class requires it
+String authParams = "param1=value1";
+boolean useTls = false;
+boolean tlsAllowInsecureConnection = false;
+String tlsTrustCertsFilePath = null;
+
+ClientConfiguration config = new ClientConfiguration();
+config.setAuthentication(authPluginClassName, authParams);
+config.setUseTls(useTls);
+config.setTlsAllowInsecureConnection(tlsAllowInsecureConnection);
+config.setTlsTrustCertsFilePath(tlsTrustCertsFilePath);
+
+PulsarAdmin admin = new PulsarAdmin(url, config);
+```
diff --git a/site2/docs/client-libraries-java.md
b/site2/docs/client-libraries-java.md
index 4c4261f..2a07981 100644
--- a/site2/docs/client-libraries-java.md
+++ b/site2/docs/client-libraries-java.md
@@ -57,6 +57,11 @@ Pulsar protocol URLs are assigned to specific clusters, use
the `pulsar` scheme
pulsar://localhost:6650
```
+If you have more than one broker, the URL may look like this:
+```http
+pulsar://localhost:6550,localhost:6651,localhost:6652
+```
+
A URL for a production Pulsar cluster may look something like this:
```http
@@ -79,6 +84,13 @@ PulsarClient client = PulsarClient.builder()
.build();
```
+If you have multiple brokers, you can initiate a PulsarClient like this:
+```java
+PulsarClient client = PulsarClient.builder()
+.serviceUrl("pulsar://localhost:6650,localhost:6651,localhost:6652")
+.build();
+```
+
> Default broker URLs for standalone clusters
> If you're running a cluster in [standalone
> mode](getting-started-standalone.md), the broker will be available at the
> `pulsar://localhost:6650` URL by default.
diff --git a/site2/docs/deploy-bare-metal.md b/site2/docs/deploy-bare-metal.md
index 7a69227..6ed6a77 100644
--- a/site2/docs/deploy-bare-metal.md
+++ b/site2/docs/deploy-bare-metal.md
@@ -43,6 +43,8 @@ To run Pulsar on bare metal, you are recommended to have:
> However if you don't have enough machines, or are trying out Pulsar in
> cluster mode (and expand the cluster later),
> you can even deploy Pulsar in one node, where it will run zookeeper, bookie
> and broker in same machine.
+> If you don't have a DNS server, you can use multi-host in service URL
instead.
+
Each machine in your cluster will need to have [Java
8](http://www.oracle.com/technetwork/java/javase/downloads/index.html) or
higher installed.
Here's a diagram showing the basic setup:
[GitHub] [pulsar] Jennifer88huang commented on issue #4554: [docs] Improve the structure of Pulsar Functions
Jennifer88huang commented on issue #4554: [docs] Improve the structure of Pulsar Functions URL: https://github.com/apache/pulsar/issues/4554#issuecomment-503384535 > 1. Function Worker -> Deployment or Cluster Deployment? >Since first timers would even know what "Function worker" is. @sijie Any thoughts on this? > 2. Deploy -> Function Deployment? >For more info on how to run functions. Sure, we can change it as "Function Deployment". > While I am ok with having separate headings/pages for Java, Python, and Go functions, we can also have them together but just have different examples for different languages like: > > https://spark.apache.org/docs/latest/quick-start.html @jerrypeng It's great suggestion to adopt this structure. Smart idea. Do you know how to implement the switch pattern on our website? I mean switch between "Java/Python/Go examples". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang edited a comment on issue #4554: [docs] Improve the structure of Pulsar Functions
Jennifer88huang edited a comment on issue #4554: [docs] Improve the structure of Pulsar Functions URL: https://github.com/apache/pulsar/issues/4554#issuecomment-503384535 > 1. Function Worker -> Deployment or Cluster Deployment? >Since first timers would even know what "Function worker" is. @sijie Any thoughts on this? > 2. Deploy -> Function Deployment? >For more info on how to run functions. Sure, we can change it as "Function Deployment". > While I am ok with having separate headings/pages for Java, Python, and Go functions, we can also have them together but just have different examples for different languages like: > > https://spark.apache.org/docs/latest/quick-start.html @jerrypeng It's great suggestion to adopt this structure. Smart idea. Do you know how to implement the switch pattern on our website? I mean switch between "Java/Python/Go examples". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jerrypeng edited a comment on issue #4554: [docs] Improve the structure of Pulsar Functions
jerrypeng edited a comment on issue #4554: [docs] Improve the structure of Pulsar Functions URL: https://github.com/apache/pulsar/issues/4554#issuecomment-503376811 @Jennifer88huang I would change: 1. Function Worker -> Deployment or Cluster Deployment? Since first timers wouldn't even know what "Function worker" is. 2. Deploy -> Function Deployment? For more info on how to run functions. While I am ok with having separate headings/pages for Java, Python, and Go functions, we can also have them together but just have different examples for different languages like: https://spark.apache.org/docs/latest/quick-start.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] Jennifer88huang edited a comment on issue #4554: [docs] Improve the structure of Pulsar Functions
Jennifer88huang edited a comment on issue #4554: [docs] Improve the structure of Pulsar Functions URL: https://github.com/apache/pulsar/issues/4554#issuecomment-503384535 > 1. Function Worker -> Deployment or Cluster Deployment? >Since first timers wouldn't even know what "Function worker" is. @sijie Any thoughts on this? > 2. Deploy -> Function Deployment? >For more info on how to run functions. Sure, we can change it as "Function Deployment". > While I am ok with having separate headings/pages for Java, Python, and Go functions, we can also have them together but just have different examples for different languages like: > > https://spark.apache.org/docs/latest/quick-start.html @jerrypeng It's great suggestion to adopt this structure. Smart idea. Do you know how to implement the switch pattern on our website? I mean switch between "Java/Python/Go examples". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] jiazhai commented on issue #4400: PIP 37: [pulsar-client] support large message size
jiazhai commented on issue #4400: PIP 37: [pulsar-client] support large message size URL: https://github.com/apache/pulsar/pull/4400#issuecomment-503387942 Hi @rdhabalia, sorry for the delay. After PIP36 (#4247 ), pulsar could support message size large than 5MB. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[pulsar] branch master updated: [Issue 4422] Support pulsar scripts to work with bookkeeper (#4555)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 34f1a58 [Issue 4422] Support pulsar scripts to work with bookkeeper
(#4555)
34f1a58 is described below
commit 34f1a58b3b9f4c60d837701cecb2fac67a04a634
Author: Fangbin Sun
AuthorDate: Wed Jun 19 11:11:28 2019 +0800
[Issue 4422] Support pulsar scripts to work with bookkeeper (#4555)
Support pulsar scripts to work with bookkeeper
---
bin/pulsar | 2 ++
conf/pulsar_env.sh | 5 +
2 files changed, 7 insertions(+)
diff --git a/bin/pulsar b/bin/pulsar
index 9cd8911..ad7736a 100755
--- a/bin/pulsar
+++ b/bin/pulsar
@@ -293,6 +293,8 @@ if [ $COMMAND == "broker" ]; then
exec $JAVA $OPTS $ASPECTJ_AGENT -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.pulsar.PulsarBrokerStarter --broker-conf $PULSAR_BROKER_CONF $@
elif [ $COMMAND == "bookie" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"bookkeeper.log"}
+# Pass BOOKIE_EXTRA_OPTS option defined in pulsar_env.sh
+OPTS="$OPTS $BOOKIE_EXTRA_OPTS"
exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.bookkeeper.proto.BookieServer --conf $PULSAR_BOOKKEEPER_CONF $@
elif [ $COMMAND == "zookeeper" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"zookeeper.log"}
diff --git a/conf/pulsar_env.sh b/conf/pulsar_env.sh
index 36034b5..7428d1d 100755
--- a/conf/pulsar_env.sh
+++ b/conf/pulsar_env.sh
@@ -58,3 +58,8 @@ PULSAR_EXTRA_OPTS=${PULSAR_EXTRA_OPTS:-"
-Dpulsar.allocator.exit_on_oom=true -Di
#Wait time before forcefully kill the pulser server instance, if the stop is
not successful
#PULSAR_STOP_TIMEOUT=
+
+# Set BOOKIE_EXTRA_OPTS option here to ensure that all pulsar scripts can work
seamless with bookkeeper
+
+# Extra options to be passed to the jvm
+BOOKIE_EXTRA_OPTS="${BOOKIE_EXTRA_OPTS} -Dio.netty.leakDetectionLevel=disabled
-Dio.netty.recycler.maxCapacity.default=1000
-Dio.netty.recycler.linkCapacity=1024"
\ No newline at end of file
[GitHub] [pulsar] codelipenghui closed issue #4422: Bookie can't start in the background by pulsar-daemon when passing extra opts
codelipenghui closed issue #4422: Bookie can't start in the background by pulsar-daemon when passing extra opts URL: https://github.com/apache/pulsar/issues/4422 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] codelipenghui merged pull request #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper
codelipenghui merged pull request #4555: [Issue 4422] Support pulsar scripts to work with bookkeeper URL: https://github.com/apache/pulsar/pull/4555 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] MarvinCai opened a new pull request #4557: [WIP][transaction][acknowledge] Persist pending ack messages to cursor meta store.
MarvinCai opened a new pull request #4557: [WIP][transaction][acknowledge] Persist pending ack messages to cursor meta store. URL: https://github.com/apache/pulsar/pull/4557 Add message SubscriptionPendingAckMessages to store pending ack message positions. Add update/getSubscriptionPendingAckMessages to managed ledger MetaStore for storing/getting pending ack message positions. Persisst pending ack message positions when acknowledging message and recover pending ack message positions when initializing subscription. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] wolfstudy commented on a change in pull request #4533: Add subscription modes for java client doc.
wolfstudy commented on a change in pull request #4533: Add subscription modes
for java client doc.
URL: https://github.com/apache/pulsar/pull/4533#discussion_r295128501
##
File path: site2/docs/client-libraries-java.md
##
@@ -306,6 +306,185 @@ consumerBuilder
});
```
+### Subscription modes
+
+Pulsar has various [subscription modes](concepts-messaging#subscription-modes)
to match different scenarios. A topic can have multiple subscriptions with
different subscription modes. However, a subscription can only have one
subscription mode at a time.
+
+A subscription is identified with the subscription name, and a subscription
name can specify only one subscription mode at a time. You can change the
subscription mode, yet you have to let all existing consumers of this
subscription offline first.
+
+Different subscription modes have different message distribution modes. This
section describes the differences of subscription modes and how to use them.
+
+In order to better describe their differences, assuming you have a topic named
"my-topic", and the producer has published 10 messages.
+
+```java
+Producer producer = client.newProducer(Schema.STRING)
+.topic("my-topic")
+.enableBatch(false)
+.create();
+// 3 messages with "key-1", 3 messages with "key-2", 2 messages with "key-3"
and 2 messages with "key-4"
+producer.newMessage().key("key-1").value("message-1-1").send();
+producer.newMessage().key("key-1").value("message-1-2").send();
+producer.newMessage().key("key-1").value("message-1-3").send();
+producer.newMessage().key("key-2").value("message-2-1").send();
+producer.newMessage().key("key-2").value("message-2-2").send();
+producer.newMessage().key("key-2").value("message-2-3").send();
+producer.newMessage().key("key-3").value("message-3-1").send();
+producer.newMessage().key("key-3").value("message-3-2").send();
+producer.newMessage().key("key-4").value("message-4-1").send();
+producer.newMessage().key("key-4").value("message-4-2").send();
+```
+
+ Exclusive
+
+Create a new consumer and subscribe with the `Exclusive` subscription mode.
+
+```java
+Consumer consumer = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Exclusive)
+.subscribe()
+```
+
+Only the first consumer is allowed to the subscription, other consumers
receive an error. The first consumer receives all 10 messages, and the
consuming order is the same as the producing order.
+
+> Note:
+>
+> If topic is a partitioned topic, the first consumer subscribes to all
partitioned topics, other consumers are not assigned with partitions and
receive an error.
+
+ Failover
+
+Create new consumers and subscribe with the`Failover` subscription mode.
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+//conumser1 is the active consumer, consumer2 is the standby consumer.
+//consumer1 receives 5 messages and then crashes, consumer2 takes over as an
active consumer.
+
+
+```
+
+Multiple consumers can attach to the same subscription, yet only the first
consumer is active, and others are standby. When the active consumer is
disconnected, messages will be dispatched to one of standby consumers, and the
standby consumer becomes active consumer.
+
+If the first active consumer receives 5 messages and is disconnected, the
standby consumer becomes active consumer. Consumer1 will receive:
+
+```
+("key-1", "message-1-1")
+("key-1", "message-1-2")
+("key-1", "message-1-3")
+("key-2", "message-2-1")
+("key-2", "message-2-2")
+```
+
+consumer2 will receive:
+
+```
+("key-2", "message-2-3")
+("key-3", "message-3-1")
+("key-3", "message-3-2")
+("key-4", "message-4-1")
+("key-4", "message-4-2")
+```
+
+> Note:
+>
+> If a topic is a partitioned topic, each partition only has one active
consumer, messages of one partition only distributed to one consumer, messages
of multiple partitions are distributed to multiple consumers.
+
+ Shared
+
+Create new consumers and subscribe with `Shared` subscription mode:
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Shared)
+.subscribe()
+
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Shared)
+.subscribe()
+//Both consumer1 and consumer 2 is active consumers.
+```
+
+In shared subscription mode, multiple consumers can attach to the same
subscription and
[GitHub] [pulsar] wolfstudy commented on a change in pull request #4533: Add subscription modes for java client doc.
wolfstudy commented on a change in pull request #4533: Add subscription modes
for java client doc.
URL: https://github.com/apache/pulsar/pull/4533#discussion_r295128815
##
File path: site2/docs/client-libraries-java.md
##
@@ -306,6 +306,185 @@ consumerBuilder
});
```
+### Subscription modes
+
+Pulsar has various [subscription modes](concepts-messaging#subscription-modes)
to match different scenarios. A topic can have multiple subscriptions with
different subscription modes. However, a subscription can only have one
subscription mode at a time.
+
+A subscription is identified with the subscription name, and a subscription
name can specify only one subscription mode at a time. You can change the
subscription mode, yet you have to let all existing consumers of this
subscription offline first.
+
+Different subscription modes have different message distribution modes. This
section describes the differences of subscription modes and how to use them.
+
+In order to better describe their differences, assuming you have a topic named
"my-topic", and the producer has published 10 messages.
+
+```java
+Producer producer = client.newProducer(Schema.STRING)
+.topic("my-topic")
+.enableBatch(false)
+.create();
+// 3 messages with "key-1", 3 messages with "key-2", 2 messages with "key-3"
and 2 messages with "key-4"
+producer.newMessage().key("key-1").value("message-1-1").send();
+producer.newMessage().key("key-1").value("message-1-2").send();
+producer.newMessage().key("key-1").value("message-1-3").send();
+producer.newMessage().key("key-2").value("message-2-1").send();
+producer.newMessage().key("key-2").value("message-2-2").send();
+producer.newMessage().key("key-2").value("message-2-3").send();
+producer.newMessage().key("key-3").value("message-3-1").send();
+producer.newMessage().key("key-3").value("message-3-2").send();
+producer.newMessage().key("key-4").value("message-4-1").send();
+producer.newMessage().key("key-4").value("message-4-2").send();
+```
+
+ Exclusive
+
+Create a new consumer and subscribe with the `Exclusive` subscription mode.
+
+```java
+Consumer consumer = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Exclusive)
+.subscribe()
+```
+
+Only the first consumer is allowed to the subscription, other consumers
receive an error. The first consumer receives all 10 messages, and the
consuming order is the same as the producing order.
+
+> Note:
+>
+> If topic is a partitioned topic, the first consumer subscribes to all
partitioned topics, other consumers are not assigned with partitions and
receive an error.
+
+ Failover
+
+Create new consumers and subscribe with the`Failover` subscription mode.
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Failover)
+.subscribe()
+//conumser1 is the active consumer, consumer2 is the standby consumer.
+//consumer1 receives 5 messages and then crashes, consumer2 takes over as an
active consumer.
+
+
+```
+
+Multiple consumers can attach to the same subscription, yet only the first
consumer is active, and others are standby. When the active consumer is
disconnected, messages will be dispatched to one of standby consumers, and the
standby consumer becomes active consumer.
+
+If the first active consumer receives 5 messages and is disconnected, the
standby consumer becomes active consumer. Consumer1 will receive:
+
+```
+("key-1", "message-1-1")
+("key-1", "message-1-2")
+("key-1", "message-1-3")
+("key-2", "message-2-1")
+("key-2", "message-2-2")
+```
+
+consumer2 will receive:
+
+```
+("key-2", "message-2-3")
+("key-3", "message-3-1")
+("key-3", "message-3-2")
+("key-4", "message-4-1")
+("key-4", "message-4-2")
+```
+
+> Note:
+>
+> If a topic is a partitioned topic, each partition only has one active
consumer, messages of one partition only distributed to one consumer, messages
of multiple partitions are distributed to multiple consumers.
+
+ Shared
+
+Create new consumers and subscribe with `Shared` subscription mode:
+
+```java
+Consumer consumer1 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Shared)
+.subscribe()
+
+Consumer consumer2 = client.newConsumer()
+.topic("my-topic")
+.subscriptionName("my-subscription")
+.subscriptionType(SubscriptionType.Shared)
+.subscribe()
+//Both consumer1 and consumer 2 is active consumers.
+```
+
+In shared subscription mode, multiple consumers can attach to the same
subscription and
