[GitHub] rdhabalia commented on issue #1223: Add a `internalConfiguration` admin restful endpoint for query internal configuration

[GitBox]

rdhabalia commented on issue #1223: Add a `internalConfiguration` admin restful 
endpoint for query internal configuration
URL: https://github.com/apache/incubator-pulsar/pull/1223#issuecomment-365839106
 
 
   LGTM.. just few minor comments..


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on a change in pull request #1223: Add a `internalConfiguration` admin restful endpoint for query internal configuration

[GitBox]

rdhabalia commented on a change in pull request #1223: Add a 
`internalConfiguration` admin restful endpoint for query internal configuration
URL: https://github.com/apache/incubator-pulsar/pull/1223#discussion_r168390204
 
 

 ##
 File path: 
pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/PulsarAdmin.java
 ##
 @@ -154,6 +156,7 @@ public PulsarAdmin(URL serviceUrl, ClientConfiguration 
pulsarConfig) throws Puls
 WebTarget root = client.target(serviceUrl.toString());
 web = root.path("/admin");
 
+this.internalConfiguration = new InternalConfigurationImpl(web, auth);
 
 Review comment:
   should it be part of `brokers` as this configuration has been configured at 
broker (eg: `pulsar().getConfiguration().getxxx()`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia commented on a change in pull request #1223: Add a `internalConfiguration` admin restful endpoint for query internal configuration

[GitBox]

rdhabalia commented on a change in pull request #1223: Add a 
`internalConfiguration` admin restful endpoint for query internal configuration
URL: https://github.com/apache/incubator-pulsar/pull/1223#discussion_r168390403
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/InternalConfiguration.java
 ##
 @@ -0,0 +1,50 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.admin;
+
+import io.swagger.annotations.Api;
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.ApiResponse;
+import io.swagger.annotations.ApiResponses;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import org.apache.bookkeeper.conf.ClientConfiguration;
+import org.apache.pulsar.common.conf.InternalConfigurationData;
+
+@Path("/internal-configuration")
+@Api(value = "/internal-configuration", description = "InternalConfiguration 
admin apis", tags = "internal-configuration")
+@Produces(MediaType.APPLICATION_JSON)
+public class InternalConfiguration extends AdminResource {
+
+@GET
+@ApiOperation(value = "Get the internal configuration data", response = 
InternalConfigurationData.class)
+@ApiResponses(value = {
+@ApiResponse(code = 403, message = "Don't have admin permission")
 
 Review comment:
   in which scenario it returns 403?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed issue #1058: Provide a mechanism to tell whether a consumer is the leader of a failover subscription

[GitBox]

merlimat closed issue #1058: Provide a mechanism to tell whether a consumer is 
the leader of a failover subscription
URL: https://github.com/apache/incubator-pulsar/issues/1058
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group (#1156)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new f2160c0  Introduce ActiveConsumerListener for realizing if a consumer 
is active in a failover subscription group (#1156)
f2160c0 is described below

commit f2160c01e3581f0e8374db5d4d713810de7533af
Author: Sijie Guo 
AuthorDate: Thu Feb 15 12:12:00 2018 +0800

Introduce ActiveConsumerListener for realizing if a consumer is active in a 
failover subscription group (#1156)

* Introduce ConsumerGroupListener for realizing if a consumer is active in 
a failover subscription group

* Rename ConsumerGroupListener to ActiveConsumerListener

* Fail subscribe if active consumer listener is provided with non-failover 
subscription.

* Notify consumer state change only after the cursor is rewinded.

* Address comments

* Rename ActiveConsumerListener to ConsumerEventListener
Use consumer object as identifier for comparison

* Fix test after rebase

* Fix license headers

* Ignore active consumer change command in cpp client

* rename "become" to "became"
---
 .../AbstractDispatcherSingleActiveConsumer.java|  38 +-
 .../org/apache/pulsar/broker/service/Consumer.java |  15 +
 .../PersistentDispatcherSingleActiveConsumer.java  |  10 +-
 .../PersistentDispatcherFailoverConsumerTest.java  | 157 ++-
 .../broker/service/PersistentFailoverE2ETest.java  |  96 +++-
 pulsar-client-cpp/lib/ClientConnection.cc  |   7 +
 pulsar-client-cpp/lib/Commands.cc  |   3 +
 .../pulsar/client/api/ConsumerConfiguration.java   |  29 ++
 .../pulsar/client/api/ConsumerEventListener.java   |  36 ++
 .../org/apache/pulsar/client/impl/ClientCnx.java   |  14 +
 .../apache/pulsar/client/impl/ConsumerBase.java|   3 +
 .../apache/pulsar/client/impl/ConsumerImpl.java|  14 +
 .../client/impl/PartitionedConsumerImpl.java   |   4 +-
 .../pulsar/client/impl/PulsarClientImpl.java   |   7 +
 .../org/apache/pulsar/common/api/Commands.java |  18 +
 .../apache/pulsar/common/api/PulsarDecoder.java|  11 +-
 .../apache/pulsar/common/api/proto/PulsarApi.java  | 486 +
 pulsar-common/src/main/proto/PulsarApi.proto   |  12 +-
 .../pulsar/common/api/PulsarDecoderTest.java   |  60 +++
 19 files changed, 993 insertions(+), 27 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
index 5181df1..1b37f89 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
@@ -26,9 +26,9 @@ import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;
 import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
 
 import org.apache.pulsar.broker.service.BrokerServiceException;
-import org.apache.pulsar.broker.service.Consumer;
 import 
org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyException;
 import 
org.apache.pulsar.broker.service.BrokerServiceException.ServerMetadataException;
+import org.apache.pulsar.broker.service.Consumer;
 import org.apache.pulsar.common.api.proto.PulsarApi.CommandSubscribe.SubType;
 import org.apache.pulsar.utils.CopyOnWriteArrayList;
 import org.slf4j.Logger;
@@ -72,7 +72,17 @@ public abstract class AbstractDispatcherSingleActiveConsumer 
{
 
 protected abstract boolean isConsumersExceededOnSubscription();
 
-protected void pickAndScheduleActiveConsumer() {
+protected void notifyActiveConsumerChanged(Consumer activeConsumer) {
+if (null != activeConsumer && subscriptionType == SubType.Failover) {
+consumers.forEach(consumer ->
+consumer.notifyActiveConsumerChange(activeConsumer));
+}
+}
+
+/**
+ * @return the previous active consumer if the consumer is changed, 
otherwise null.
+ */
+protected boolean pickAndScheduleActiveConsumer() {
 checkArgument(!consumers.isEmpty());
 
 consumers.sort((c1, c2) -> 
c1.consumerName().compareTo(c2.consumerName()));
@@ -80,12 +90,15 @@ public abstract class 
AbstractDispatcherSingleActiveConsumer {
 int index = partitionIndex % consumers.size();
 Consumer prevConsumer = ACTIVE_CONSUMER_UPDATER.getAndSet(this, 
consumers.get(index));
 
-if (prevConsumer == ACTIVE_CONSUMER_UPDATER.get(this)) {
+Consumer activeConsumer = ACTIVE_CONSUMER_UPDATER.get(this);
+if (prevConsumer == activeConsumer) {
 // Active consumer did not change. Do nothing at this point
- 

[GitHub] merlimat closed pull request #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

merlimat closed pull request #1156: Introduce ActiveConsumerListener for 
realizing if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
index 5181df177..1b37f8946 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherSingleActiveConsumer.java
@@ -26,9 +26,9 @@
 import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
 
 import org.apache.pulsar.broker.service.BrokerServiceException;
-import org.apache.pulsar.broker.service.Consumer;
 import 
org.apache.pulsar.broker.service.BrokerServiceException.ConsumerBusyException;
 import 
org.apache.pulsar.broker.service.BrokerServiceException.ServerMetadataException;
+import org.apache.pulsar.broker.service.Consumer;
 import org.apache.pulsar.common.api.proto.PulsarApi.CommandSubscribe.SubType;
 import org.apache.pulsar.utils.CopyOnWriteArrayList;
 import org.slf4j.Logger;
@@ -72,7 +72,17 @@ public AbstractDispatcherSingleActiveConsumer(SubType 
subscriptionType, int part
 
 protected abstract boolean isConsumersExceededOnSubscription();
 
-protected void pickAndScheduleActiveConsumer() {
+protected void notifyActiveConsumerChanged(Consumer activeConsumer) {
+if (null != activeConsumer && subscriptionType == SubType.Failover) {
+consumers.forEach(consumer ->
+consumer.notifyActiveConsumerChange(activeConsumer));
+}
+}
+
+/**
+ * @return the previous active consumer if the consumer is changed, 
otherwise null.
+ */
+protected boolean pickAndScheduleActiveConsumer() {
 checkArgument(!consumers.isEmpty());
 
 consumers.sort((c1, c2) -> 
c1.consumerName().compareTo(c2.consumerName()));
@@ -80,12 +90,15 @@ protected void pickAndScheduleActiveConsumer() {
 int index = partitionIndex % consumers.size();
 Consumer prevConsumer = ACTIVE_CONSUMER_UPDATER.getAndSet(this, 
consumers.get(index));
 
-if (prevConsumer == ACTIVE_CONSUMER_UPDATER.get(this)) {
+Consumer activeConsumer = ACTIVE_CONSUMER_UPDATER.get(this);
+if (prevConsumer == activeConsumer) {
 // Active consumer did not change. Do nothing at this point
-return;
+return false;
+} else {
+// If the active consumer is changed, send notification.
+scheduleReadOnActiveConsumer();
+return true;
 }
-
-scheduleReadOnActiveConsumer();
 }
 
 public synchronized void addConsumer(Consumer consumer) throws 
BrokerServiceException {
@@ -109,8 +122,17 @@ public synchronized void addConsumer(Consumer consumer) 
throws BrokerServiceExce
 
 consumers.add(consumer);
 
-// Pick an active consumer and start it
-pickAndScheduleActiveConsumer();
+if (!pickAndScheduleActiveConsumer()) {
+// the active consumer is not changed
+Consumer currentActiveConsumer = ACTIVE_CONSUMER_UPDATER.get(this);
+if (null == currentActiveConsumer) {
+if (log.isDebugEnabled()) {
+log.debug("Current active consumer disappears while adding 
consumer {}", consumer);
+}
+} else {
+consumer.notifyActiveConsumerChange(currentActiveConsumer);
+}
+}
 
 }
 
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
index fd77ef290..0bb30cfb6 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
@@ -152,6 +152,21 @@ public String consumerName() {
 return consumerName;
 }
 
+void notifyActiveConsumerChange(Consumer activeConsumer) {
+if 
(!Commands.peerSupportsActiveConsumerListener(cnx.getRemoteEndpointProtocolVersion()))
 {
+// if the client is older than `v12`, we don't need to send 
consumer group changes.
+return;
+}
+
+if (log.isDebugEnabled()) {
+log.debug("notify consumer {} - that [{}] for subscription {} has 
new active consumer : {}",
+consumerId, topicName, subscription.getName(), activeConsumer);
+}
+

svn commit: r25081 - /dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/

[jai1]

Author: jai1
Date: Thu Feb 15 02:06:00 2018
New Revision: 25081

Log:
Staging artifacts and signature for Pulsar release 1.22.0-incubating

Added:

dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz
   (with props)

dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.asc

dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.md5

dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.sha512
Modified:

dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-src.tar.gz.asc

Added: 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz
==
Binary file - no diff available.

Propchange: 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.asc
==
--- 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.asc
 (added)
+++ 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.asc
 Thu Feb 15 02:06:00 2018
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCAAdFiEEDQCP4t9TLRC/fG0se6GmTLvBFOwFAlqE6tAACgkQe6GmTLvB
+FOxOCg/+LOavQKTSrdrtpX6gUCVuXvnDABAAgE37tRNJfsLwmfC8G+ZaNyb6vSLf
+nbCnAwNwi/Nj0X/4H9ht4mBxrJZ78Ii3UtBjT3lA4naMDZcyPmN/5ISY5RNscVln
+wMi9nt8DdWKV3XR0XWHFx9tgrKOhPfkb9xdTturljj8/a3oW3yhOFpBaUve+KFkJ
+CX3ibEAv9Cd5iIvrv0rPTUdPxsK2ht7JYewdLUxtHhAJmWkFKcdT8InLqrqb2tmE
+IqT9i/SY2LGW+hyq28pdnjzwa5g6Qu6ilrN69LRBJ/5oSwvfS8+H3C9yPYDh2WVF
+vsYclpy1BQftRPqR+LAyIZcmotywmR6CNkGRBEj+/M+CsFpc06E0ixPWlJDV9UEb
+adSCnHmohwwD3TS8OmMNI/O7ipcLB5Mamor6Gi9SYjHA+pM0AG+sSRpzV67YxdIi
++x61YBeZ1OnmYhlj9b0tFSMCkQXwlnxoIC1liU8rG133V2+9wt6W4BN4TjB74rWW
+50mECdvGDhjty9AF1HbuA54PA2/GwKV2eW0eAd7DYCq1U+CaP7yUrSagSknAnvXB
+50tm6EaSt4Hx2S32xvUVg4pn86XAfO6ePOKywCExHQxqDzenTkvMeYssZbnPHxxe
+FBYUa0EqSJmbkJ7cq+Hjzv8RNchEjmEWdL1qOquYZeN/qfXXazE=
+=/xWX
+-END PGP SIGNATURE-

Added: 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.md5
==
--- 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.md5
 (added)
+++ 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.md5
 Thu Feb 15 02:06:00 2018
@@ -0,0 +1,2 @@
+apache-pulsar-1.22.0-incubating-bin.tar.gz: 
+98 41 A5 01 81 98 9F 4F  44 E7 0D 6B 1D 15 3E E2

Added: 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.sha512
==
--- 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.sha512
 (added)
+++ 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-bin.tar.gz.sha512
 Thu Feb 15 02:06:00 2018
@@ -0,0 +1,3 @@
+apache-pulsar-1.22.0-incubating-bin.tar.gz: 
+408DDB47 98355343 23626766 EB2FFD64 7E77105C F7F6FE92 AC42502C A2FBC632 
6BCECD5C
+ 43177395 643A10E7 F080B159 ECB4A2A5 D2D01508 83EBED26 6CECDA9A

Modified: 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-src.tar.gz.asc
==
--- 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-src.tar.gz.asc
 (original)
+++ 
dev/incubator/pulsar/pulsar-1.22.0-incubating-candidate-0/apache-pulsar-1.22.0-incubating-src.tar.gz.asc
 Thu Feb 15 02:06:00 2018
@@ -1,16 +1,16 @@
 -BEGIN PGP SIGNATURE-
 
-iQIzBAABCAAdFiEEDQCP4t9TLRC/fG0se6GmTLvBFOwFAlqCtq8ACgkQe6GmTLvB
-FOyEwhAAt0OFdejzQTJi7KA2Qv+mhazVuBEZfygwyRjNy7pNoX+CI0tIDCo6myDd
-jfUjx3Fg6hFR3WGzLW0+n+meHxsWrcpiqoR6vV7PjH+CpxJXY6+ZrS3/0QuDdAT/
-/ZlSStC8S3AT6mSRJzjab9HGWIwsE+QZmuiLzjzkUTBDNZ4QAstbN57MY4DrDDe3
-HbSKoCDrPWp+eWmuimD2U1fhxMpPKaUonXLvH1jAhsLYNUt+yS7o0UOlFm+gbktK
-h2t4T+t2QeHzefkh8+Tud1EV0wObTbel54qda3LKGwgGUwPCMs7UxZ/oeSJTd25E
-oKAizymEcdcnpPHBYfq7pr/QhPW2bpO1OVjVKBk8mN3SjVO1366MawxrAfr1AKby
-wQcy8kVN2YskM0VSYl+Ck5BRJl//NO59oGFIPtp7IXy0F/yqokwkVJC4I7YPM3ua
-r5Ozixl8+NFDPE+8SVrmUASiHJLbjG7RBT3SyWuzEMV642uQY6FHp6NMlncBczkK
-slkXz/U5i+E/GM+5Q+MZ7HmwzrCmicYgqiUI8pryYLLPxRjtpiZ9f074jiRtXxFO
-5HRZFackUg0Zz2e0pBhv8FCfLvMc4NiCt8LvqmcPNnXEjMc6m4/Vz+6UheLRBRK+

[GitHub] merlimat commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

merlimat commented on issue #1156: Introduce ActiveConsumerListener for 
realizing if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365801659
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

merlimat commented on issue #1156: Introduce ActiveConsumerListener for 
realizing if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365801659
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch branch-1.22 updated: add httpcore dep required by httpclient in hostname-verification (#1239)

[rdhabalia]

This is an automated email from the ASF dual-hosted git repository.

rdhabalia pushed a commit to branch branch-1.22
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/branch-1.22 by this push:
 new 96d720c  add httpcore dep required by httpclient in 
hostname-verification (#1239)
96d720c is described below

commit 96d720c928b21485df8e055dde379aeeb7f81624
Author: Rajan Dhabalia 
AuthorDate: Wed Feb 14 16:29:37 2018 -0800

add httpcore dep required by httpclient in hostname-verification (#1239)
---
 all/src/assemble/LICENSE.bin.txt   | 1 +
 pulsar-broker-shaded/pom.xml   | 7 +++
 pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml | 9 +
 pulsar-client-shaded/pom.xml   | 7 +++
 pulsar-client/pom.xml  | 7 ++-
 5 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/all/src/assemble/LICENSE.bin.txt b/all/src/assemble/LICENSE.bin.txt
index a7e70de..944ba2d 100644
--- a/all/src/assemble/LICENSE.bin.txt
+++ b/all/src/assemble/LICENSE.bin.txt
@@ -333,6 +333,7 @@ The Apache Software License, Version 2.0
  * SnakeYaml -- org.yaml-snakeyaml-*.jar
  * RocksDB - org.rocksdb.*.jar
  * HttpClient - org.apache.httpcomponents.httpclient.jar
+ * HttCore - org.apache.httpcomponents.httpcore.jar
  * CommonsLogging - commons-logging-*.jar
 
 BSD 3-clause "New" or "Revised" License
diff --git a/pulsar-broker-shaded/pom.xml b/pulsar-broker-shaded/pom.xml
index 5794983..6d552bf 100644
--- a/pulsar-broker-shaded/pom.xml
+++ b/pulsar-broker-shaded/pom.xml
@@ -106,6 +106,7 @@
   com.wordnik:swagger-annotations
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
@@ -121,6 +122,12 @@
 **
   
 
+
+   commons-logging:commons-logging
+   
+   **
+   
+
   
   
 
diff --git a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml 
b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
index fe601bf..ea4b71c 100644
--- a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
+++ b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
@@ -127,8 +127,17 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
+   
+
+   commons-logging:commons-logging
+   
+   **
+   
+
+  
   
 
   
org.apache.kafka.clients.producer.KafkaProducer
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index bebc36d..81f873e 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -83,6 +83,7 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
@@ -98,6 +99,12 @@
 **
   
 
+
+   commons-logging:commons-logging
+   
+   **
+   
+
   
   
 
diff --git a/pulsar-client/pom.xml b/pulsar-client/pom.xml
index df6cc8d..abd0300 100644
--- a/pulsar-client/pom.xml
+++ b/pulsar-client/pom.xml
@@ -86,12 +86,17 @@
   
 
 
- 
+ 
 
   commons-logging
   commons-logging
   1.1.1
 
+
+  org.apache.httpcomponents
+  httpcore
+  4.4.9
+
 
   
 

-- 
To stop receiving notification emails like this one, please contact
rdhaba...@apache.org.

[GitHub] merlimat closed pull request #1239: add httpcore dep required by httpclient in hostname-verification

[GitBox]

merlimat closed pull request #1239: add httpcore dep required by httpclient in 
hostname-verification
URL: https://github.com/apache/incubator-pulsar/pull/1239
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/all/src/assemble/LICENSE.bin.txt b/all/src/assemble/LICENSE.bin.txt
index a7e70defb..944ba2d1a 100644
--- a/all/src/assemble/LICENSE.bin.txt
+++ b/all/src/assemble/LICENSE.bin.txt
@@ -333,6 +333,7 @@ The Apache Software License, Version 2.0
  * SnakeYaml -- org.yaml-snakeyaml-*.jar
  * RocksDB - org.rocksdb.*.jar
  * HttpClient - org.apache.httpcomponents.httpclient.jar
+ * HttCore - org.apache.httpcomponents.httpcore.jar
  * CommonsLogging - commons-logging-*.jar
 
 BSD 3-clause "New" or "Revised" License
diff --git a/pulsar-broker-shaded/pom.xml b/pulsar-broker-shaded/pom.xml
index efcfa76dc..e8539e751 100644
--- a/pulsar-broker-shaded/pom.xml
+++ b/pulsar-broker-shaded/pom.xml
@@ -106,6 +106,7 @@
   com.wordnik:swagger-annotations
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
@@ -121,6 +122,12 @@
 **
   
 
+
+   commons-logging:commons-logging
+   
+   **
+   
+
   
   
 
diff --git a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml 
b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
index 906000372..1be125024 100644
--- a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
+++ b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
@@ -127,8 +127,17 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
+   
+
+   commons-logging:commons-logging
+   
+   **
+   
+
+  
   
 
   
org.apache.kafka.clients.producer.KafkaProducer
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index cf1d7b567..58bba0e52 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -83,6 +83,7 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
@@ -98,6 +99,12 @@
 **
   
 
+
+   commons-logging:commons-logging
+   
+   **
+   
+
   
   
 
diff --git a/pulsar-client/pom.xml b/pulsar-client/pom.xml
index 57c69c8ea..3ab403fbe 100644
--- a/pulsar-client/pom.xml
+++ b/pulsar-client/pom.xml
@@ -86,12 +86,17 @@
   
 
 
- 
+ 
 
   commons-logging
   commons-logging
   1.1.1
 
+
+  org.apache.httpcomponents
+  httpcore
+  4.4.9
+
 
   
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: add httpcore dep required by httpclient in hostname-verification (#1239)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 350d297  add httpcore dep required by httpclient in 
hostname-verification (#1239)
350d297 is described below

commit 350d2973703f99f3a36e6a56a4348a6b76415c3f
Author: Rajan Dhabalia 
AuthorDate: Wed Feb 14 16:29:37 2018 -0800

add httpcore dep required by httpclient in hostname-verification (#1239)
---
 all/src/assemble/LICENSE.bin.txt   | 1 +
 pulsar-broker-shaded/pom.xml   | 7 +++
 pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml | 9 +
 pulsar-client-shaded/pom.xml   | 7 +++
 pulsar-client/pom.xml  | 7 ++-
 5 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/all/src/assemble/LICENSE.bin.txt b/all/src/assemble/LICENSE.bin.txt
index a7e70de..944ba2d 100644
--- a/all/src/assemble/LICENSE.bin.txt
+++ b/all/src/assemble/LICENSE.bin.txt
@@ -333,6 +333,7 @@ The Apache Software License, Version 2.0
  * SnakeYaml -- org.yaml-snakeyaml-*.jar
  * RocksDB - org.rocksdb.*.jar
  * HttpClient - org.apache.httpcomponents.httpclient.jar
+ * HttCore - org.apache.httpcomponents.httpcore.jar
  * CommonsLogging - commons-logging-*.jar
 
 BSD 3-clause "New" or "Revised" License
diff --git a/pulsar-broker-shaded/pom.xml b/pulsar-broker-shaded/pom.xml
index efcfa76..e8539e7 100644
--- a/pulsar-broker-shaded/pom.xml
+++ b/pulsar-broker-shaded/pom.xml
@@ -106,6 +106,7 @@
   com.wordnik:swagger-annotations
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
@@ -121,6 +122,12 @@
 **
   
 
+
+   commons-logging:commons-logging
+   
+   **
+   
+
   
   
 
diff --git a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml 
b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
index 9060003..1be1250 100644
--- a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
+++ b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
@@ -127,8 +127,17 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
+   
+
+   commons-logging:commons-logging
+   
+   **
+   
+
+  
   
 
   
org.apache.kafka.clients.producer.KafkaProducer
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index cf1d7b5..58bba0e 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -83,6 +83,7 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
@@ -98,6 +99,12 @@
 **
   
 
+
+   commons-logging:commons-logging
+   
+   **
+   
+
   
   
 
diff --git a/pulsar-client/pom.xml b/pulsar-client/pom.xml
index 57c69c8..3ab403f 100644
--- a/pulsar-client/pom.xml
+++ b/pulsar-client/pom.xml
@@ -86,12 +86,17 @@
   
 
 
- 
+ 
 
   commons-logging
   commons-logging
   1.1.1
 
+
+  org.apache.httpcomponents
+  httpcore
+  4.4.9
+
 
   
 

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] sijie opened a new issue #1240: Implement ActiveConsumerListener for pulsar cpp client

[GitBox]

sijie opened a new issue #1240: Implement ActiveConsumerListener for pulsar cpp 
client
URL: https://github.com/apache/incubator-pulsar/issues/1240
 
 
   ActiveConsumerListener is introduced in #1156 . We need similar feature 
implemented in cpp client.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing 
if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365780104
 
 
   the CI failed with license header. fixed license header.
   
   I forgot another item to make CPP client ignore this notification.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Make max clients per topic/subscription configurable (#1234)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 5613c2b  Make max clients per topic/subscription configurable (#1234)
5613c2b is described below

commit 5613c2b92f6b7989d6a4c65e259713953e933e8f
Author: hrsakai 
AuthorDate: Thu Feb 15 08:00:23 2018 +0900

Make max clients per topic/subscription configurable (#1234)
---
 conf/broker.conf   |  15 ++
 conf/standalone.conf   |  15 ++
 .../apache/pulsar/broker/ServiceConfiguration.java |  40 -
 .../AbstractDispatcherSingleActiveConsumer.java|  27 ++--
 .../broker/service/BrokerServiceException.java |   8 +-
 .../NonPersistentDispatcherMultipleConsumers.java  |  39 -
 ...onPersistentDispatcherSingleActiveConsumer.java |  22 ++-
 .../service/nonpersistent/NonPersistentTopic.java  |  23 ++-
 .../PersistentDispatcherMultipleConsumers.java |  29 +++-
 .../PersistentDispatcherSingleActiveConsumer.java  |  16 ++
 .../broker/service/persistent/PersistentTopic.java |  22 +++
 .../pulsar/broker/service/PersistentTopicTest.java | 172 -
 12 files changed, 406 insertions(+), 22 deletions(-)

diff --git a/conf/broker.conf b/conf/broker.conf
index d18ebac..dbea41d 100644
--- a/conf/broker.conf
+++ b/conf/broker.conf
@@ -168,6 +168,21 @@ enableRunBookieTogether=false
 # Enable to run bookie autorecovery along with broker
 enableRunBookieAutoRecoveryTogether=false
 
+// Max number of producers allowed to connect to topic. Once this limit 
reaches, Broker will reject new producers
+// until the number of connected producers decrease.
+// Using a value of 0, is disabling maxProducersPerTopic-limit check.
+maxProducersPerTopic=0
+
+// Max number of consumers allowed to connect to topic. Once this limit 
reaches, Broker will reject new consumers
+// until the number of connected consumers decrease.
+// Using a value of 0, is disabling maxConsumersPerTopic-limit check.
+maxConsumersPerTopic=0
+
+// Max number of consumers allowed to connect to subscription. Once this limit 
reaches, Broker will reject new consumers
+// until the number of connected consumers decrease.
+// Using a value of 0, is disabling maxConsumersPerSubscription-limit check.
+maxConsumersPerSubscription=0
+
 ### --- Authentication --- ###
 # Role names that are treated as "proxy roles". If the broker sees a request 
with
 #role as proxyRoles - it will demand to see a valid original principal.
diff --git a/conf/standalone.conf b/conf/standalone.conf
index 1201dce..a6c915e 100644
--- a/conf/standalone.conf
+++ b/conf/standalone.conf
@@ -151,6 +151,21 @@ enablePersistentTopics=true
 # Enable broker to load non-persistent topics
 enableNonPersistentTopics=true
 
+// Max number of producers allowed to connect to topic. Once this limit 
reaches, Broker will reject new producers
+// until the number of connected producers decrease.
+// Using a value of 0, is disabling maxProducersPerTopic-limit check.
+maxProducersPerTopic=0
+
+// Max number of consumers allowed to connect to topic. Once this limit 
reaches, Broker will reject new consumers
+// until the number of connected consumers decrease.
+// Using a value of 0, is disabling maxConsumersPerTopic-limit check.
+maxConsumersPerTopic=0
+
+// Max number of consumers allowed to connect to subscription. Once this limit 
reaches, Broker will reject new consumers
+// until the number of connected consumers decrease.
+// Using a value of 0, is disabling maxConsumersPerSubscription-limit check.
+maxConsumersPerSubscription=0
+
 ### --- Authentication --- ###
 # Role names that are treated as "proxy roles". If the broker sees a request 
with
 #role as proxyRoles - it will demand to see a valid original principal.
diff --git 
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
 
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
index fcd9a92..4be2195 100644
--- 
a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
+++ 
b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
@@ -169,6 +169,21 @@ public class ServiceConfiguration implements 
PulsarConfiguration {
 // Enable to run bookie autorecovery along with broker
 private boolean enableRunBookieAutoRecoveryTogether = false;
 
+// Max number of producers allowed to connect to topic. Once this limit 
reaches, Broker will reject new producers
+// until the number of connected producers decrease.
+// Using a value of 0, is disabling maxProducersPerTopic-limit check.
+private int maxProducersPerTopic = 0;
+
+// Max number of consumers allowed to connect to topic. Once this limit 
reaches, Broker will reject new consumers
+// 

[GitHub] rdhabalia opened a new pull request #1239: add httpcore dep required by httpclient in hostname-verification

[GitBox]

rdhabalia opened a new pull request #1239: add httpcore dep required by 
httpclient in hostname-verification
URL: https://github.com/apache/incubator-pulsar/pull/1239
 
 
   ### Motivation
   
   - when a broker's tls certificate present, httpclient's 
DefaultHostnameVerifier requires httpcore dependencies to use 
`org.apache.http.util.Args` class.
   
   ```
   11:12:33.486 [pulsar-discovery-io-2-3] WARN  
o.a.p.p.server.DirectProxyHandler- [[id: 0x300e8efa, L:/10.213.246.124:6651 
- R:/10.215.134.167:51374]] [[id: 0x2b69265f, L:/10.213.246.124:48686 - R:perbs
   java.lang.NoClassDefFoundError: org/apache/http/util/Args
   at org.apache.http.conn.ssl.SubjectName.(SubjectName.java:48) 
~[httpclient-4.5.5.jar:4.5.5]
   at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.getSubjectAltNames(DefaultHostnameVerifier.java:310)
 ~[httpclient-4.5.5.jar:4.5.5]
   at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:112)
 ~[httpclient-4.5.5.jar:4.5.5]
   at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:99)
 ~[httpclient-4.5.5.jar:4.5.5]
   at 
   ```
   - Also `DefaultHostnameVerifier` requires all classes of commons-logging.
   
   ### Modifications
   add httpcore dependencies into client/proxy where hostname verification 
happens.
   
   ### Result
   
   It fixes `java.lang.NoClassDefFoundError: org/apache/http/util/Args` while 
hostname verification.
   Note: I will cherry-pick in 1.22 branch as well.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1103: PIP-13-1/3: Provide `TopicsConsumer` to consume from several topics under same namespace

[GitBox]

merlimat commented on a change in pull request #1103: PIP-13-1/3: Provide 
`TopicsConsumer` to consume from several topics under same namespace
URL: https://github.com/apache/incubator-pulsar/pull/1103#discussion_r168322621
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/TopicsConsumerImpl.java
 ##
 @@ -0,0 +1,881 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl;
+
+import static com.google.common.base.Preconditions.checkArgument;
+import static com.google.common.base.Preconditions.checkState;
+
+import com.google.common.collect.Lists;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerConfiguration;
+import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.client.api.SubscriptionType;
+import org.apache.pulsar.client.util.ConsumerName;
+import org.apache.pulsar.client.util.FutureUtil;
+import org.apache.pulsar.common.api.proto.PulsarApi.CommandAck.AckType;
+import org.apache.pulsar.common.naming.DestinationName;
+import org.apache.pulsar.common.naming.NamespaceName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TopicsConsumerImpl extends ConsumerBase {
+
+// All topics should be in same namespace
+protected NamespaceName namespaceName;
+
+// Map , when get do ACK, consumer will by find 
by topic name
+private final ConcurrentHashMap consumers;
 
 Review comment:
   I would suggest to just rename `PartitionedConsumerImpl`, change the 
constructor to accept a list of topics (rather than the number of partitions) 
and switch to use `TopicMessageIdImpl`. I don't think that 
`UnAckedMessageTracker` usage would need to be changed. 
   
   That would avoid to add a lot of new code and having to merge it later.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] rdhabalia closed pull request #1238: add httpcore dep required by httpclient in hostname-verification

[GitBox]

rdhabalia closed pull request #1238: add httpcore dep required by httpclient in 
hostname-verification
URL: https://github.com/apache/incubator-pulsar/pull/1238
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/all/src/assemble/LICENSE.bin.txt b/all/src/assemble/LICENSE.bin.txt
index a7e70defb..944ba2d1a 100644
--- a/all/src/assemble/LICENSE.bin.txt
+++ b/all/src/assemble/LICENSE.bin.txt
@@ -333,6 +333,7 @@ The Apache Software License, Version 2.0
  * SnakeYaml -- org.yaml-snakeyaml-*.jar
  * RocksDB - org.rocksdb.*.jar
  * HttpClient - org.apache.httpcomponents.httpclient.jar
+ * HttCore - org.apache.httpcomponents.httpcore.jar
  * CommonsLogging - commons-logging-*.jar
 
 BSD 3-clause "New" or "Revised" License
diff --git a/pulsar-broker-shaded/pom.xml b/pulsar-broker-shaded/pom.xml
index efcfa76dc..3b2650960 100644
--- a/pulsar-broker-shaded/pom.xml
+++ b/pulsar-broker-shaded/pom.xml
@@ -106,6 +106,7 @@
   com.wordnik:swagger-annotations
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
diff --git a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml 
b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
index 906000372..b1e5cad05 100644
--- a/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
+++ b/pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
@@ -127,6 +127,7 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
diff --git a/pulsar-client-shaded/pom.xml b/pulsar-client-shaded/pom.xml
index cf1d7b567..dd58d5dca 100644
--- a/pulsar-client-shaded/pom.xml
+++ b/pulsar-client-shaded/pom.xml
@@ -83,6 +83,7 @@
   com.yahoo.datasketches:sketches-core
   org.apache.httpcomponents:httpclient
   commons-logging:commons-logging
+  org.apache.httpcomponents:httpcore
 
   
   
diff --git a/pulsar-client/pom.xml b/pulsar-client/pom.xml
index 57c69c8ea..3ab403fbe 100644
--- a/pulsar-client/pom.xml
+++ b/pulsar-client/pom.xml
@@ -86,12 +86,17 @@
   
 
 
- 
+ 
 
   commons-logging
   commons-logging
   1.1.1
 
+
+  org.apache.httpcomponents
+  httpcore
+  4.4.9
+
 
   
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168294644
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   OK. I had originally proposed that but it was eliminated based on some input 
but I don't recall the specific circumstances.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Add non-persistent topic stats separately in brokers-stat (#1235)

[rdhabalia]

This is an automated email from the ASF dual-hosted git repository.

rdhabalia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 3c9e281  Add non-persistent topic stats separately in brokers-stat 
(#1235)
3c9e281 is described below

commit 3c9e28172d486bfb4c39db5248e1033d5079c48f
Author: Rajan Dhabalia 
AuthorDate: Wed Feb 14 12:03:38 2018 -0800

Add non-persistent topic stats separately in brokers-stat (#1235)
---
 .../apache/pulsar/broker/service/PulsarStats.java  | 45 +-
 1 file changed, 36 insertions(+), 9 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java
index cdc5ada..835cabc 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java
@@ -25,6 +25,7 @@ import java.util.concurrent.locks.ReentrantReadWriteLock;
 import java.util.function.Consumer;
 
 import org.apache.pulsar.broker.PulsarService;
+import org.apache.pulsar.broker.service.nonpersistent.NonPersistentTopic;
 import org.apache.pulsar.broker.service.persistent.PersistentTopic;
 import org.apache.pulsar.broker.stats.BrokerOperabilityMetrics;
 import org.apache.pulsar.broker.stats.ClusterReplicationMetrics;
@@ -55,6 +56,7 @@ public class PulsarStats implements Closeable {
 private Map bundleStats;
 private List tempMetricsCollection;
 private List metricsCollection;
+private List tempNonPersistentTopics;
 private final BrokerOperabilityMetrics brokerOperabilityMetrics;
 
 private final ReentrantReadWriteLock bufferLock = new 
ReentrantReadWriteLock();
@@ -71,6 +73,7 @@ public class PulsarStats implements Closeable {
 this.metricsCollection = Lists.newArrayList();
 this.brokerOperabilityMetrics = new 
BrokerOperabilityMetrics(pulsar.getConfiguration().getClusterName(),
 pulsar.getAdvertisedAddress());
+this.tempNonPersistentTopics = Lists.newArrayList();
 }
 
 @Override
@@ -118,22 +121,46 @@ public class PulsarStats implements Closeable {
 currentBundleStats.topics = topics.size();
 
 
topicStatsStream.startObject(NamespaceBundle.getBundleRange(bundle));
+
+tempNonPersistentTopics.clear();
+// start persistent topic
 topicStatsStream.startObject("persistent");
 topics.forEach((name, topic) -> {
-try {
-topic.updateRates(nsStats, currentBundleStats, 
topicStatsStream,
-clusterReplicationMetrics, 
namespaceName);
-} catch (Exception e) {
-log.error("Failed to generate topic stats for 
topic {}: {}", name, e.getMessage(), e);
-}
-// this task: helps to activate 
inactive-backlog-cursors which have caught up and
-// connected, also deactivate 
active-backlog-cursors which has backlog
 if (topic instanceof PersistentTopic) {
+try {
+topic.updateRates(nsStats, 
currentBundleStats, topicStatsStream,
+clusterReplicationMetrics, 
namespaceName);
+} catch (Exception e) {
+log.error("Failed to generate topic stats 
for topic {}: {}", name, e.getMessage(), e);
+}
+// this task: helps to activate 
inactive-backlog-cursors which have caught up and
+// connected, also deactivate 
active-backlog-cursors which has backlog
 ((PersistentTopic) 
topic).getManagedLedger().checkBackloggedCursors();
+}else if (topic instanceof NonPersistentTopic) {
+
tempNonPersistentTopics.add((NonPersistentTopic) topic);
+} else {
+log.warn("Unsupported type of topic {}", 
topic.getClass().getName());
 }
 });
-
+// end persistent topics section
 topicStatsStream.endObject();
+
+if(!tempNonPersistentTopics.isEmpty()) {
+ // start non-persistent topic
+topicStatsStream.startObject("non-persistent");
+

[GitHub] rdhabalia closed pull request #1235: Add non-persistent topic stats separately in brokers-stat

[GitBox]

rdhabalia closed pull request #1235: Add non-persistent topic stats separately 
in brokers-stat
URL: https://github.com/apache/incubator-pulsar/pull/1235
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java
index cdc5ada99..835cabccd 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java
@@ -25,6 +25,7 @@
 import java.util.function.Consumer;
 
 import org.apache.pulsar.broker.PulsarService;
+import org.apache.pulsar.broker.service.nonpersistent.NonPersistentTopic;
 import org.apache.pulsar.broker.service.persistent.PersistentTopic;
 import org.apache.pulsar.broker.stats.BrokerOperabilityMetrics;
 import org.apache.pulsar.broker.stats.ClusterReplicationMetrics;
@@ -55,6 +56,7 @@
 private Map bundleStats;
 private List tempMetricsCollection;
 private List metricsCollection;
+private List tempNonPersistentTopics;
 private final BrokerOperabilityMetrics brokerOperabilityMetrics;
 
 private final ReentrantReadWriteLock bufferLock = new 
ReentrantReadWriteLock();
@@ -71,6 +73,7 @@ public PulsarStats(PulsarService pulsar) {
 this.metricsCollection = Lists.newArrayList();
 this.brokerOperabilityMetrics = new 
BrokerOperabilityMetrics(pulsar.getConfiguration().getClusterName(),
 pulsar.getAdvertisedAddress());
+this.tempNonPersistentTopics = Lists.newArrayList();
 }
 
 @Override
@@ -118,22 +121,46 @@ public synchronized void updateStats(
 currentBundleStats.topics = topics.size();
 
 
topicStatsStream.startObject(NamespaceBundle.getBundleRange(bundle));
+
+tempNonPersistentTopics.clear();
+// start persistent topic
 topicStatsStream.startObject("persistent");
 topics.forEach((name, topic) -> {
-try {
-topic.updateRates(nsStats, currentBundleStats, 
topicStatsStream,
-clusterReplicationMetrics, 
namespaceName);
-} catch (Exception e) {
-log.error("Failed to generate topic stats for 
topic {}: {}", name, e.getMessage(), e);
-}
-// this task: helps to activate 
inactive-backlog-cursors which have caught up and
-// connected, also deactivate 
active-backlog-cursors which has backlog
 if (topic instanceof PersistentTopic) {
+try {
+topic.updateRates(nsStats, 
currentBundleStats, topicStatsStream,
+clusterReplicationMetrics, 
namespaceName);
+} catch (Exception e) {
+log.error("Failed to generate topic stats 
for topic {}: {}", name, e.getMessage(), e);
+}
+// this task: helps to activate 
inactive-backlog-cursors which have caught up and
+// connected, also deactivate 
active-backlog-cursors which has backlog
 ((PersistentTopic) 
topic).getManagedLedger().checkBackloggedCursors();
+}else if (topic instanceof NonPersistentTopic) {
+
tempNonPersistentTopics.add((NonPersistentTopic) topic);
+} else {
+log.warn("Unsupported type of topic {}", 
topic.getClass().getName());
 }
 });
-
+// end persistent topics section
 topicStatsStream.endObject();
+
+if(!tempNonPersistentTopics.isEmpty()) {
+ // start non-persistent topic
+topicStatsStream.startObject("non-persistent");
+tempNonPersistentTopics.forEach(topic -> {
+try {
+topic.updateRates(nsStats, 
currentBundleStats, topicStatsStream,
+clusterReplicationMetrics, 
namespaceName);
+} catch (Exception e) {
+log.error("Failed to generate 

[GitHub] rdhabalia opened a new pull request #1238: add httpcore dep required by httpclient in hostname-verification

[GitBox]

rdhabalia opened a new pull request #1238: add httpcore dep required by 
httpclient in hostname-verification
URL: https://github.com/apache/incubator-pulsar/pull/1238
 
 
   ### Motivation
   
   when a broker's tls certificate present, httpclient's 
DefaultHostnameVerifier requires httpcore dependencies to use 
`org.apache.http.util.Args` class.
   
   ```
   11:12:33.486 [pulsar-discovery-io-2-3] WARN  
o.a.p.p.server.DirectProxyHandler- [[id: 0x300e8efa, L:/10.213.246.124:6651 
- R:/10.215.134.167:51374]] [[id: 0x2b69265f, L:/10.213.246.124:48686 - R:perbs
   java.lang.NoClassDefFoundError: org/apache/http/util/Args
   at org.apache.http.conn.ssl.SubjectName.(SubjectName.java:48) 
~[httpclient-4.5.5.jar:4.5.5]
   at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.getSubjectAltNames(DefaultHostnameVerifier.java:310)
 ~[httpclient-4.5.5.jar:4.5.5]
   at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:112)
 ~[httpclient-4.5.5.jar:4.5.5]
   at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:99)
 ~[httpclient-4.5.5.jar:4.5.5]
   at 
   ```
   
   ### Modifications
   add httpcore dependencies into client/proxy where hostname verification 
happens.
   
   ### Result
   
   It fixes `java.lang.NoClassDefFoundError: org/apache/http/util/Args` while 
hostname verification.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

merlimat commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168293225
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   I think it would make sense to have a schema at the namespace level (with 
per-topic override), since we already have a lot of administration tasks at 
that level.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168283925
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   @dave2wave I've been thinking about this more. If this is a common use case 
I think I can make this happen with little upheaval. Let me know if this is a 
deal breaker for you.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168277614
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   That is not impossible, but perhaps inconvenient. What is the use case here? 
Are we talking within the same namespace or across namespaces? It would be 
possible to add another layer of indirection to the schema and store the name 
associated with the topic somewhere (not sure where at the moment).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168275516
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -79,8 +86,9 @@ message MessageMetadata {
repeated EncryptionKeys encryption_keys = 13;
// Algorithm used to encrypt data key
optional string encryption_algo = 14;
+   optional bytes schema_version = 15;
// Additional parameters required by encryption
-   optional bytes encryption_param = 15;
+   optional bytes encryption_param = 16;
 
 Review comment:
   Sorry, I missed that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168275383
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
+   required bytes version = 2;
+   required bytes schema_data = 3;
 
 Review comment:
   The properties list is available to store that info. This makes the most 
sense to me but I'm willing to change it if you are dead set on specifying this 
information statically.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

merlimat commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168274945
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -79,8 +86,9 @@ message MessageMetadata {
repeated EncryptionKeys encryption_keys = 13;
// Algorithm used to encrypt data key
optional string encryption_algo = 14;
+   optional bytes schema_version = 15;
// Additional parameters required by encryption
-   optional bytes encryption_param = 15;
+   optional bytes encryption_param = 16;
 
 Review comment:
   We cannot change the integer id of an existing field


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch branch-1.20-yahoo deleted (was d5175b9)

[jai1]

This is an automated email from the ASF dual-hosted git repository.

jai1 pushed a change to branch branch-1.20-yahoo
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git.


 was d5175b9  Bumped to 1.20.8-incubating-yahoo-SNAPSHOT

This change permanently discards the following revisions:

 discard d5175b9  Bumped to 1.20.8-incubating-yahoo-SNAPSHOT
 discard 870268e  Cancel keep-alive timer task after the proxy switch to TCP 
proxy (#1210)
 discard 7c460ac  Reset kafka dep (#17)
 discard 02ec84b  Bumped to 1.20.7-incubating-yahoo
 discard 8ae92d3  CMS-2984: Enable specification of TLS Protocol Versions and 
Cipher Suites
 discard d8f6068  Use SecurityUtility class (#16)
 discard 136c241  Support hostname verification on proxy to broker connection 
(#1214) (#14)
 discard 1336b85  Add hostname-verification at client tls connection (#1208) 
(#13)
 discard a86e472  Pulsar proxy (#11)
 discard 5c35550  Bumped to 1.20.7-incubating-yahoo-SNAPSHOT
 discard fdd255a  Release to 1.20.6-incubating-yahoo
 discard 1b2d720  Add timeout for bundle unloading (#990)
 discard c24f03a  Bumped to 1.20.6-incubating-yahoo-SNAPSHOT
 discard ab4b30d  Release to 1.20.5-incubating-yahoo
 discard 031d64a  add NoSuchEntryException as not-recoverable error (#10)
 discard fc954a0  Bumped up 1.20.5-incubating-yahoo-SNAPSHOT
 discard 3cbdb76  Release pulsar to 1.20.4-incubating-yahoo
 discard 3773830  Make sure nextTuple emits tuple with non-null values #1000
 discard 61c6d75  Fix: cursor-ledger leak by deleting cursor-ledger after 
recovering managed-cursor
 discard 8a1132a  Bumped to 1.20.4-incubating-yahoo-SNAPSHOT
 discard 403a409  Release pulsar to 1.20.3-incubating-yahoo
 discard 9b0a98d  shade data-sketch (#8)
 discard 58f  Bumped to 1.20.3-incubating-yahoo-SNAPSHOT
 discard 0b15038  Release pulsar to 1.20.2-incubating-yahoo
 discard 5eba7c1  Avoid dead-lock on broker service when broker failed to load 
managed-ledger (#934)
 discard 90be3c7  Fix: failed producer creation leak (#927)
 discard 05e6a20  Bumped to 1.20.2-incubating-yahoo-SNAPSHOT
 discard 67e0b88  Release pulsar to 1.20.1-incubating-yahoo
 discard dda80f8  Log incompatible client version on partition-metadata access
 discard 65722cc  Fail broker start if broker-znode created by other zk-session 
(#913)
 discard 05742e6  PIP-8: Introduce peer cluster for global namespace 
redirection (#903)
 discard 564d1cc  Avoid logging stacktrace for exclusive consumer-busy 
exception (#906)
 discard b979691  Cleanup unsused ApiVersionFilter (#885)
 discard c9c0a4b  Specify rsa transform, aes & tag length (#839)
 discard 054cc49  Use thread local to allocate temp byte[] instead of allocator 
(#882)
 discard 1482cd0  Fix: Non-persistent topic acking sequence for dropped message 
(#874)
 discard 624e2f4  Bumped to 1.20.1-incubating-yahoo-SNAPSHOT
 discard eaf5587  Release pulsar to 1.20.0-incubating-yahoo

-- 
To stop receiving notification emails like this one, please contact
j...@apache.org.

[GitHub] merlimat commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

merlimat commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168274469
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
+   required bytes version = 2;
+   required bytes schema_data = 3;
 
 Review comment:
   I agree that storing the schema data is agnostic to the format, however 
these protobufs are relative to the interactions between client and broker. 
   
   In this case, the producer might use something like 
`Schema.avroOf(MyClass.class)` or `Schema.jsonOf(MyClass.class)`. In this case, 
 don't we need to communicate to the server what kind of schema the client is 
using?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] dave2wave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

dave2wave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168274341
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   I would want to assign a schema to multiple topics - perhaps a large number.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168271879
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
+   required bytes version = 2;
+   required bytes schema_data = 3;
 
 Review comment:
   I tried to address this in a question posed on the original reference PR. I 
had originally specced this out with that intent but after much thought I 
decided not to add it. My intent was that the schema registry should just store 
a "Schema" no matter what it is, it does not need to know what it contains. The 
schema only really means something to the end systems (producer/consumer) and 
by extension the broker. We can allow the broker to "plug-in" modules to allow 
semantic parsing of the schema as needed. This way we can support any number of 
crazy business cases.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

mgodave commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168271025
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
 
 Review comment:
   grrr, I HATE TABS ;-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

ivankelly commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168262139
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   ok, in that case, this makes sense. perhaps it should be a repeated rather 
than an optional though, for the case that there's different versions.
   
   I guess that the schema name must be immutable for a topic then also.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing 
if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365688094
 
 
   rebased to the latest master


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

ivankelly commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168240669
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   Will the whole schema go with all messages? If that's the case, isn't this 
field here redundant?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

merlimat commented on a change in pull request #1089: PIP-12 Introduce builder 
for creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168237399
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ClientBuilder.java
 ##
 @@ -0,0 +1,263 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import 
org.apache.pulsar.client.api.PulsarClientException.UnsupportedAuthenticationException;
+
+/**
+ * Builder interface that is used to construct a {@link PulsarClient} instance.
+ */
+public interface ClientBuilder extends Serializable, Cloneable {
+
+/**
+ * @return the new {@link PulsarClient} instance
+ */
+PulsarClient build() throws PulsarClientException;
+
+/**
+ * Create a copy of the current client builder.
+ * 
+ * Cloning the builder can be used to share an incomplete configuration 
and specialize it multiple times. For
+ * example:
+ *
+ * 
+ * ClientBuilder builder = 
PulsarClient.builder().ioThreads(8).listenerThreads(4);
+ *
+ * PulsarClient client1 = builder.clone().serviceUrl(URL_1).build();
+ * PulsarClient client2 = builder.clone().serviceUrl(URL_2).build();
+ * 
+ */
+ClientBuilder clone();
 
 Review comment:
   `clone()` is a bit weird in the way it's treated. It cannot be marked as 
`@Override`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

merlimat commented on a change in pull request #1089: PIP-12 Introduce builder 
for creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168236993
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ClientBuilder.java
 ##
 @@ -0,0 +1,263 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import 
org.apache.pulsar.client.api.PulsarClientException.UnsupportedAuthenticationException;
+
+/**
+ * Builder interface that is used to construct a {@link PulsarClient} instance.
 
 Review comment:
   Sure, will add `@since`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

merlimat commented on a change in pull request #1089: PIP-12 Introduce builder 
for creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168236725
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ConsumerBuilder.java
 ##
 @@ -0,0 +1,230 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * {@link ConsumerBuilder} is used to configure and create instances of {@link 
Consumer}.
+ *
+ * @see PulsarClient#newConsumer()
+ */
+public interface ConsumerBuilder extends Serializable, Cloneable {
+
+/**
+ * Create a copy of the current consumer builder.
+ * 
+ * Cloning the builder can be used to share an incomplete configuration 
and specialize it multiple times. For
+ * example:
+ *
+ * 
+ * ConsumerBuilder builder = client.newConsumer() //
+ * .subscriptionName("my-subscription-name") //
+ * .subscriptionType(SubscriptionType.Shared) //
+ * .receiverQueueSize(10);
+ *
+ * Consumer consumer1 = builder.clone().topic(TOPIC_1).subscribe();
+ * Consumer consumer2 = builder.clone().topic(TOPIC_2).subscribe();
+ * 
+ */
+ConsumerBuilder clone();
+
+/**
+ * Finalize the consumer creation by subscribing to the topic.
+ *
+ * 
+ * Subscribing to the c
 
 Review comment:
   ups, forgot to finish


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

merlimat commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168236331
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -272,6 +289,8 @@ message CommandProducer {
 
 /// Add optional metadata key=value to this producer
 repeated KeyValue metadata= 6;
+
+   optional int64 schema_version = 7;
 
 Review comment:
   It should carry the whole schema here. The application/producer doesn't 
necessarely know what the version is. 
   
   Eg: if I'm using a type like : 
   
   ```java
   class MyClass {
  public int a;
  public int b;
   };
   ```
   
   When I try to publish, I just know the format I'm about to send, and we can 
extract the avro/json schema out of it. When I try to create producer, I can 
send that information, and the broker can compare with the schemas already 
availalable for this topic. If the schema is know, we just return the schema 
version in the `ProducerSuccess`  response. If it's a new (compatible) version 
of the schema, it will also be stored.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing 
if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365666967
 
 
   > This could also have been done by each consumer periodically adding a 
message to the topic. if you start getting the messages, you are the active 
consumer.
   
   @ivankelly : yes that's the trick we are using now for the other project. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

sijie commented on a change in pull request #1089: PIP-12 Introduce builder for 
creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168229704
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ClientBuilder.java
 ##
 @@ -0,0 +1,263 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import 
org.apache.pulsar.client.api.PulsarClientException.UnsupportedAuthenticationException;
+
+/**
+ * Builder interface that is used to construct a {@link PulsarClient} instance.
 
 Review comment:
   nit: it might be good to add "@since 2.0". so we can track when we introduce 
the build methods.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

sijie commented on a change in pull request #1089: PIP-12 Introduce builder for 
creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168230414
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ConsumerBuilder.java
 ##
 @@ -0,0 +1,230 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * {@link ConsumerBuilder} is used to configure and create instances of {@link 
Consumer}.
+ *
+ * @see PulsarClient#newConsumer()
+ */
+public interface ConsumerBuilder extends Serializable, Cloneable {
+
+/**
+ * Create a copy of the current consumer builder.
+ * 
+ * Cloning the builder can be used to share an incomplete configuration 
and specialize it multiple times. For
+ * example:
+ *
+ * 
+ * ConsumerBuilder builder = client.newConsumer() //
+ * .subscriptionName("my-subscription-name") //
+ * .subscriptionType(SubscriptionType.Shared) //
+ * .receiverQueueSize(10);
+ *
+ * Consumer consumer1 = builder.clone().topic(TOPIC_1).subscribe();
+ * Consumer consumer2 = builder.clone().topic(TOPIC_2).subscribe();
+ * 
+ */
+ConsumerBuilder clone();
+
+/**
+ * Finalize the consumer creation by subscribing to the topic.
+ *
+ * 
+ * Subscribing to the c
 
 Review comment:
   incomplete comment?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

sijie commented on a change in pull request #1089: PIP-12 Introduce builder for 
creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168230587
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ConsumerBuilder.java
 ##
 @@ -0,0 +1,230 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * {@link ConsumerBuilder} is used to configure and create instances of {@link 
Consumer}.
+ *
+ * @see PulsarClient#newConsumer()
+ */
+public interface ConsumerBuilder extends Serializable, Cloneable {
+
+/**
+ * Create a copy of the current consumer builder.
+ * 
+ * Cloning the builder can be used to share an incomplete configuration 
and specialize it multiple times. For
+ * example:
+ *
+ * 
+ * ConsumerBuilder builder = client.newConsumer() //
+ * .subscriptionName("my-subscription-name") //
+ * .subscriptionType(SubscriptionType.Shared) //
+ * .receiverQueueSize(10);
+ *
+ * Consumer consumer1 = builder.clone().topic(TOPIC_1).subscribe();
+ * Consumer consumer2 = builder.clone().topic(TOPIC_2).subscribe();
+ * 
+ */
+ConsumerBuilder clone();
+
+/**
+ * Finalize the consumer creation by subscribing to the topic.
+ *
+ * 
+ * Subscribing to the c
+ *
+ * @return
 
 Review comment:
   `@return a consumer`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

sijie commented on a change in pull request #1089: PIP-12 Introduce builder for 
creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168231825
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ReaderConfiguration.java
 ##
 @@ -84,8 +89,9 @@ public ReaderConfiguration 
setCryptoKeyReader(CryptoKeyReader cryptoKeyReader) {
 
 /**
  * Sets the ConsumerCryptoFailureAction to the value specified
- * 
- * @param The consumer action
+ *
+ * @param The
 
 Review comment:
   the param field seems to be wrong?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

sijie commented on a change in pull request #1089: PIP-12 Introduce builder for 
creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r168229917
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ClientBuilder.java
 ##
 @@ -0,0 +1,263 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.io.Serializable;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import 
org.apache.pulsar.client.api.PulsarClientException.UnsupportedAuthenticationException;
+
+/**
+ * Builder interface that is used to construct a {@link PulsarClient} instance.
+ */
+public interface ClientBuilder extends Serializable, Cloneable {
+
+/**
+ * @return the new {@link PulsarClient} instance
+ */
+PulsarClient build() throws PulsarClientException;
+
+/**
+ * Create a copy of the current client builder.
+ * 
+ * Cloning the builder can be used to share an incomplete configuration 
and specialize it multiple times. For
+ * example:
+ *
+ * 
+ * ClientBuilder builder = 
PulsarClient.builder().ioThreads(8).listenerThreads(4);
+ *
+ * PulsarClient client1 = builder.clone().serviceUrl(URL_1).build();
+ * PulsarClient client2 = builder.clone().serviceUrl(URL_2).build();
+ * 
+ */
+ClientBuilder clone();
 
 Review comment:
   nit: `@Override`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

ivankelly commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168196011
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
 
 Review comment:
   tabs


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

ivankelly commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168230410
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
+   required bytes version = 2;
+   required bytes schema_data = 3;
 
 Review comment:
   Does the schema data itself have a schema? If we're supporting json, arvo, 
pb, etc, shouldn't we have a separate field to give the reader a hint of how 
they need to parse the schema?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

ivankelly commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168229944
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -22,6 +22,20 @@ package pulsar.proto;
 option java_package = "org.apache.pulsar.common.api.proto";
 option optimize_for = LITE_RUNTIME;
 
+message Schema {
+   required string name = 1;
+   required bytes version = 2;
+   required bytes schema_data = 3;
+repeated KeyValue properties = 4;
+}
+
+message Tombstone {}
+
+message SchemaEntry {
 
 Review comment:
   I assume this is part of a the schema registry part? Why is tombstone 
separate from the schema? You have schema_data as required? If tombstoning a 
schema (which I assume is deleting the schema) what should the data be? 
Couldn't schema data itself being empty denote the tombstone?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

ivankelly commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168231228
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -368,6 +387,7 @@ message CommandRedeliverUnacknowledgedMessages {
 
 message CommandSuccess {
required uint64 request_id = 1;
+   optional Schema schema = 2;
 
 Review comment:
   what happens if the message schema changes mid topic?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1232: Schema registry (1/4)

[GitBox]

merlimat commented on a change in pull request #1232: Schema registry (1/4)
URL: https://github.com/apache/incubator-pulsar/pull/1232#discussion_r168231034
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -272,6 +289,8 @@ message CommandProducer {
 
 /// Add optional metadata key=value to this producer
 repeated KeyValue metadata= 6;
+
+   optional int64 schema_version = 7;
 
 Review comment:
   Sure, no problem. I think it would be good to have the "final" version of 
the `PulsarApi.proto` to get a sense of all the changes in the interaction 
between client & broker.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1089: PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

merlimat commented on issue #1089: PIP-12 Introduce builder for creating 
Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#issuecomment-365505550
 
 
   >  Once we have subscription for topics and pattern, we need add them in.
   
   Yes, the idea was to have different ways to specify: 
   ```java
   // Single topic
   client.newConsumer().topic(MY_TOPIC).subscriptioName(SUB).build();
   
   // List of topics
   List myList = ...;
   client.newConsumer().topics(myList).subscriptioName(SUB).build();
   
   // Regex
   client.newConsumer().topicsRegex("test.*").subscriptioName(SUB).build();
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

merlimat commented on issue #1156: Introduce ActiveConsumerListener for 
realizing if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365660980
 
 
   > Does this provide a way to figure out the current active consumer? 
   
   Not right now, I think it would be interesting to mention who's the active 
consumer in the topic in the topics stats. We could have a `Boolean` field in 
the `ConsumerStats` that is set to `null` in non-failover subscriptions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1223: Add a `backend` admin restful endpoint for query backend information

[GitBox]

sijie commented on issue #1223: Add a `backend` admin restful endpoint for 
query backend information
URL: https://github.com/apache/incubator-pulsar/pull/1223#issuecomment-365658827
 
 
   @rdhabalia @yush1ga I renamed "backend" to "internalConfiguration". Can you 
guys review this change again?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

ivankelly commented on a change in pull request #1156: Introduce 
ActiveConsumerListener for realizing if a consumer is active in a failover 
subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#discussion_r168221779
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ConsumerEventListener.java
 ##
 @@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+/**
+ * Listener on the consumer state changes.
+ */
+public interface ConsumerEventListener {
+
+/**
+ * Notified when the consumer group is changed, and the consumer becomes 
the active consumer.
+ */
+void becomeActive(Consumer consumer, int partitionId);
 
 Review comment:
   became?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1228: Force to pull docker build image to regenerate PulsarApi.java generat?

[GitBox]

merlimat closed pull request #1228: Force to pull docker build image to 
regenerate PulsarApi.java generat?
URL: https://github.com/apache/incubator-pulsar/pull/1228
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pulsar-common/generate_protobuf_docker.sh 
b/pulsar-common/generate_protobuf_docker.sh
index c568c9a07..376d72e4c 100755
--- a/pulsar-common/generate_protobuf_docker.sh
+++ b/pulsar-common/generate_protobuf_docker.sh
@@ -32,6 +32,9 @@ IMAGE="$BUILD_IMAGE_NAME:$BUILD_IMAGE_VERSION"
 
 echo $IMAGE
 
+# Force to pull image in case it was updated
+docker pull $IMAGE
+
 docker run -i \
 -v ${COMMON_DIR}:${COMMON_DIR} $IMAGE \
 bash -c "cd ${COMMON_DIR}; /pulsar/protobuf/src/protoc 
--java_out=src/main/java src/main/proto/PulsarApi.proto"


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1186: Pulsar FAQ

[GitBox]

sijie commented on issue #1186: Pulsar FAQ
URL: https://github.com/apache/incubator-pulsar/pull/1186#issuecomment-365653665
 
 
   @lucperkins do you have time on helping with the FAQ content here?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing if a consumer is active in a failover subscription group

[GitBox]

sijie commented on issue #1156: Introduce ActiveConsumerListener for realizing 
if a consumer is active in a failover subscription group
URL: https://github.com/apache/incubator-pulsar/pull/1156#issuecomment-365652446
 
 
   @merlimat as you suggested, I did two changes here:
   
   - rename `ActiveConsumerListener` to `ConsumerEventListener`. currently 
since we only use it for failover subscription, so I am still disallowing 
shared/exclusive subscription to use the listener. we can enable if we support 
more events in future.
   
   - I changed the dispatcher to use `consumer` object as the comparison 
identifier.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1137: Schema registry

[GitBox]

ivankelly commented on issue #1137: Schema registry
URL: https://github.com/apache/incubator-pulsar/pull/1137#issuecomment-365634195
 
 
   I don't see any client changes in this. How is a schema set for a topic?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1103: PIP-13-1/3: Provide `TopicsConsumer` to consume from several topics under same namespace

[GitBox]

zhaijack commented on issue #1103: PIP-13-1/3: Provide `TopicsConsumer` to 
consume from several topics under same namespace
URL: https://github.com/apache/incubator-pulsar/pull/1103#issuecomment-365620069
 
 
   @merlimat  @ivankelly, Thanks for your comments,  Would you please help 
review it again?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1103: PIP-13-1/3: Provide `TopicsConsumer` to consume from several topics under same namespace

[GitBox]

zhaijack commented on issue #1103: PIP-13-1/3: Provide `TopicsConsumer` to 
consume from several topics under same namespace
URL: https://github.com/apache/incubator-pulsar/pull/1103#issuecomment-365613677
 
 
   opened issue #1237 tracking using builder to create topics consumer.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on a change in pull request #1103: PIP-13-1/3: Provide `TopicsConsumer` to consume from several topics under same namespace

[GitBox]

zhaijack commented on a change in pull request #1103: PIP-13-1/3: Provide 
`TopicsConsumer` to consume from several topics under same namespace
URL: https://github.com/apache/incubator-pulsar/pull/1103#discussion_r168178825
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/TopicsConsumerImpl.java
 ##
 @@ -0,0 +1,881 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl;
+
+import static com.google.common.base.Preconditions.checkArgument;
+import static com.google.common.base.Preconditions.checkState;
+
+import com.google.common.collect.Lists;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerConfiguration;
+import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.client.api.SubscriptionType;
+import org.apache.pulsar.client.util.ConsumerName;
+import org.apache.pulsar.client.util.FutureUtil;
+import org.apache.pulsar.common.api.proto.PulsarApi.CommandAck.AckType;
+import org.apache.pulsar.common.naming.DestinationName;
+import org.apache.pulsar.common.naming.NamespaceName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TopicsConsumerImpl extends ConsumerBase {
+
+// All topics should be in same namespace
+protected NamespaceName namespaceName;
+
+// Map , when get do ACK, consumer will by find 
by topic name
+private final ConcurrentHashMap consumers;
 
 Review comment:
   Yes, the changes for TopicMessageIdImpl, TopicMessageImpl, and 
UnAckedMessageTracker is aimed to make PartitionedConsumerImpl and 
TopicsConsumerImpl could be easy merged in the future.  opened issue #1236 
tracking this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack opened a new issue #1236: Make TopicsConsumerImpl extends PartitionedConsumerImpl

[GitBox]

zhaijack opened a new issue #1236: Make TopicsConsumerImpl extends 
PartitionedConsumerImpl
URL: https://github.com/apache/incubator-pulsar/issues/1236
 
 
   IN PIP-13 and its following PRs: 
https://github.com/apache/incubator-pulsar/pull/1103, We have made 
TopicsConsumerImpl behaviour  similar to PartitionedConsumerImpl,  This issue 
is tracking the following work to make TopicsConsumerImpl extends 
PartitionedConsumerImpl.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on a change in pull request #1103: PIP-13-1/3: Provide `TopicsConsumer` to consume from several topics under same namespace

[GitBox]

zhaijack commented on a change in pull request #1103: PIP-13-1/3: Provide 
`TopicsConsumer` to consume from several topics under same namespace
URL: https://github.com/apache/incubator-pulsar/pull/1103#discussion_r167877509
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/TopicsConsumerImpl.java
 ##
 @@ -0,0 +1,881 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.impl;
+
+import static com.google.common.base.Preconditions.checkArgument;
+import static com.google.common.base.Preconditions.checkState;
+
+import com.google.common.collect.Lists;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerConfiguration;
+import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.apache.pulsar.client.api.SubscriptionType;
+import org.apache.pulsar.client.util.ConsumerName;
+import org.apache.pulsar.client.util.FutureUtil;
+import org.apache.pulsar.common.api.proto.PulsarApi.CommandAck.AckType;
+import org.apache.pulsar.common.naming.DestinationName;
+import org.apache.pulsar.common.naming.NamespaceName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class TopicsConsumerImpl extends ConsumerBase {
+
+// All topics should be in same namespace
+protected NamespaceName namespaceName;
+
+// Map , when get do ACK, consumer will by find 
by topic name
+private final ConcurrentHashMap consumers;
 
 Review comment:
   Thanks. Yes. There is a plan to make this extends PartitionedConsumerImpl, 
that is also the reason that this change keep use a lot of same interface. 
   But it maybe good to make PartitionedConsumerImpl and TopicsConsumerImpl 
separate at first, this could avoid bring bugs into PartitionedConsumerImpl.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch asf-site updated: Updated site at revision 7404952

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 2716be2  Updated site at revision 7404952
2716be2 is described below

commit 2716be27a9a9d3fd959055f19bda732a9e68879f
Author: jenkins 
AuthorDate: Wed Feb 14 08:16:04 2018 +

Updated site at revision 7404952
---
 content/api/client/index-all.html  |   8 ++
 .../org/apache/pulsar/client/api/Reader.html   |  47 +-
 .../docs/latest/project/BinaryProtocol/index.html  | 157 -
 content/ja/project/BinaryProtocol/index.html   | 157 -
 4 files changed, 358 insertions(+), 11 deletions(-)

diff --git a/content/api/client/index-all.html 
b/content/api/client/index-all.html
index 7a9aa8a..296466e 100644
--- a/content/api/client/index-all.html
+++ b/content/api/client/index-all.html
@@ -557,6 +557,14 @@
 
 Check whether the message has a key
 
+hasMessageAvailable()
 - Method in interface org.apache.pulsar.client.api.Reader
+
+Check if there is any message available to read from the 
current position.
+
+hasMessageAvailableAsync()
 - Method in interface org.apache.pulsar.client.api.Reader
+
+Asynchronously Check if there is message that has been 
published successfully to the broker in the topic.
+
 hasProperty(String)
 - Method in interface org.apache.pulsar.client.api.Message
 
 Check whether the message has a specific property 
attached.
diff --git a/content/api/client/org/apache/pulsar/client/api/Reader.html 
b/content/api/client/org/apache/pulsar/client/api/Reader.html
index 052fce9..c8ae116 100644
--- a/content/api/client/org/apache/pulsar/client/api/Reader.html
+++ b/content/api/client/org/apache/pulsar/client/api/Reader.html
@@ -17,7 +17,7 @@
 catch(err) {
 }
 //-->
-var methods = {"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6};
+var methods = {"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6};
 var tabs = {65535:["t0","All Methods"],2:["t2","Instance 
Methods"],4:["t3","Abstract Methods"]};
 var altColor = "altColor";
 var rowColor = "rowColor";
@@ -136,24 +136,36 @@ extends Closeable
 
 
 boolean
+hasMessageAvailable()
+Check if there is any message available to read from the 
current position.
+
+
+
+CompletableFutureBoolean
+hasMessageAvailableAsync()
+Asynchronously Check if there is message that has been 
published successfully to the broker in the topic.
+
+
+
+boolean
 hasReachedEndOfTopic()
 Return true if the topic was terminated and this reader has 
reached the end of the topic
 
 
-
+
 Message
 readNext()
 Read the next message in the topic
 
 
-
+
 Message
 readNext(inttimeout,
 TimeUnitunit)
 Read the next message in the topic.
 
 
-
+
 CompletableFutureMessage
 readNextAsync()
 
@@ -253,13 +265,38 @@ extends Closeable
 
 
 
-
+
 
 hasReachedEndOfTopic
 booleanhasReachedEndOfTopic()
 Return true if the topic was terminated and this reader has 
reached the end of the topic
 
 
+
+
+
+
+
+hasMessageAvailable
+booleanhasMessageAvailable()
+ throws PulsarClientException
+Check if there is any message available to read from the 
current position.
+
+Throws:
+PulsarClientException
+
+
+
+
+
+
+
+
+hasMessageAvailableAsync
+CompletableFutureBooleanhasMessageAvailableAsync()
+Asynchronously Check if there is message that has been 
published successfully to the broker in the topic.
+
+
 
 
 
diff --git a/content/docs/latest/project/BinaryProtocol/index.html 
b/content/docs/latest/project/BinaryProtocol/index.html
index 29b81ec..d67dda9 100644
--- a/content/docs/latest/project/BinaryProtocol/index.html
+++ b/content/docs/latest/project/BinaryProtocol/index.html
@@ -1985,6 +1985,18 @@ response will contain actual metadata.
   
   
   
+
+  CommandGetLastMessageId
+
+  
+  
+  
+
+  CommandGetLastMessageIdResponse
+
+  
+  
+  
 
   CommandLookupTopic
 
@@ -2505,6 +2517,26 @@ response will contain actual metadata.
   
   
 
+
+
+  getLastMessageId
+  
+  CommandGetLastMessageId
+  
+  optional
+  
+  
+
+
+
+  getLastMessageIdResponse
+  
+  CommandGetLastMessageIdResponse
+  
+  optional
+  
+  
+
  
   
 
@@ -2794,7 +2826,7 @@ will be the auth of the proxy itself
   
   Original auth role and auth Method that was passed
 to the proxy. In this case the auth info above
-will be the auth of the proxy itself   
+will be