[GitHub] Licht-T commented on issue #1029: Fix MessageRouter hash inconsistent on C++/Java client

[GitBox]

Licht-T commented on issue #1029: Fix MessageRouter hash inconsistent on 
C++/Java client
URL: https://github.com/apache/incubator-pulsar/pull/1029#issuecomment-362509088
 
 
   @saandrews Oops! Excuse me! I'll fix soon!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] saandrews commented on issue #1029: Fix MessageRouter hash inconsistent on C++/Java client

[GitBox]

saandrews commented on issue #1029: Fix MessageRouter hash inconsistent on 
C++/Java client
URL: https://github.com/apache/incubator-pulsar/pull/1029#issuecomment-362508435
 
 
   @Licht-T Some of the changes like "constexpr" seems to be c++11 specific and 
it's breaking our build. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yush1ga commented on a change in pull request #1087: Add basic authentication plugin

[GitBox]

yush1ga commented on a change in pull request #1087: Add basic authentication 
plugin
URL: https://github.com/apache/incubator-pulsar/pull/1087#discussion_r165557612
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/web/AuthenticationFilter.java
 ##
 @@ -60,8 +63,11 @@ public void doFilter(ServletRequest request, 
ServletResponse response, FilterCha
 if (LOG.isDebugEnabled()) {
 LOG.debug("[{}] Authenticated HTTP request with role {}", 
request.getRemoteAddr(), role);
 }
-} catch (AuthenticationException e) {
+} catch (PulsarHttpAuthenticationException e) {
 HttpServletResponse httpResponse = (HttpServletResponse) response;
+if (isNotBlank(e.getRealmInformation())) {
+((HttpServletResponse) response).setHeader("WWW-Authenticate", 
e.getRealmInformation());
 
 Review comment:
   @maskit 
   I roughly agree with you.
   
   > using an exception to indicate simple authentication failure is a bad 
design
   
   I think this should not be fixed in this PR adding authentication plugin.
   We should consider good design and make another PR.
   That's why, I exclude the code about authentication header from this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yush1ga commented on issue #1087: Add basic authentication plugin

[GitBox]

yush1ga commented on issue #1087: Add basic authentication plugin
URL: https://github.com/apache/incubator-pulsar/pull/1087#issuecomment-362483546
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] yush1ga commented on issue #1087: Add basic authentication plugin

[GitBox]

yush1ga commented on issue #1087: Add basic authentication plugin
URL: https://github.com/apache/incubator-pulsar/pull/1087#issuecomment-362483546
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1129: Added end to end encryption in C++ client

[GitBox]

merlimat closed pull request #1129: Added end to end encryption in C++ client
URL: https://github.com/apache/incubator-pulsar/pull/1129
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h 
b/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h
index c73f9422a..274abd52e 100644
--- a/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h
+++ b/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h
@@ -24,6 +24,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 
 #pragma GCC visibility push(default)
 namespace pulsar {
@@ -139,6 +141,14 @@ class ConsumerConfiguration {
  * @return the configured timeout in milliseconds caching 
BrokerConsumerStats.
  */
 long getBrokerConsumerStatsCacheTimeInMs() const;
+
+bool isEncryptionEnabled() const;
+const CryptoKeyReaderPtr getCryptoKeyReader() const;
+ConsumerConfiguration& setCryptoKeyReader(CryptoKeyReaderPtr 
cryptoKeyReader);
+
+ConsumerCryptoFailureAction getCryptoFailureAction() const;
+ConsumerConfiguration& setCryptoFailureAction(ConsumerCryptoFailureAction 
action);
+
 friend class PulsarWrapper;
 
private:
diff --git a/pulsar-client-cpp/include/pulsar/ConsumerCryptoFailureAction.h 
b/pulsar-client-cpp/include/pulsar/ConsumerCryptoFailureAction.h
new file mode 100644
index 0..d9d845c76
--- /dev/null
+++ b/pulsar-client-cpp/include/pulsar/ConsumerCryptoFailureAction.h
@@ -0,0 +1,40 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#ifndef CONSUMERCRYPTOFAILUREACTION_H_
+#define CONSUMERCRYPTOFAILUREACTION_H_
+
+#pragma GCC visibility push(default)
+
+namespace pulsar {
+
+enum class ConsumerCryptoFailureAction
+{
+FAIL, // This is the default option to fail consume until crypto 
succeeds
+DISCARD,  // Message is silently acknowledged and not delivered to the 
application
+CONSUME   // Deliver the encrypted message to the application. It's the 
application's
+  // responsibility to decrypt the message. If message is also 
compressed,
+  // decompression will fail. If message contain batch messages, 
client will
+  // not be able to retrieve individual messages in the batch
+};
+
+} /* namespace pulsar */
+
+#pragma GCC visibility pop
+
+#endif /* CONSUMERCRYPTOFAILUREACTION_H_ */
diff --git a/pulsar-client-cpp/include/pulsar/CryptoKeyReader.h 
b/pulsar-client-cpp/include/pulsar/CryptoKeyReader.h
new file mode 100644
index 0..cb4cf1516
--- /dev/null
+++ b/pulsar-client-cpp/include/pulsar/CryptoKeyReader.h
@@ -0,0 +1,70 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#ifndef CRYPTOKEYREADER_H_
+#define CRYPTOKEYREADER_H_
+
+#include 
+#include 
+
+#pragma GCC visibility push(default)
+
+namespace pulsar {
+
+class CryptoKeyReader {
+   public:
+CryptoKeyReader() {}
+
+/*
+ * Return the encryption key corresponding to the key name in the argument
+ * 
+ * This method should be implemented to return the EncryptionKeyInfo. This 
method will be
+ * called at the time of producer creation as well as consumer receiving 
messages.
+ * Hence, application should not make any blocking calls within

[incubator-pulsar] branch master updated: Added end to end encryption in C++ client (#1129)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 9a081ad  Added end to end encryption in C++ client (#1129)
9a081ad is described below

commit 9a081ad9d69aa3f250d5d821ab1d8169b728eeda
Author: Andrews 
AuthorDate: Thu Feb 1 17:17:10 2018 -0800

Added end to end encryption in C++ client (#1129)
---
 .../include/pulsar/ConsumerConfiguration.h |  10 +
 .../include/pulsar/ConsumerCryptoFailureAction.h   |  40 ++
 pulsar-client-cpp/include/pulsar/CryptoKeyReader.h |  70 
 .../include/pulsar/EncryptionKeyInfo.h |  68 +++
 .../include/pulsar/ProducerConfiguration.h |  15 +
 .../include/pulsar/ProducerCryptoFailureAction.h   |  36 ++
 pulsar-client-cpp/include/pulsar/Result.h  |   4 +-
 pulsar-client-cpp/lib/BatchMessageContainer.cc |   4 +
 pulsar-client-cpp/lib/ConsumerConfiguration.cc |  19 +
 pulsar-client-cpp/lib/ConsumerConfigurationImpl.h  |   6 +-
 pulsar-client-cpp/lib/ConsumerImpl.cc  |  55 ++-
 pulsar-client-cpp/lib/ConsumerImpl.h   |   7 +
 pulsar-client-cpp/lib/EncryptionKeyInfo.cc |  38 ++
 pulsar-client-cpp/lib/EncryptionKeyInfoImpl.cc |  39 ++
 pulsar-client-cpp/lib/EncryptionKeyInfoImpl.h  |  55 +++
 pulsar-client-cpp/lib/MessageCrypto.cc | 458 +
 pulsar-client-cpp/lib/MessageCrypto.h  | 142 +++
 pulsar-client-cpp/lib/ProducerConfiguration.cc |  28 ++
 pulsar-client-cpp/lib/ProducerConfigurationImpl.h  |   9 +-
 pulsar-client-cpp/lib/ProducerImpl.cc  |  59 ++-
 pulsar-client-cpp/lib/ProducerImpl.h   |  10 +
 pulsar-client-cpp/lib/Result.cc|   3 +
 pulsar-client-cpp/perf/PerfConsumer.cc |  44 +-
 pulsar-client-cpp/perf/PerfProducer.cc |  49 ++-
 pulsar-client-cpp/tests/BasicEndToEndTest.cc   | 206 +
 25 files changed, 1461 insertions(+), 13 deletions(-)

diff --git a/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h 
b/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h
index c73f942..274abd5 100644
--- a/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h
+++ b/pulsar-client-cpp/include/pulsar/ConsumerConfiguration.h
@@ -24,6 +24,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 
 #pragma GCC visibility push(default)
 namespace pulsar {
@@ -139,6 +141,14 @@ class ConsumerConfiguration {
  * @return the configured timeout in milliseconds caching 
BrokerConsumerStats.
  */
 long getBrokerConsumerStatsCacheTimeInMs() const;
+
+bool isEncryptionEnabled() const;
+const CryptoKeyReaderPtr getCryptoKeyReader() const;
+ConsumerConfiguration& setCryptoKeyReader(CryptoKeyReaderPtr 
cryptoKeyReader);
+
+ConsumerCryptoFailureAction getCryptoFailureAction() const;
+ConsumerConfiguration& setCryptoFailureAction(ConsumerCryptoFailureAction 
action);
+
 friend class PulsarWrapper;
 
private:
diff --git a/pulsar-client-cpp/include/pulsar/ConsumerCryptoFailureAction.h 
b/pulsar-client-cpp/include/pulsar/ConsumerCryptoFailureAction.h
new file mode 100644
index 000..d9d845c
--- /dev/null
+++ b/pulsar-client-cpp/include/pulsar/ConsumerCryptoFailureAction.h
@@ -0,0 +1,40 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#ifndef CONSUMERCRYPTOFAILUREACTION_H_
+#define CONSUMERCRYPTOFAILUREACTION_H_
+
+#pragma GCC visibility push(default)
+
+namespace pulsar {
+
+enum class ConsumerCryptoFailureAction
+{
+FAIL, // This is the default option to fail consume until crypto 
succeeds
+DISCARD,  // Message is silently acknowledged and not delivered to the 
application
+CONSUME   // Deliver the encrypted message to the application. It's the 
application's
+  // responsibility to decrypt the message. If message is also 
compressed,
+  // decompression will fail. If message contain batch messages, 
client will
+  // not be able to retrieve individual messages in the batch
+};
+
+} /* namespace pulsar */
+
+#pragma G

[GitHub] zhaijack commented on a change in pull request #1066: Issue 937: add CommandGetLastMessageId to make reader know the end of topic

[GitBox]

zhaijack commented on a change in pull request #1066: Issue 937: add 
CommandGetLastMessageId to make reader know the end of topic
URL: https://github.com/apache/incubator-pulsar/pull/1066#discussion_r165534036
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -415,6 +415,16 @@ message CommandConsumerStatsResponse {
 optional uint64 msgBacklog  = 15;
 }
 
+message CommandGetLastMessageId {
 
 Review comment:
   bumped proto version to v11


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1166: Introduce retries for flaky tests

[GitBox]

merlimat commented on issue #1166: Introduce retries for flaky tests
URL: https://github.com/apache/incubator-pulsar/pull/1166#issuecomment-362443741
 
 
   retest this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1166: Introduce retries for flaky tests

[GitBox]

merlimat commented on issue #1166: Introduce retries for flaky tests
URL: https://github.com/apache/incubator-pulsar/pull/1166#issuecomment-362443741
 
 
   retest this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Project-wide Checkstyle Config (#1162)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 034b2b2  Project-wide Checkstyle Config (#1162)
034b2b2 is described below

commit 034b2b2c47e7b1b42c52edeb27f8c629b4fcbb30
Author: Dave Rusek 
AuthorDate: Thu Feb 1 16:42:10 2018 -0700

Project-wide Checkstyle Config (#1162)

* Add projet wide checkstyle configuration

* Optimize imports for managed-ledger

* Clean checkstyle for managed-ledger

* Revert "Clean checkstyle for managed-ledger"

This reverts commit 455db2c40e055c4fd98fb2219af6b66df2217de6.

* Revert "Optimize imports for managed-ledger"

This reverts commit c691bc66a08176d671b340c1fee5a7bc58aae35a.

* Revert wrongly merged changes

* Rename builttools project to Pulsar Build Tools

* Add missing license headers
---
 buildtools/pom.xml |  36 ++
 .../src/main/resources/pulsar/checkstyle.xml   | 445 +
 .../src/main/resources/pulsar/suppressions.xml |  51 +++
 pom.xml|  23 ++
 4 files changed, 555 insertions(+)

diff --git a/buildtools/pom.xml b/buildtools/pom.xml
new file mode 100644
index 000..f4e4172
--- /dev/null
+++ b/buildtools/pom.xml
@@ -0,0 +1,36 @@
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
+  4.0.0
+
+  
+org.apache.pulsar
+pulsar
+1.22.0-incubating-SNAPSHOT
+..
+  
+
+  buildtools
+  jar
+  Pulsar Build Tools
+
+
diff --git a/buildtools/src/main/resources/pulsar/checkstyle.xml 
b/buildtools/src/main/resources/pulsar/checkstyle.xml
new file mode 100644
index 000..890a862
--- /dev/null
+++ b/buildtools/src/main/resources/pulsar/checkstyle.xml
@@ -0,0 +1,445 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+   

[GitHub] merlimat closed pull request #1162: Project-wide Checkstyle Config

[GitBox]

merlimat closed pull request #1162: Project-wide Checkstyle Config
URL: https://github.com/apache/incubator-pulsar/pull/1162
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/buildtools/pom.xml b/buildtools/pom.xml
new file mode 100644
index 0..f4e417246
--- /dev/null
+++ b/buildtools/pom.xml
@@ -0,0 +1,36 @@
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
+  4.0.0
+
+  
+org.apache.pulsar
+pulsar
+1.22.0-incubating-SNAPSHOT
+..
+  
+
+  buildtools
+  jar
+  Pulsar Build Tools
+
+
diff --git a/buildtools/src/main/resources/pulsar/checkstyle.xml 
b/buildtools/src/main/resources/pulsar/checkstyle.xml
new file mode 100644
index 0..890a8628a
--- /dev/null
+++ b/buildtools/src/main/resources/pulsar/checkstyle.xml
@@ -0,0 +1,445 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/buildtools/src/main/resources/pulsar/suppressions.xml 
b/buildtools/src/main/resources/pulsar/suppressions.xml
new file mode 100644
index 0..21a4fed70
--- /dev/null
+++ b/buildtools/src/main/resources/pulsar/suppressions.xml
@@ -0,0 +1,51 @@
+
+
+
+http://www.puppycrawl.com/dtds/suppressions_1_1.dtd";>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/pom.xml b/pom.xml
index d58103447..9eb152446 100644
--- a/pom.xml
+++ b/pom.xml
@@ -73,6 +73,7 @@ flexible messaging model and an intuitive client 
API.
   
 
   
+buildtools
 managed-ledger
 pulsar-common
 pulsar-broker-common
@@ -119,6 +120,7 @@ flexi

[incubator-pulsar] branch master updated: Added missing C++ enum values (#1125)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new b8fa8c1  Added missing C++ enum values (#1125)
b8fa8c1 is described below

commit b8fa8c1696172d619eb51f38bf66fa51c2cc4c6e
Author: Matteo Merli 
AuthorDate: Thu Feb 1 14:12:42 2018 -0800

Added missing C++ enum values (#1125)

* Added missing C++ enum values

* Fixed formatting
---
 pulsar-client-cpp/include/pulsar/Result.h | 4 +++-
 pulsar-client-cpp/lib/ClientConnection.cc | 6 ++
 pulsar-client-cpp/lib/Commands.cc | 6 ++
 pulsar-client-cpp/lib/Result.cc   | 3 +++
 4 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/pulsar-client-cpp/include/pulsar/Result.h 
b/pulsar-client-cpp/include/pulsar/Result.h
index de72a71..8085d48 100644
--- a/pulsar-client-cpp/include/pulsar/Result.h
+++ b/pulsar-client-cpp/include/pulsar/Result.h
@@ -71,7 +71,9 @@ enum Result
 ResultTopicNotFound,  /// Topic not found
 ResultSubscriptionNotFound,   /// Subscription not found
 ResultConsumerNotFound,   /// Consumer not found
-ResultUnsupportedVersionError  /// Error when an older client/version 
doesn't support a required feature
+ResultUnsupportedVersionError,  /// Error when an older client/version 
doesn't support a required feature
+
+ResultTopicTerminated  /// Topic was already terminated
 };
 
 // Return string representation of result code
diff --git a/pulsar-client-cpp/lib/ClientConnection.cc 
b/pulsar-client-cpp/lib/ClientConnection.cc
index 8bdd7a2..08d5572 100644
--- a/pulsar-client-cpp/lib/ClientConnection.cc
+++ b/pulsar-client-cpp/lib/ClientConnection.cc
@@ -97,6 +97,12 @@ static Result getResult(ServerError serverError) {
 
 case UnsupportedVersionError:
 return ResultUnsupportedVersionError;
+
+case TooManyRequests:
+return ResultTooManyLookupRequestException;
+
+case TopicTerminatedError:
+return ResultTopicTerminated;
 }
 // NOTE : Do not add default case in the switch above. In future if we get 
new cases for
 // ServerError and miss them in the switch above we would like to get 
notified. Adding
diff --git a/pulsar-client-cpp/lib/Commands.cc 
b/pulsar-client-cpp/lib/Commands.cc
index 0c3fdbd..70d61df 100644
--- a/pulsar-client-cpp/lib/Commands.cc
+++ b/pulsar-client-cpp/lib/Commands.cc
@@ -375,6 +375,12 @@ std::string Commands::messageType(BaseCommand_Type type) {
 case BaseCommand::CONSUMER_STATS_RESPONSE:
 return "CONSUMER_STATS_RESPONSE";
 break;
+case BaseCommand::REACHED_END_OF_TOPIC:
+return "REACHED_END_OF_TOPIC";
+break;
+case BaseCommand::SEEK:
+return "SEEK";
+break;
 };
 }
 
diff --git a/pulsar-client-cpp/lib/Result.cc b/pulsar-client-cpp/lib/Result.cc
index c988f01..1deb5f2 100644
--- a/pulsar-client-cpp/lib/Result.cc
+++ b/pulsar-client-cpp/lib/Result.cc
@@ -119,6 +119,9 @@ const char* pulsar::strResult(Result result) {
 
 case ResultUnsupportedVersionError:
 return "UnsupportedVersionError";
+
+case ResultTopicTerminated:
+return "TopicTerminated";
 };
 // NOTE : Do not add default case in the switch above. In future if we get 
new cases for
 // ServerError and miss them in the switch above we would like to get 
notified. Adding

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] merlimat closed pull request #1125: Added missing C++ enum values

[GitBox]

merlimat closed pull request #1125: Added missing C++ enum values
URL: https://github.com/apache/incubator-pulsar/pull/1125
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pulsar-client-cpp/include/pulsar/Result.h 
b/pulsar-client-cpp/include/pulsar/Result.h
index de72a71e7..8085d4837 100644
--- a/pulsar-client-cpp/include/pulsar/Result.h
+++ b/pulsar-client-cpp/include/pulsar/Result.h
@@ -71,7 +71,9 @@ enum Result
 ResultTopicNotFound,  /// Topic not found
 ResultSubscriptionNotFound,   /// Subscription not found
 ResultConsumerNotFound,   /// Consumer not found
-ResultUnsupportedVersionError  /// Error when an older client/version 
doesn't support a required feature
+ResultUnsupportedVersionError,  /// Error when an older client/version 
doesn't support a required feature
+
+ResultTopicTerminated  /// Topic was already terminated
 };
 
 // Return string representation of result code
diff --git a/pulsar-client-cpp/lib/ClientConnection.cc 
b/pulsar-client-cpp/lib/ClientConnection.cc
index 7e954cd64..50584c598 100644
--- a/pulsar-client-cpp/lib/ClientConnection.cc
+++ b/pulsar-client-cpp/lib/ClientConnection.cc
@@ -97,6 +97,12 @@ static Result getResult(ServerError serverError) {
 
 case UnsupportedVersionError:
 return ResultUnsupportedVersionError;
+
+case TooManyRequests:
+return ResultTooManyLookupRequestException;
+
+case TopicTerminatedError:
+return ResultTopicTerminated;
 }
 // NOTE : Do not add default case in the switch above. In future if we get 
new cases for
 // ServerError and miss them in the switch above we would like to get 
notified. Adding
diff --git a/pulsar-client-cpp/lib/Commands.cc 
b/pulsar-client-cpp/lib/Commands.cc
index 59cf2a24c..cc210c897 100644
--- a/pulsar-client-cpp/lib/Commands.cc
+++ b/pulsar-client-cpp/lib/Commands.cc
@@ -367,6 +367,12 @@ std::string Commands::messageType(BaseCommand_Type type) {
 case BaseCommand::CONSUMER_STATS_RESPONSE:
 return "CONSUMER_STATS_RESPONSE";
 break;
+case BaseCommand::REACHED_END_OF_TOPIC:
+return "REACHED_END_OF_TOPIC";
+break;
+case BaseCommand::SEEK:
+return "SEEK";
+break;
 };
 }
 
diff --git a/pulsar-client-cpp/lib/Result.cc b/pulsar-client-cpp/lib/Result.cc
index c988f0169..1deb5f2e5 100644
--- a/pulsar-client-cpp/lib/Result.cc
+++ b/pulsar-client-cpp/lib/Result.cc
@@ -119,6 +119,9 @@ const char* pulsar::strResult(Result result) {
 
 case ResultUnsupportedVersionError:
 return "UnsupportedVersionError";
+
+case ResultTopicTerminated:
+return "TopicTerminated";
 };
 // NOTE : Do not add default case in the switch above. In future if we get 
new cases for
 // ServerError and miss them in the switch above we would like to get 
notified. Adding


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on a change in pull request #1168: Proxy roles enforcement

[GitBox]

merlimat commented on a change in pull request #1168: Proxy roles enforcement
URL: https://github.com/apache/incubator-pulsar/pull/1168#discussion_r165504070
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java
 ##
 @@ -196,6 +202,16 @@ protected void handleLookup(CommandLookupTopic lookup) {
 if (lookupSemaphore.tryAcquire()) {
 final String originalPrincipal = lookup.hasOriginalPrincipal() ? 
lookup.getOriginalPrincipal()
 : this.originalPrincipal;
+if (service.isAuthorizationEnabled() && 
proxyRoles.contains(authRole)) {
 
 Review comment:
   We should have the 3 similar blocks of code in a single method


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jai1 commented on issue #1168: Proxy roles enforcement

[GitBox]

jai1 commented on issue #1168: Proxy roles enforcement
URL: https://github.com/apache/incubator-pulsar/pull/1168#issuecomment-362415807
 
 
   Forgot to mention: This enhancement is configurable - if you don't populate 
the proxyRoles then the behavior falls back to default.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jai1 opened a new pull request #1168: Proxy roles enforcement

[GitBox]

jai1 opened a new pull request #1168: Proxy roles enforcement
URL: https://github.com/apache/incubator-pulsar/pull/1168
 
 
   The broker should be able to distinguish between a proxy and a client so 
that no compromised proxy can impersonate a client. 
   
   To be specific - once a proxy host is compromised it can choose to send 
originalPrincipal as null which will make the broker treat it as a normal 
client and authenticate/authorize the proxy as a normal client rather than 
demanding to authorize the originalPrincipal.
   
   So for example:- 
   A client using topic T1 with roleToken R1 going through proxy with roleToken 
R3 will require granting AuthAction produce/consume on both R1 and R3 (proxy).
   Similarly, a client using topic T2 with roleToken R2 going through the same 
proxy will require granting AuthAction produce/consume on both R1 and R3 
(proxy).
   
   Now with my current code if proxy is compromised it can access topics T1 and 
T2 without requiring client roleToken (R1 and R2) at all by passing 
originalPrincipal as null. 
   
   If I add a broker side setting "proxyRoles=R3, R4", then as soon as I see 
that the roleToken (R3) extracted from the incoming cert is in proxyRoles (R3, 
R4) then I can return an error if originalPrincipal is null.
   
   Thanks to @msb-at-yahoo for suggesting this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Remove toString() invocation in dispatcher path (#1164)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 5257db0  Remove toString() invocation in dispatcher path (#1164)
5257db0 is described below

commit 5257db083ad2a8a712424d07cb503cea64db533f
Author: Matteo Merli 
AuthorDate: Thu Feb 1 12:46:06 2018 -0800

Remove toString() invocation in dispatcher path (#1164)
---
 .../java/org/apache/pulsar/broker/service/Consumer.java  |  4 ++--
 .../NonPersistentDispatcherMultipleConsumers.java| 16 ++--
 .../NonPersistentDispatcherSingleActiveConsumer.java | 15 ++-
 .../service/nonpersistent/NonPersistentSubscription.java |  8 
 4 files changed, 22 insertions(+), 21 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
index 64bd424..decdc31 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
@@ -226,7 +226,7 @@ public class Consumer {
 }
 }
 
-public static int getBatchSizeforEntry(ByteBuf metadataAndPayload, String 
subscription, long consumerId) {
+public static int getBatchSizeforEntry(ByteBuf metadataAndPayload, 
Subscription subscription, long consumerId) {
 try {
 // save the reader index and restore after parsing
 metadataAndPayload.markReaderIndex();
@@ -253,7 +253,7 @@ public class Consumer {
 while (iter.hasNext()) {
 Entry entry = iter.next();
 ByteBuf metadataAndPayload = entry.getDataBuffer();
-int batchSize = getBatchSizeforEntry(metadataAndPayload, 
subscription.toString(), consumerId);
+int batchSize = getBatchSizeforEntry(metadataAndPayload, 
subscription, consumerId);
 if (batchSize == -1) {
 // this would suggest that the message might have been 
corrupted
 iter.remove();
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
index 2b2902b..5fc7676 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
@@ -18,6 +18,8 @@
  */
 package org.apache.pulsar.broker.service.nonpersistent;
 
+import static org.apache.pulsar.broker.service.Consumer.getBatchSizeforEntry;
+
 import java.util.List;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;
@@ -27,7 +29,7 @@ import org.apache.bookkeeper.mledger.util.Rate;
 import org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers;
 import org.apache.pulsar.broker.service.BrokerServiceException;
 import org.apache.pulsar.broker.service.Consumer;
-import static org.apache.pulsar.broker.service.Consumer.getBatchSizeforEntry;
+import org.apache.pulsar.broker.service.Subscription;
 import org.apache.pulsar.common.api.proto.PulsarApi.CommandSubscribe.SubType;
 import org.apache.pulsar.utils.CopyOnWriteArrayList;
 import org.slf4j.Logger;
@@ -38,18 +40,20 @@ import org.slf4j.LoggerFactory;
 public class NonPersistentDispatcherMultipleConsumers extends 
AbstractDispatcherMultipleConsumers
 implements NonPersistentDispatcher {
 
-private final NonPersistentTopic topic;
 private CompletableFuture closeFuture = null;
 private final String name;
 private final Rate msgDrop;
 protected static final 
AtomicIntegerFieldUpdater 
TOTAL_AVAILABLE_PERMITS_UPDATER = AtomicIntegerFieldUpdater
 .newUpdater(NonPersistentDispatcherMultipleConsumers.class, 
"totalAvailablePermits");
+@SuppressWarnings("unused")
 private volatile int totalAvailablePermits = 0;
 
-public NonPersistentDispatcherMultipleConsumers(NonPersistentTopic topic, 
String dispatcherName) {
-this.name = topic.getName() + " / " + dispatcherName;
-this.topic = topic;
+private final Subscription subscription;
+
+public NonPersistentDispatcherMultipleConsumers(NonPersistentTopic topic, 
Subscription subscription) {
+this.name = topic.getName() + " / " + subscription.getName();
 this.msgDrop = new Rate();
+this.subscription = subscription;
 }
 
 @Override
@@ -148,7 +152,7 @@ public class NonPersistentDispatcherMultipleConsumers 
extends AbstractDispatcher
 TOTAL_AVAILABLE_PERMITS_UPDATER.addAndGet(this, 
-consumer.sendMessages(entries).getT

[GitHub] merlimat closed pull request #1164: Remove toString() invocation in dispatcher path

[GitBox]

merlimat closed pull request #1164: Remove toString() invocation in dispatcher 
path
URL: https://github.com/apache/incubator-pulsar/pull/1164
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
index 64bd4243c..decdc3190 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
@@ -226,7 +226,7 @@ private void incrementUnackedMessages(int ackedMessages) {
 }
 }
 
-public static int getBatchSizeforEntry(ByteBuf metadataAndPayload, String 
subscription, long consumerId) {
+public static int getBatchSizeforEntry(ByteBuf metadataAndPayload, 
Subscription subscription, long consumerId) {
 try {
 // save the reader index and restore after parsing
 metadataAndPayload.markReaderIndex();
@@ -253,7 +253,7 @@ void updatePermitsAndPendingAcks(final List entries, 
SendMessageInfo sent
 while (iter.hasNext()) {
 Entry entry = iter.next();
 ByteBuf metadataAndPayload = entry.getDataBuffer();
-int batchSize = getBatchSizeforEntry(metadataAndPayload, 
subscription.toString(), consumerId);
+int batchSize = getBatchSizeforEntry(metadataAndPayload, 
subscription, consumerId);
 if (batchSize == -1) {
 // this would suggest that the message might have been 
corrupted
 iter.remove();
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
index 2b2902b69..5fc7676cf 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java
@@ -18,6 +18,8 @@
  */
 package org.apache.pulsar.broker.service.nonpersistent;
 
+import static org.apache.pulsar.broker.service.Consumer.getBatchSizeforEntry;
+
 import java.util.List;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;
@@ -27,7 +29,7 @@
 import org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers;
 import org.apache.pulsar.broker.service.BrokerServiceException;
 import org.apache.pulsar.broker.service.Consumer;
-import static org.apache.pulsar.broker.service.Consumer.getBatchSizeforEntry;
+import org.apache.pulsar.broker.service.Subscription;
 import org.apache.pulsar.common.api.proto.PulsarApi.CommandSubscribe.SubType;
 import org.apache.pulsar.utils.CopyOnWriteArrayList;
 import org.slf4j.Logger;
@@ -38,18 +40,20 @@
 public class NonPersistentDispatcherMultipleConsumers extends 
AbstractDispatcherMultipleConsumers
 implements NonPersistentDispatcher {
 
-private final NonPersistentTopic topic;
 private CompletableFuture closeFuture = null;
 private final String name;
 private final Rate msgDrop;
 protected static final 
AtomicIntegerFieldUpdater 
TOTAL_AVAILABLE_PERMITS_UPDATER = AtomicIntegerFieldUpdater
 .newUpdater(NonPersistentDispatcherMultipleConsumers.class, 
"totalAvailablePermits");
+@SuppressWarnings("unused")
 private volatile int totalAvailablePermits = 0;
 
-public NonPersistentDispatcherMultipleConsumers(NonPersistentTopic topic, 
String dispatcherName) {
-this.name = topic.getName() + " / " + dispatcherName;
-this.topic = topic;
+private final Subscription subscription;
+
+public NonPersistentDispatcherMultipleConsumers(NonPersistentTopic topic, 
Subscription subscription) {
+this.name = topic.getName() + " / " + subscription.getName();
 this.msgDrop = new Rate();
+this.subscription = subscription;
 }
 
 @Override
@@ -148,7 +152,7 @@ public void sendMessages(List entries) {
 TOTAL_AVAILABLE_PERMITS_UPDATER.addAndGet(this, 
-consumer.sendMessages(entries).getTotalSentMessages());
 } else {
 entries.forEach(entry -> {
-int totalMsgs = getBatchSizeforEntry(entry.getDataBuffer(), 
name, -1);
+int totalMsgs = getBatchSizeforEntry(entry.getDataBuffer(), 
subscription, -1);
 if (totalMsgs > 0) {
 msgDrop.recordEvent();
 }
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherSingleActive

[GitHub] mgodave commented on issue #1162: Project-wide Checkstyle Config

[GitBox]

mgodave commented on issue #1162: Project-wide Checkstyle Config
URL: https://github.com/apache/incubator-pulsar/pull/1162#issuecomment-362385776
 
 
   Enabled module by module as they are fixed.
   
   On Feb 1, 2018 11:58, "Matteo Merli"  wrote:
   
   > Assigned #1162  to
   > @mgodave .
   >
   > ?
   > You are receiving this because you were assigned.
   > Reply to this email directly, view it on GitHub
   > ,
   > or mute the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1124: Support Pulsar proxy from C++/Python client library

[GitBox]

merlimat closed pull request #1124: Support Pulsar proxy from C++/Python client 
library
URL: https://github.com/apache/incubator-pulsar/pull/1124
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pulsar-client-cpp/lib/BinaryProtoLookupService.cc 
b/pulsar-client-cpp/lib/BinaryProtoLookupService.cc
index 630344992..e5b8d42fd 100644
--- a/pulsar-client-cpp/lib/BinaryProtoLookupService.cc
+++ b/pulsar-client-cpp/lib/BinaryProtoLookupService.cc
@@ -53,7 +53,7 @@ Future 
BinaryProtoLookupService::lookupAsync(
 }
 std::string lookupName = dn->toString();
 LookupDataResultPromisePtr promise = 
boost::make_shared();
-Future future = 
cnxPool_.getConnectionAsync(serviceUrl_);
+Future future = 
cnxPool_.getConnectionAsync(serviceUrl_, serviceUrl_);
 
future.addListener(boost::bind(&BinaryProtoLookupService::sendTopicLookupRequest,
 this, lookupName, false,
_1, _2, promise));
 return promise->getFuture();
@@ -71,7 +71,7 @@ Future 
BinaryProtoLookupService::getPartitionMetada
 return promise->getFuture();
 }
 std::string lookupName = dn->toString();
-Future future = 
cnxPool_.getConnectionAsync(serviceUrl_);
+Future future = 
cnxPool_.getConnectionAsync(serviceUrl_, serviceUrl_);
 
future.addListener(boost::bind(&BinaryProtoLookupService::sendPartitionMetadataLookupRequest,
 this,
lookupName, _1, _2, promise));
 return promise->getFuture();
@@ -100,8 +100,12 @@ void BinaryProtoLookupService::handleLookup(const 
std::string& destinationName,
 if (data->isRedirect()) {
 LOG_DEBUG("Lookup request is for " << destinationName << " 
redirected to "
<< data->getBrokerUrl());
+
+const std::string& logicalAddress = data->getBrokerUrl();
+const std::string& physicalAddress =
+data->shouldProxyThroughServiceUrl() ? serviceUrl_ : 
logicalAddress;
 Future future =
-cnxPool_.getConnectionAsync(data->getBrokerUrl());
+cnxPool_.getConnectionAsync(logicalAddress, physicalAddress);
 
future.addListener(boost::bind(&BinaryProtoLookupService::sendTopicLookupRequest,
 this,
destinationName, 
data->isAuthoritative(), _1, _2, promise));
 } else {
diff --git a/pulsar-client-cpp/lib/ClientConnection.cc 
b/pulsar-client-cpp/lib/ClientConnection.cc
index 7e954cd64..8bdd7a243 100644
--- a/pulsar-client-cpp/lib/ClientConnection.cc
+++ b/pulsar-client-cpp/lib/ClientConnection.cc
@@ -104,7 +104,8 @@ static Result getResult(ServerError serverError) {
 return ResultUnknownError;
 }
 
-ClientConnection::ClientConnection(const std::string& endpoint, 
ExecutorServicePtr executor,
+ClientConnection::ClientConnection(const std::string& logicalAddress, const 
std::string& physicalAddress,
+   ExecutorServicePtr executor,
const ClientConfiguration& 
clientConfiguration,
const AuthenticationPtr& authentication)
 : state_(Pending),
@@ -114,8 +115,9 @@ ClientConnection::ClientConnection(const std::string& 
endpoint, ExecutorServiceP
   executor_(executor),
   resolver_(executor->createTcpResolver()),
   socket_(executor->createSocket()),
-  address_(endpoint),
-  cnxString_("[ -> " + endpoint + "] "),
+  logicalAddress_(logicalAddress),
+  physicalAddress_(physicalAddress),
+  cnxString_("[ -> " + physicalAddress + "] "),
   error_(boost::system::error_code()),
   incomingBuffer_(SharedBuffer::allocate(DefaultBufferSize)),
   incomingCmd_(),
@@ -267,7 +269,11 @@ void ClientConnection::handleTcpConnected(const 
boost::system::error_code& err,
 cnxStringStream << "[" << socket_->local_endpoint() << " -> " << 
socket_->remote_endpoint() << "] ";
 cnxString_ = cnxStringStream.str();
 
-LOG_INFO(cnxString_ << "Connected to broker");
+if (logicalAddress_ == physicalAddress_) {
+LOG_INFO(cnxString_ << "Connected to broker");
+} else {
+LOG_INFO(cnxString_ << "Connected to broker through proxy. Logical 
broker: " << logicalAddress_);
+}
 state_ = TcpConnected;
 socket_->set_option(tcp::no_delay(true));
 
@@ -288,7 +294,7 @@ void ClientConnection::handleTcpConnected(const 
boost::system::error_code& err,
 if (!isTlsAllowInsecureConnection_) {
 boost::system::error_code err;
 Url service_url;
-if (!Url::parse(address_, service_url)) {
+if (!Url::parse(physicalAddress_, service_u

[incubator-pulsar] branch master updated: Support Pulsar proxy from C++/Python client library (#1124)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 58d5727  Support Pulsar proxy from C++/Python client library (#1124)
58d5727 is described below

commit 58d572797ebd780fecb1c4f4478fad1d590f14c3
Author: Matteo Merli 
AuthorDate: Thu Feb 1 11:03:56 2018 -0800

Support Pulsar proxy from C++/Python client library (#1124)
---
 pulsar-client-cpp/lib/BinaryProtoLookupService.cc | 10 ++---
 pulsar-client-cpp/lib/ClientConnection.cc | 25 +++
 pulsar-client-cpp/lib/ClientConnection.h  | 15 +-
 pulsar-client-cpp/lib/ClientImpl.cc   |  6 +-
 pulsar-client-cpp/lib/Commands.cc | 10 -
 pulsar-client-cpp/lib/Commands.h  |  3 ++-
 pulsar-client-cpp/lib/ConnectionPool.cc   | 19 +
 pulsar-client-cpp/lib/ConnectionPool.h| 19 -
 pulsar-client-cpp/lib/HTTPLookupService.cc|  1 +
 pulsar-client-cpp/lib/LookupDataResult.h  | 15 +++---
 pulsar-client-cpp/lib/LookupService.h |  4 
 pulsar-client-cpp/lib/Url.cc  |  6 ++
 pulsar-client-cpp/lib/Url.h   |  2 ++
 pulsar-client-cpp/lib/lz4/lz4.h   |  5 +
 pulsar-client-cpp/python/test_producer.py |  4 ++--
 15 files changed, 110 insertions(+), 34 deletions(-)

diff --git a/pulsar-client-cpp/lib/BinaryProtoLookupService.cc 
b/pulsar-client-cpp/lib/BinaryProtoLookupService.cc
index 6303449..e5b8d42 100644
--- a/pulsar-client-cpp/lib/BinaryProtoLookupService.cc
+++ b/pulsar-client-cpp/lib/BinaryProtoLookupService.cc
@@ -53,7 +53,7 @@ Future 
BinaryProtoLookupService::lookupAsync(
 }
 std::string lookupName = dn->toString();
 LookupDataResultPromisePtr promise = 
boost::make_shared();
-Future future = 
cnxPool_.getConnectionAsync(serviceUrl_);
+Future future = 
cnxPool_.getConnectionAsync(serviceUrl_, serviceUrl_);
 
future.addListener(boost::bind(&BinaryProtoLookupService::sendTopicLookupRequest,
 this, lookupName, false,
_1, _2, promise));
 return promise->getFuture();
@@ -71,7 +71,7 @@ Future 
BinaryProtoLookupService::getPartitionMetada
 return promise->getFuture();
 }
 std::string lookupName = dn->toString();
-Future future = 
cnxPool_.getConnectionAsync(serviceUrl_);
+Future future = 
cnxPool_.getConnectionAsync(serviceUrl_, serviceUrl_);
 
future.addListener(boost::bind(&BinaryProtoLookupService::sendPartitionMetadataLookupRequest,
 this,
lookupName, _1, _2, promise));
 return promise->getFuture();
@@ -100,8 +100,12 @@ void BinaryProtoLookupService::handleLookup(const 
std::string& destinationName,
 if (data->isRedirect()) {
 LOG_DEBUG("Lookup request is for " << destinationName << " 
redirected to "
<< data->getBrokerUrl());
+
+const std::string& logicalAddress = data->getBrokerUrl();
+const std::string& physicalAddress =
+data->shouldProxyThroughServiceUrl() ? serviceUrl_ : 
logicalAddress;
 Future future =
-cnxPool_.getConnectionAsync(data->getBrokerUrl());
+cnxPool_.getConnectionAsync(logicalAddress, physicalAddress);
 
future.addListener(boost::bind(&BinaryProtoLookupService::sendTopicLookupRequest,
 this,
destinationName, 
data->isAuthoritative(), _1, _2, promise));
 } else {
diff --git a/pulsar-client-cpp/lib/ClientConnection.cc 
b/pulsar-client-cpp/lib/ClientConnection.cc
index 7e954cd..8bdd7a2 100644
--- a/pulsar-client-cpp/lib/ClientConnection.cc
+++ b/pulsar-client-cpp/lib/ClientConnection.cc
@@ -104,7 +104,8 @@ static Result getResult(ServerError serverError) {
 return ResultUnknownError;
 }
 
-ClientConnection::ClientConnection(const std::string& endpoint, 
ExecutorServicePtr executor,
+ClientConnection::ClientConnection(const std::string& logicalAddress, const 
std::string& physicalAddress,
+   ExecutorServicePtr executor,
const ClientConfiguration& 
clientConfiguration,
const AuthenticationPtr& authentication)
 : state_(Pending),
@@ -114,8 +115,9 @@ ClientConnection::ClientConnection(const std::string& 
endpoint, ExecutorServiceP
   executor_(executor),
   resolver_(executor->createTcpResolver()),
   socket_(executor->createSocket()),
-  address_(endpoint),
-  cnxString_("[ -> " + endpoint + "] "),
+  logicalAddress_(logicalAddress),
+  physicalAddress_(physicalAddress),
+  cnxString_("[ -> " + physicalAddress

[GitHub] merlimat commented on issue #1124: Support Pulsar proxy from C++/Python client library

[GitBox]

merlimat commented on issue #1124: Support Pulsar proxy from C++/Python client 
library
URL: https://github.com/apache/incubator-pulsar/pull/1124#issuecomment-362368322
 
 
   Created #1167
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new issue #1167: Add test for C++ client with proxy

[GitBox]

merlimat opened a new issue #1167: Add test for C++ client with proxy
URL: https://github.com/apache/incubator-pulsar/issues/1167
 
 
   We need to have test for C++ client library with proxy.
   This will mean to also start up a proxy instance during unit tests, same as 
we're starting standalone broker now.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat commented on issue #1162: Project-wide Checkstyle Config

[GitBox]

merlimat commented on issue #1162: Project-wide Checkstyle Config
URL: https://github.com/apache/incubator-pulsar/pull/1162#issuecomment-362366529
 
 
   @mgodave Change LGTM. Will this start enforcing the style on new PRs or will 
need to be enabled module by module after fixing them?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat opened a new pull request #1166: Introduce retries for flaky tests

[GitBox]

merlimat opened a new pull request #1166: Introduce retries for flaky tests
URL: https://github.com/apache/incubator-pulsar/pull/1166
 
 
   ### Motivation
   
   Due to many test intermittently failing on Jenkins build, introduce a 1 time 
retry for failing test before the build is marked as failed.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1089: WIP - PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

mgodave commented on a change in pull request #1089: WIP - PIP-12 Introduce 
builder for creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r165431266
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java
 ##
 @@ -0,0 +1,229 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+import 
org.apache.pulsar.client.api.PulsarClientException.ProducerQueueIsFullError;
+
+public interface ProducerBuilder {
+
+Producer create() throws PulsarClientException;
 
 Review comment:
   nit: since this is a builder, do we want to go with the canonical 'build' 
terminology instead?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1089: WIP - PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

mgodave commented on a change in pull request #1089: WIP - PIP-12 Introduce 
builder for creating Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#discussion_r165431606
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java
 ##
 @@ -0,0 +1,229 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.client.api;
+
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+import 
org.apache.pulsar.client.api.PulsarClientException.ProducerQueueIsFullError;
+
+public interface ProducerBuilder {
+
+Producer create() throws PulsarClientException;
+
+CompletableFuture createAsync();
 
 Review comment:
   I feel like having async and sync versions cloud things. The nature of the 
library is that it is async and if all we are doing is calling 
`CompletableFuture::get` and catching exceptions for the user, is it really 
worth it?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Remove deprecation warnings related to access of PoolArenaMetric (#1161)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new fb72f72  Remove deprecation warnings related to access of 
PoolArenaMetric (#1161)
fb72f72 is described below

commit fb72f72e5a3eb53728b35f464fc9f0ae18cf8f65
Author: Dave Rusek 
AuthorDate: Thu Feb 1 10:17:02 2018 -0700

Remove deprecation warnings related to access of PoolArenaMetric (#1161)
---
 .../broker/stats/AllocatorStatsGenerator.java  | 33 ++
 .../stats/metrics/ManagedLedgerCacheMetrics.java   |  2 +-
 2 files changed, 22 insertions(+), 13 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
index 4640927..998145b 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
@@ -47,16 +47,19 @@ public class AllocatorStatsGenerator {
 }
 
 AllocatorStats stats = new AllocatorStats();
-stats.directArenas = allocator.directArenas().stream().map(x -> 
newPoolArenaStats(x))
-.collect(Collectors.toList());
-stats.heapArenas = allocator.heapArenas().stream().map(x -> 
newPoolArenaStats(x)).collect(Collectors.toList());
+stats.directArenas = allocator.metric().directArenas().stream()
+.map(AllocatorStatsGenerator::newPoolArenaStats)
+.collect(Collectors.toList());
+stats.heapArenas = allocator.metric().heapArenas().stream()
+.map(AllocatorStatsGenerator::newPoolArenaStats)
+.collect(Collectors.toList());
 
-stats.numDirectArenas = allocator.numDirectArenas();
-stats.numHeapArenas = allocator.numHeapArenas();
-stats.numThreadLocalCaches = allocator.numThreadLocalCaches();
-stats.normalCacheSize = allocator.normalCacheSize();
-stats.smallCacheSize = allocator.smallCacheSize();
-stats.tinyCacheSize = allocator.tinyCacheSize();
+stats.numDirectArenas = allocator.metric().numDirectArenas();
+stats.numHeapArenas = allocator.metric().numHeapArenas();
+stats.numThreadLocalCaches = allocator.metric().numThreadLocalCaches();
+stats.normalCacheSize = allocator.metric().normalCacheSize();
+stats.smallCacheSize = allocator.metric().smallCacheSize();
+stats.tinyCacheSize = allocator.metric().tinyCacheSize();
 return stats;
 }
 
@@ -66,9 +69,15 @@ public class AllocatorStatsGenerator {
 stats.numSmallSubpages = m.numSmallSubpages();
 stats.numChunkLists = m.numChunkLists();
 
-stats.tinySubpages = m.tinySubpages().stream().map(x -> 
newPoolSubpageStats(x)).collect(Collectors.toList());
-stats.smallSubpages = m.smallSubpages().stream().map(x -> 
newPoolSubpageStats(x)).collect(Collectors.toList());
-stats.chunkLists = m.chunkLists().stream().map(x -> 
newPoolChunkListStats(x)).collect(Collectors.toList());
+stats.tinySubpages = m.tinySubpages().stream()
+.map(AllocatorStatsGenerator::newPoolSubpageStats)
+.collect(Collectors.toList());
+stats.smallSubpages = m.smallSubpages().stream()
+.map(AllocatorStatsGenerator::newPoolSubpageStats)
+.collect(Collectors.toList());
+stats.chunkLists = m.chunkLists().stream()
+.map(AllocatorStatsGenerator::newPoolChunkListStats)
+.collect(Collectors.toList());
 
 stats.numAllocations = m.numAllocations();
 stats.numTinyAllocations = m.numTinyAllocations();
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
index 95b828c..e16dfd1 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
@@ -66,7 +66,7 @@ public class ManagedLedgerCacheMetrics extends 
AbstractMetrics {
 long totalAllocated = 0;
 long totalUsed = 0;
 
-for (PoolArenaMetric arena : allocator.directArenas()) {
+for (PoolArenaMetric arena : allocator.metric().directArenas()) {
 activeAllocations += arena.numActiveAllocations();
 activeAllocationsTiny += arena.numActiveTinyAllocations();
 activeAllocationsSmall += arena.numActiveSmallAllocations();

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] merlimat closed pull request #1161: Remove deprecation warnings related to access of PoolArenaMetric

[GitBox]

merlimat closed pull request #1161: Remove deprecation warnings related to 
access of PoolArenaMetric
URL: https://github.com/apache/incubator-pulsar/pull/1161
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
index 4640927a4..998145bee 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/AllocatorStatsGenerator.java
@@ -47,16 +47,19 @@ public static AllocatorStats generate(String allocatorName) 
{
 }
 
 AllocatorStats stats = new AllocatorStats();
-stats.directArenas = allocator.directArenas().stream().map(x -> 
newPoolArenaStats(x))
-.collect(Collectors.toList());
-stats.heapArenas = allocator.heapArenas().stream().map(x -> 
newPoolArenaStats(x)).collect(Collectors.toList());
+stats.directArenas = allocator.metric().directArenas().stream()
+.map(AllocatorStatsGenerator::newPoolArenaStats)
+.collect(Collectors.toList());
+stats.heapArenas = allocator.metric().heapArenas().stream()
+.map(AllocatorStatsGenerator::newPoolArenaStats)
+.collect(Collectors.toList());
 
-stats.numDirectArenas = allocator.numDirectArenas();
-stats.numHeapArenas = allocator.numHeapArenas();
-stats.numThreadLocalCaches = allocator.numThreadLocalCaches();
-stats.normalCacheSize = allocator.normalCacheSize();
-stats.smallCacheSize = allocator.smallCacheSize();
-stats.tinyCacheSize = allocator.tinyCacheSize();
+stats.numDirectArenas = allocator.metric().numDirectArenas();
+stats.numHeapArenas = allocator.metric().numHeapArenas();
+stats.numThreadLocalCaches = allocator.metric().numThreadLocalCaches();
+stats.normalCacheSize = allocator.metric().normalCacheSize();
+stats.smallCacheSize = allocator.metric().smallCacheSize();
+stats.tinyCacheSize = allocator.metric().tinyCacheSize();
 return stats;
 }
 
@@ -66,9 +69,15 @@ private static PoolArenaStats 
newPoolArenaStats(PoolArenaMetric m) {
 stats.numSmallSubpages = m.numSmallSubpages();
 stats.numChunkLists = m.numChunkLists();
 
-stats.tinySubpages = m.tinySubpages().stream().map(x -> 
newPoolSubpageStats(x)).collect(Collectors.toList());
-stats.smallSubpages = m.smallSubpages().stream().map(x -> 
newPoolSubpageStats(x)).collect(Collectors.toList());
-stats.chunkLists = m.chunkLists().stream().map(x -> 
newPoolChunkListStats(x)).collect(Collectors.toList());
+stats.tinySubpages = m.tinySubpages().stream()
+.map(AllocatorStatsGenerator::newPoolSubpageStats)
+.collect(Collectors.toList());
+stats.smallSubpages = m.smallSubpages().stream()
+.map(AllocatorStatsGenerator::newPoolSubpageStats)
+.collect(Collectors.toList());
+stats.chunkLists = m.chunkLists().stream()
+.map(AllocatorStatsGenerator::newPoolChunkListStats)
+.collect(Collectors.toList());
 
 stats.numAllocations = m.numAllocations();
 stats.numTinyAllocations = m.numTinyAllocations();
diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
index 95b828cec..e16dfd1dc 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/ManagedLedgerCacheMetrics.java
@@ -66,7 +66,7 @@ public ManagedLedgerCacheMetrics(PulsarService pulsar) {
 long totalAllocated = 0;
 long totalUsed = 0;
 
-for (PoolArenaMetric arena : allocator.directArenas()) {
+for (PoolArenaMetric arena : allocator.metric().directArenas()) {
 activeAllocations += arena.numActiveAllocations();
 activeAllocationsTiny += arena.numActiveTinyAllocations();
 activeAllocationsSmall += arena.numActiveSmallAllocations();


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on issue #1161: Remove deprecation warnings related to access of PoolArenaMetric

[GitBox]

mgodave commented on issue #1161: Remove deprecation warnings related to access 
of PoolArenaMetric
URL: https://github.com/apache/incubator-pulsar/pull/1161#issuecomment-362336971
 
 
   Retested


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on issue #1162: Project-wide Checkstyle Config

[GitBox]

mgodave commented on issue #1162: Project-wide Checkstyle Config
URL: https://github.com/apache/incubator-pulsar/pull/1162#issuecomment-362336106
 
 
   @sijie fixed the comment on naming and restricted the changes to just adding 
the checkstyle config and infrastructure. Once this is merged I will submit the 
PR for the managed-ledger changes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] merlimat closed pull request #1154: Fixed use of static synchronized in LoadManagerShared

[GitBox]

merlimat closed pull request #1154: Fixed use of static synchronized in 
LoadManagerShared
URL: https://github.com/apache/incubator-pulsar/pull/1154
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
index e0af8ec0d..2658a95fa 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
@@ -28,19 +28,17 @@
 import java.util.HashSet;
 import java.util.List;
 import java.util.Map;
-import java.util.Optional;
 import java.util.Map.Entry;
 import java.util.Set;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ExecutionException;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Collectors;
-import org.apache.pulsar.broker.BrokerData;
-import java.util.concurrent.TimeoutException;
+
 import org.apache.commons.lang3.StringUtils;
+import org.apache.pulsar.broker.BrokerData;
 import org.apache.pulsar.broker.PulsarService;
-import org.apache.pulsar.broker.admin.AdminResource;
+import org.apache.pulsar.broker.ServiceConfiguration;
 import org.apache.pulsar.broker.loadbalance.BrokerHostUsage;
 import org.apache.pulsar.broker.loadbalance.LoadData;
 import org.apache.pulsar.broker.stats.metrics.JvmMetrics;
@@ -57,6 +55,9 @@
 import com.beust.jcommander.internal.Lists;
 import com.google.common.collect.Maps;
 
+import io.netty.util.concurrent.FastThreadLocal;
+import io.netty.util.internal.PlatformDependent;
+
 /**
  * This class contains code which in shared between the two load manager 
implementations.
  */
@@ -67,11 +68,21 @@
 public static final int MIBI = 1024 * 1024;
 
 // Cache for primary brokers according to policies.
-private static final Set primariesCache = new HashSet<>();
+private static final FastThreadLocal> localPrimariesCache = 
new FastThreadLocal>() {
+@Override
+protected Set initialValue() throws Exception {
+return new HashSet<>();
+}
+};
 
 // Cache for shard brokers according to policies.
-private static final Set secondaryCache = new HashSet<>();
-
+private static final FastThreadLocal> localSecondaryCache = 
new FastThreadLocal>() {
+@Override
+protected Set initialValue() throws Exception {
+return new HashSet<>();
+}
+};
+
 // update LoadReport at most every 5 seconds
 public static final long LOAD_REPORT_UPDATE_MIMIMUM_INTERVAL = 
TimeUnit.SECONDS.toMillis(5);
 
@@ -83,12 +94,15 @@ private LoadManagerShared() {
 
 // Determines the brokers available for the given service unit according 
to the given policies.
 // The brokers are put into brokerCandidateCache.
-public static synchronized void applyNamespacePolicies(final ServiceUnitId 
serviceUnit,
+public static void applyNamespacePolicies(final ServiceUnitId serviceUnit,
 final SimpleResourceAllocationPolicies policies, final Set 
brokerCandidateCache,
-final Set availableBrokers,
-final BrokerTopicLoadingPredicate brokerTopicLoadingPredicate) {
+final Set availableBrokers, final 
BrokerTopicLoadingPredicate brokerTopicLoadingPredicate) {
+Set primariesCache = localPrimariesCache.get();
 primariesCache.clear();
+
+Set secondaryCache = localSecondaryCache.get();
 secondaryCache.clear();
+
 NamespaceName namespace = serviceUnit.getNamespaceObject();
 boolean isIsolationPoliciesPresent = 
policies.areIsolationPoliciesPresent(namespace);
 boolean isNonPersistentTopic = (serviceUnit instanceof NamespaceBundle)
@@ -141,7 +155,7 @@ public static synchronized void 
applyNamespacePolicies(final ServiceUnitId servi
 }
 } else if (!isNonPersistentTopic
 && 
!brokerTopicLoadingPredicate.isEnablePersistentTopics(brokerUrlString)) {
-// persistent topic can be assigned to only brokers that 
enabled for persistent-topic 
+// persistent topic can be assigned to only brokers that 
enabled for persistent-topic
 if (log.isDebugEnabled()) {
 log.debug("Filter broker- [{}] because broker only 
supports non-persistent namespace - [{}]",
 brokerUrl.getHost(), namespace.toString());
@@ -217,7 +231,7 @@ public static SystemResourceUsage 
getSystemResourceUsage(final BrokerHostUsage b
 
 // Coll

[incubator-pulsar] branch master updated: Fixed use of static synchronized in LoadManagerShared (#1154)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 69d2b66  Fixed use of static synchronized in LoadManagerShared (#1154)
69d2b66 is described below

commit 69d2b66c3c125216ba7bae64300b4224fec0c633
Author: Matteo Merli 
AuthorDate: Thu Feb 1 09:00:29 2018 -0800

Fixed use of static synchronized in LoadManagerShared (#1154)
---
 .../broker/loadbalance/impl/LoadManagerShared.java | 62 +-
 .../broker/loadbalance/impl/OverloadShedder.java   |  8 ++-
 .../pulsar/broker/service/ReplicatorTest.java  | 12 ++---
 3 files changed, 50 insertions(+), 32 deletions(-)

diff --git 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
index e0af8ec..2658a95 100644
--- 
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
+++ 
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerShared.java
@@ -28,19 +28,17 @@ import java.net.URL;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Map;
-import java.util.Optional;
 import java.util.Map.Entry;
 import java.util.Set;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ExecutionException;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Collectors;
-import org.apache.pulsar.broker.BrokerData;
-import java.util.concurrent.TimeoutException;
+
 import org.apache.commons.lang3.StringUtils;
+import org.apache.pulsar.broker.BrokerData;
 import org.apache.pulsar.broker.PulsarService;
-import org.apache.pulsar.broker.admin.AdminResource;
+import org.apache.pulsar.broker.ServiceConfiguration;
 import org.apache.pulsar.broker.loadbalance.BrokerHostUsage;
 import org.apache.pulsar.broker.loadbalance.LoadData;
 import org.apache.pulsar.broker.stats.metrics.JvmMetrics;
@@ -57,6 +55,9 @@ import org.slf4j.LoggerFactory;
 import com.beust.jcommander.internal.Lists;
 import com.google.common.collect.Maps;
 
+import io.netty.util.concurrent.FastThreadLocal;
+import io.netty.util.internal.PlatformDependent;
+
 /**
  * This class contains code which in shared between the two load manager 
implementations.
  */
@@ -67,11 +68,21 @@ public class LoadManagerShared {
 public static final int MIBI = 1024 * 1024;
 
 // Cache for primary brokers according to policies.
-private static final Set primariesCache = new HashSet<>();
+private static final FastThreadLocal> localPrimariesCache = 
new FastThreadLocal>() {
+@Override
+protected Set initialValue() throws Exception {
+return new HashSet<>();
+}
+};
 
 // Cache for shard brokers according to policies.
-private static final Set secondaryCache = new HashSet<>();
-
+private static final FastThreadLocal> localSecondaryCache = 
new FastThreadLocal>() {
+@Override
+protected Set initialValue() throws Exception {
+return new HashSet<>();
+}
+};
+
 // update LoadReport at most every 5 seconds
 public static final long LOAD_REPORT_UPDATE_MIMIMUM_INTERVAL = 
TimeUnit.SECONDS.toMillis(5);
 
@@ -83,12 +94,15 @@ public class LoadManagerShared {
 
 // Determines the brokers available for the given service unit according 
to the given policies.
 // The brokers are put into brokerCandidateCache.
-public static synchronized void applyNamespacePolicies(final ServiceUnitId 
serviceUnit,
+public static void applyNamespacePolicies(final ServiceUnitId serviceUnit,
 final SimpleResourceAllocationPolicies policies, final Set 
brokerCandidateCache,
-final Set availableBrokers,
-final BrokerTopicLoadingPredicate brokerTopicLoadingPredicate) {
+final Set availableBrokers, final 
BrokerTopicLoadingPredicate brokerTopicLoadingPredicate) {
+Set primariesCache = localPrimariesCache.get();
 primariesCache.clear();
+
+Set secondaryCache = localSecondaryCache.get();
 secondaryCache.clear();
+
 NamespaceName namespace = serviceUnit.getNamespaceObject();
 boolean isIsolationPoliciesPresent = 
policies.areIsolationPoliciesPresent(namespace);
 boolean isNonPersistentTopic = (serviceUnit instanceof NamespaceBundle)
@@ -141,7 +155,7 @@ public class LoadManagerShared {
 }
 } else if (!isNonPersistentTopic
 && 
!brokerTopicLoadingPredicate.isEnablePersistentTopics(brokerUrlString)) {
-// persistent topic can be assigned to only brokers that 
enabled for persistent-topic 
+// persistent topic can be assigned to only

[GitHub] merlimat closed pull request #1163: Debump netty version by 1 due to ObjectCleaner bug

[GitBox]

merlimat closed pull request #1163: Debump netty version by 1 due to 
ObjectCleaner bug
URL: https://github.com/apache/incubator-pulsar/pull/1163
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/pom.xml b/pom.xml
index ec3effbee..d58103447 100644
--- a/pom.xml
+++ b/pom.xml
@@ -109,7 +109,7 @@ flexible messaging model and an intuitive client 
API.
 
 4.3.1.85-yahoo
 3.4.10
-4.1.20.Final
+4.1.19.Final
 1.0.5
 9.3.11.v20160721
 1.7.17


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Debump netty version by 1 due to ObjectCleaner bug (#1163)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new f2073d1  Debump netty version by 1 due to ObjectCleaner bug (#1163)
f2073d1 is described below

commit f2073d1f4922553aa0d526c12f53cfa3712ed487
Author: Ivan Kelly 
AuthorDate: Thu Feb 1 18:00:13 2018 +0100

Debump netty version by 1 due to ObjectCleaner bug (#1163)

The ObjectCleaner can possibly keep a process alive forever with netty
4.1.20 since it is not a daemon thread.

See: https://github.com/netty/netty/issues/7617

This causes initialize-cluster-metadata to hang forever. 4.1.19
doesn't have this issue, so we should drop down to that until the fix
is in a release.
---
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pom.xml b/pom.xml
index ec3effb..d581034 100644
--- a/pom.xml
+++ b/pom.xml
@@ -109,7 +109,7 @@ flexible messaging model and an intuitive client 
API.
 
 4.3.1.85-yahoo
 3.4.10
-4.1.20.Final
+4.1.19.Final
 1.0.5
 9.3.11.v20160721
 1.7.17

-- 
To stop receiving notification emails like this one, please contact
mme...@apache.org.

[GitHub] mgodave commented on issue #1162: Project-wide Checkstyle Config

[GitBox]

mgodave commented on issue #1162: Project-wide Checkstyle Config
URL: https://github.com/apache/incubator-pulsar/pull/1162#issuecomment-362322256
 
 
   OOF, I messed this up, I had meant to just add the config and the 
managed-ledger changes in a separate PR. Let me fix this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mgodave commented on a change in pull request #1162: Project-wide Checkstyle Config

[GitBox]

mgodave commented on a change in pull request #1162: Project-wide Checkstyle 
Config
URL: https://github.com/apache/incubator-pulsar/pull/1162#discussion_r165411438
 
 

 ##
 File path: 
managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/EntryCacheTest.java
 ##
 @@ -20,19 +20,14 @@
 
 import static org.mockito.Matchers.any;
 import static org.mockito.Matchers.anyLong;
-import static org.mockito.Mockito.doAnswer;
-import static org.mockito.Mockito.doReturn;
-import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.never;
-import static org.mockito.Mockito.verify;
-import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.*;
 
 Review comment:
   This change was actually a mistake. The Checkstyle config explicitly 
disallows star imports. I'll remove it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #1136: Read compacted consumer flag

[GitBox]

sijie commented on a change in pull request #1136: Read compacted consumer flag
URL: https://github.com/apache/incubator-pulsar/pull/1136#discussion_r165340654
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ConsumerConfiguration.java
 ##
 @@ -270,6 +272,23 @@ public void setPriorityLevel(int priorityLevel) {
 this.priorityLevel = priorityLevel;
 }
 
+public boolean getReadCompacted() {
+return readCompacted;
+}
+
+/**
+ * If enabled, the consumer will read messages from the compacted topic 
rather than reading the full message
+ * backlog of the topic. This means that, if the topic has been compacted, 
the consumer will only see the latest
+ * value for each key in the topic, up until the point in the topic 
message backlog that has been compacted.
+ * Beyond that point, the messages will be sent as normal.
+ *
+ * @param readCompacted whether to read from the compacted topic
+ */
+public ConsumerConfiguration setReadCompacted(boolean readCompacted) {
 
 Review comment:
   yes, I know that from reviewing your code. However that question I am asking 
is from user perspective, when user sees this setting in ConsumerConfiguration, 
they need to know what behavior that should expect. javadoc should be updated 
then. Also it would be good to fail the subscribe request if shared 
subscription is specified and readCompacted is configured.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch master updated: Property name in documentation is incorrect (#1133)

[massakam]

This is an automated email from the ASF dual-hosted git repository.

massakam pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 1569d61  Property name in documentation is incorrect (#1133)
1569d61 is described below

commit 1569d6175644a763296460435bc7a34bf8d3534f
Author: Ivan Kelly 
AuthorDate: Thu Feb 1 13:13:54 2018 +0100

Property name in documentation is incorrect (#1133)

The setup guides specify that test-prop is the property name, but then
all later examples use test as the property name. This is obviously
wrong, and this patch fixes that.
---
 site/docs/latest/deployment/instance.md | 10 +-
 site/ja/deployment/InstanceSetup.md | 12 ++--
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/site/docs/latest/deployment/instance.md 
b/site/docs/latest/deployment/instance.md
index 5030b08..6a6b34b 100644
--- a/site/docs/latest/deployment/instance.md
+++ b/site/docs/latest/deployment/instance.md
@@ -160,7 +160,7 @@ that can contain many topic. Common practice is to create a 
namespace for each
 different use case from a single tenant.
 
 ```shell
-$ bin/pulsar-admin namespaces create test/us-west/ns1
+$ bin/pulsar-admin namespaces create test-prop/us-west/ns1
 ```
 
 # Testing producer and consumer
@@ -173,24 +173,24 @@ created the first time a producer or a consumer tries to 
use them.
 
 The topic name in this case could be:
 
-{% include topic.html p="test" c="us-west" n="ns1" t="my-topic" %}
+{% include topic.html p="test-prop" c="us-west" n="ns1" t="my-topic" %}
 
 Start a consumer that will create a subscription on the topic and will wait
 for messages:
 
 ```shell
-$ bin/pulsar-perf consume persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf consume persistent://test-prop/us-west/ns1/my-topic
 ```
 
 Start a producer that publishes messages at a fixed rate and report stats every
 10 seconds:
 
 ```shell
-$ bin/pulsar-perf produce persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf produce persistent://test-prop/us-west/ns1/my-topic
 ```
 
 To report the topic stats:
 
 ```shell
-$ bin/pulsar-admin persistent stats persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-admin persistent stats persistent://test-prop/us-west/ns1/my-topic
 ```
diff --git a/site/ja/deployment/InstanceSetup.md 
b/site/ja/deployment/InstanceSetup.md
index 8c7e0b0..bfc76c8 100644
--- a/site/ja/deployment/InstanceSetup.md
+++ b/site/ja/deployment/InstanceSetup.md
@@ -149,14 +149,14 @@ $ bin/pulsar-admin properties create test-prop \
   --admin-roles test-admin-role
 ```
 
-これによって、`test-admin-role`ロールを持つユーザが`us-west`クラスタのみを使用できる`test`プロパティの設定を管理できるようになります。これ以降は、テナントはリソースを自分自身で管理できます。
+これによって、`test-admin-role`ロールを持つユーザが`us-west`クラスタのみを使用できる`test-prop`プロパティの設定を管理できるようになります。これ以降は、テナントはリソースを自分自身で管理できます。
 
 テナントが作成されたら、そのプロパティ内のトピックの{% popover_ja ネームスペース %}を作成する必要があります。
 
 
最初のステップはネームスペースを作成する事です。ネームスペースは多くのトピックを含む事のできる管理単位です。一般的な方法は、単一のテナントからユースケースごとにネームスペースを作成するというものです。
 
 ```shell
-$ bin/pulsar-admin namespaces create test/us-west/ns1
+$ bin/pulsar-admin namespaces create test-prop/us-west/ns1
 ```
 
 # ProducerとConsumerのテスト
@@ -167,22 +167,22 @@ $ bin/pulsar-admin namespaces create test/us-west/ns1
 
 この場合のトピック名は次のようになります:
 
-{% include topic.html p="test" c="us-west" n="ns1" t="my-topic" %}
+{% include topic.html p="test-prop" c="us-west" n="ns1" t="my-topic" %}
 
 トピックのサブスクリプションを作成し、メッセージを待ち受けるConsumerを開始します:
 
 ```shell
-$ bin/pulsar-perf consume persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf consume persistent://test-prop/us-west/ns1/my-topic
 ```
 
 一定のレートでメッセージを送信し、10秒ごとに統計情報をレポートするProducerを開始します:
 
 ```shell
-$ bin/pulsar-perf produce persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf produce persistent://test-prop/us-west/ns1/my-topic
 ```
 
 トピックの統計情報を閲覧するには次のコマンドを実行します:
 
 ```shell
-$ bin/pulsar-admin persistent stats persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-admin persistent stats persistent://test-prop/us-west/ns1/my-topic
 ```

-- 
To stop receiving notification emails like this one, please contact
massa...@apache.org.

[GitHub] massakam closed pull request #1133: Property name in documentation is incorrect

[GitBox]

massakam closed pull request #1133: Property name in documentation is incorrect
URL: https://github.com/apache/incubator-pulsar/pull/1133
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/site/docs/latest/deployment/instance.md 
b/site/docs/latest/deployment/instance.md
index 5030b086e..6a6b34ba3 100644
--- a/site/docs/latest/deployment/instance.md
+++ b/site/docs/latest/deployment/instance.md
@@ -160,7 +160,7 @@ that can contain many topic. Common practice is to create a 
namespace for each
 different use case from a single tenant.
 
 ```shell
-$ bin/pulsar-admin namespaces create test/us-west/ns1
+$ bin/pulsar-admin namespaces create test-prop/us-west/ns1
 ```
 
 # Testing producer and consumer
@@ -173,24 +173,24 @@ created the first time a producer or a consumer tries to 
use them.
 
 The topic name in this case could be:
 
-{% include topic.html p="test" c="us-west" n="ns1" t="my-topic" %}
+{% include topic.html p="test-prop" c="us-west" n="ns1" t="my-topic" %}
 
 Start a consumer that will create a subscription on the topic and will wait
 for messages:
 
 ```shell
-$ bin/pulsar-perf consume persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf consume persistent://test-prop/us-west/ns1/my-topic
 ```
 
 Start a producer that publishes messages at a fixed rate and report stats every
 10 seconds:
 
 ```shell
-$ bin/pulsar-perf produce persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf produce persistent://test-prop/us-west/ns1/my-topic
 ```
 
 To report the topic stats:
 
 ```shell
-$ bin/pulsar-admin persistent stats persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-admin persistent stats persistent://test-prop/us-west/ns1/my-topic
 ```
diff --git a/site/ja/deployment/InstanceSetup.md 
b/site/ja/deployment/InstanceSetup.md
index 8c7e0b0a9..bfc76c83a 100644
--- a/site/ja/deployment/InstanceSetup.md
+++ b/site/ja/deployment/InstanceSetup.md
@@ -149,14 +149,14 @@ $ bin/pulsar-admin properties create test-prop \
   --admin-roles test-admin-role
 ```
 
-???`test-admin-role`??`us-west``test`??
+???`test-admin-role`??`us-west``test-prop`??
 
 ??{% popover_ja ??? %}?
 
 

 
 ```shell
-$ bin/pulsar-admin namespaces create test/us-west/ns1
+$ bin/pulsar-admin namespaces create test-prop/us-west/ns1
 ```
 
 # Producer?Consumer
@@ -167,22 +167,22 @@ $ bin/pulsar-admin namespaces create test/us-west/ns1
 
 :
 
-{% include topic.html p="test" c="us-west" n="ns1" t="my-topic" %}
+{% include topic.html p="test-prop" c="us-west" n="ns1" t="my-topic" %}
 
 ??Consumer??:
 
 ```shell
-$ bin/pulsar-perf consume persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf consume persistent://test-prop/us-west/ns1/my-topic
 ```
 
 ?10???Producer??:
 
 ```shell
-$ bin/pulsar-perf produce persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-perf produce persistent://test-prop/us-west/ns1/my-topic
 ```
 
 :
 
 ```shell
-$ bin/pulsar-admin persistent stats persistent://test/us-west/ns1/my-topic
+$ bin/pulsar-admin persistent stats persistent://test-prop/us-west/ns1/my-topic
 ```


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1089: WIP - PIP-12 Introduce builder for creating Producer Consumer Reader

[GitBox]

zhaijack commented on issue #1089: WIP - PIP-12 Introduce builder for creating 
Producer Consumer Reader
URL: https://github.com/apache/incubator-pulsar/pull/1089#issuecomment-362240746
 
 
   +1, LGTM, currently.
   Once we have subscription for topics and pattern, we need add them in.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1101: PIP-13-2: support regex based subscription

[GitBox]

zhaijack commented on issue #1101: PIP-13-2: support regex based subscription
URL: 
https://github.com/apache/incubator-pulsar/issues/1101#issuecomment-362240471
 
 
   PR #1165 is for this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack opened a new pull request #1165: PIP-13-2: support regex based subscription

[GitBox]

zhaijack opened a new pull request #1165: PIP-13-2: support regex based 
subscription
URL: https://github.com/apache/incubator-pulsar/pull/1165
 
 
   This change is based on work of #1103, which already contains the first 3 
commits there. In this PR, the commits after the first 3 are for this sub-task. 
   
   ### Motivation
   This is a second sub-task for pip-13, which would like to leverage the first 
task to support regex based subscription.
   
   ### Modifications
   - add subscribe methods with regex pattern in `PulsarClient` and 
`PulsarClientImpl`.
   - add `PatternTopicsConsumerImpl`, which extends `TopicsConsumerImpl`.
   - add a binary proto command to get topics under a namespace. 
   - add a test to verify.
   
   ### Result
   old methods behaviour not changed, 
   user could use new method to subscribe to topics based on regex pattern


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1100: PIP-13-1: sub-regex: Provide `TopicsConsumer` to consume from several topics under same namespace.

[GitBox]

zhaijack commented on issue #1100: PIP-13-1: sub-regex: Provide 
`TopicsConsumer` to consume from several topics under same namespace.
URL: 
https://github.com/apache/incubator-pulsar/issues/1100#issuecomment-362239365
 
 
   PR #1103 is for this


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on a change in pull request #1066: Issue 937: add CommandGetLastMessageId to make reader know the end of topic

[GitBox]

zhaijack commented on a change in pull request #1066: Issue 937: add 
CommandGetLastMessageId to make reader know the end of topic
URL: https://github.com/apache/incubator-pulsar/pull/1066#discussion_r165329904
 
 

 ##
 File path: pulsar-common/src/main/proto/PulsarApi.proto
 ##
 @@ -415,6 +415,16 @@ message CommandConsumerStatsResponse {
 optional uint64 msgBacklog  = 15;
 }
 
+message CommandGetLastMessageId {
 
 Review comment:
   Thanks, will add it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhaijack commented on issue #1066: Issue 937: add CommandGetLastMessageId to make reader know the end of topic

[GitBox]

zhaijack commented on issue #1066: Issue 937: add CommandGetLastMessageId to 
make reader know the end of topic
URL: https://github.com/apache/incubator-pulsar/pull/1066#issuecomment-362237867
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1133: Property name in documentation is incorrect

[GitBox]

ivankelly commented on issue #1133: Property name in documentation is incorrect
URL: https://github.com/apache/incubator-pulsar/pull/1133#issuecomment-362219699
 
 
   retest this please
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1136: Read compacted consumer flag

[GitBox]

ivankelly commented on issue #1136: Read compacted consumer flag
URL: https://github.com/apache/incubator-pulsar/pull/1136#issuecomment-362203018
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #1136: Read compacted consumer flag

[GitBox]

ivankelly commented on issue #1136: Read compacted consumer flag
URL: https://github.com/apache/incubator-pulsar/pull/1136#issuecomment-362202973
 
 
   retest this please
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #1136: Read compacted consumer flag

[GitBox]

ivankelly commented on a change in pull request #1136: Read compacted consumer 
flag
URL: https://github.com/apache/incubator-pulsar/pull/1136#discussion_r165292778
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/api/ConsumerConfiguration.java
 ##
 @@ -270,6 +272,23 @@ public void setPriorityLevel(int priorityLevel) {
 this.priorityLevel = priorityLevel;
 }
 
+public boolean getReadCompacted() {
+return readCompacted;
+}
+
+/**
+ * If enabled, the consumer will read messages from the compacted topic 
rather than reading the full message
+ * backlog of the topic. This means that, if the topic has been compacted, 
the consumer will only see the latest
+ * value for each key in the topic, up until the point in the topic 
message backlog that has been compacted.
+ * Beyond that point, the messages will be sent as normal.
+ *
+ * @param readCompacted whether to read from the compacted topic
+ */
+public ConsumerConfiguration setReadCompacted(boolean readCompacted) {
 
 Review comment:
   In what I have now, it only works with subs with a single active consumer 
[1]. I don't see why it couldn't work with multiple active consumers though, 
the mechanism for reading should be fairly similar.
   
   [1] 
https://github.com/ivankelly/incubator-pulsar/blob/54964897c91ec972d9ac3fbbbd096e41e028cec6/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherSingleActiveConsumer.java#L300


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-pulsar] branch asf-site updated: Updated site at revision 6154e3c

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 3c0099a  Updated site at revision 6154e3c
3c0099a is described below

commit 3c0099acefa98bc3f0b02c0a3a7642259e7e8cf3
Author: jenkins 
AuthorDate: Thu Feb 1 08:14:16 2018 +

Updated site at revision 6154e3c
---
 .../docs/latest/adaptors/KafkaWrapper/index.html   |  13 ++
 .../docs/latest/adaptors/PulsarSpark/index.html|  17 ++-
 .../docs/latest/adaptors/PulsarStorm/index.html|  15 ++-
 content/docs/latest/admin-api/brokers/index.html   |   7 ++
 content/docs/latest/admin-api/clusters/index.html  |   7 ++
 .../docs/latest/admin-api/namespaces/index.html|   7 ++
 .../admin-api/non-persistent-topics/index.html |   7 ++
 content/docs/latest/admin-api/overview/index.html  |  25 +++-
 .../latest/admin-api/partitioned-topics/index.html |   7 ++
 .../docs/latest/admin-api/permissions/index.html   |   7 ++
 .../latest/admin-api/persistent-topics/index.html  |   7 ++
 .../docs/latest/admin-api/properties/index.html|   7 ++
 content/docs/latest/admin/Authz/index.html |  41 +--
 content/docs/latest/admin/Dashboard/index.html |   7 ++
 .../docs/latest/admin/GeoReplication/index.html|   7 ++
 .../latest/admin/ModularLoadManager/index.html |   7 ++
 .../latest/admin/{Dashboard => Proxy}/index.html   |  72 +++
 content/docs/latest/admin/Stats/index.html |   7 ++
 .../latest/admin/ZooKeeperBookKeeper/index.html|   7 ++
 content/docs/latest/advanced/Encryption/index.html |  15 ++-
 .../latest/advanced/PartitionedTopics/index.html   |  25 +++-
 .../latest/advanced/RetentionExpiry/index.html |  19 ++-
 content/docs/latest/clients/Cpp/index.html |  19 ++-
 content/docs/latest/clients/Java/index.html|  19 ++-
 content/docs/latest/clients/Python/index.html  |  15 ++-
 content/docs/latest/clients/WebSocket/index.html   |  13 ++
 .../docs/latest/deployment/Kubernetes/index.html   |  15 ++-
 .../docs/latest/deployment/Monitoring/index.html   |   7 ++
 .../docs/latest/deployment/aws-cluster/index.html  |  21 +++-
 content/docs/latest/deployment/cluster/index.html  |  19 ++-
 content/docs/latest/deployment/dcos/index.html |   7 ++
 content/docs/latest/deployment/instance/index.html |  19 ++-
 .../docs/latest/getting-started/Clients/index.html |   7 ++
 .../ConceptsAndArchitecture/index.html |  89 +-
 .../latest/getting-started/LocalCluster/index.html |  15 ++-
 .../docs/latest/getting-started/docker/index.html  |  17 ++-
 .../docs/latest/project/BinaryProtocol/index.html  |  15 ++-
 content/docs/latest/project/Codebase/index.html|   7 ++
 .../docs/latest/project/SimulationTools/index.html |   9 ++
 content/docs/latest/reference/CliTools/index.html  | 131 +++--
 .../docs/latest/reference/Configuration/index.html |   7 ++
 content/docs/latest/reference/RestApi/index.html   |   7 ++
 content/ja/adaptors/PulsarSpark/index.html |  14 ++-
 content/ja/adaptors/PulsarStorm/index.html |  10 +-
 content/ja/admin/AdminInterface/index.html |  20 +++-
 content/ja/admin/Authz/index.html  |  26 +++-
 content/ja/admin/ClustersBrokers/index.html|  20 +++-
 content/ja/admin/PropertiesNamespaces/index.html   |  14 ++-
 content/ja/advanced/PartitionedTopics/index.html   |  16 ++-
 content/ja/advanced/RetentionExpiry/index.html |  22 ++--
 content/ja/clients/Cpp/index.html  |   6 +-
 content/ja/clients/Java/index.html |  10 +-
 content/ja/clients/Python/index.html   |  10 +-
 content/ja/clients/WebSocket/index.html|  14 ++-
 content/ja/deployment/InstanceSetup/index.html |  12 +-
 content/ja/deployment/Kubernetes/index.html|   4 +
 .../ConceptsAndArchitecture/index.html |   2 +
 content/ja/getting-started/LocalCluster/index.html |   4 +
 content/ja/project/BinaryProtocol/index.html   |   4 +
 content/ja/project/SimulationTools/index.html  |   4 +-
 content/ja/reference/CliTools/index.html   | 118 ++-
 61 files changed, 981 insertions(+), 140 deletions(-)

diff --git a/content/docs/latest/adaptors/KafkaWrapper/index.html 
b/content/docs/latest/adaptors/KafkaWrapper/index.html
index 0e58f1b..ea23058 100644
--- a/content/docs/latest/adaptors/KafkaWrapper/index.html
+++ b/content/docs/latest/adaptors/KafkaWrapper/index.html
@@ -444,6 +444,13 @@
 
   
   
+  
+  
+
+  Pulsar proxy
+
+  
+  
 
   
 
@@ -922,6 +929,8 @@
   
   
   
+  
+  
 
   
 
@@ -1114,6 +1123,8 @@
   
   
   
+  
+