date:20191101

[GitHub] [pulsar] codelipenghui removed a comment on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce

2019-11-01 Thread GitBox

codelipenghui removed a comment on issue #5491: Fix message deduplicate issue 
while using external sequence id with batch produce
URL: https://github.com/apache/pulsar/pull/5491#issuecomment-549013173
 
 
   @merlimat Please help take a look, i have addressed your comments


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce

2019-11-01 Thread GitBox

codelipenghui commented on issue #5491: Fix message deduplicate issue while 
using external sequence id with batch produce
URL: https://github.com/apache/pulsar/pull/5491#issuecomment-549013173
 
 
   @merlimat Please help take a look, i have addressed your comments


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce

2019-11-01 Thread GitBox

codelipenghui commented on issue #5491: Fix message deduplicate issue while 
using external sequence id with batch produce
URL: https://github.com/apache/pulsar/pull/5491#issuecomment-549013128
 
 
   @merlimat Please help take a look, i have addressed your comments


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce

2019-11-01 Thread GitBox

codelipenghui commented on issue #5491: Fix message deduplicate issue while 
using external sequence id with batch produce
URL: https://github.com/apache/pulsar/pull/5491#issuecomment-549013110
 
 
   run java8 tests
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ronaldofreitas commented on issue #5373: Pulsar Standalone is shutdown before complete starting

2019-11-01 Thread GitBox

ronaldofreitas commented on issue #5373: Pulsar Standalone is shutdown before 
complete starting
URL: https://github.com/apache/pulsar/issues/5373#issuecomment-549011837
 
 
   Ok, I'll try, it makes sense what you said.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] sijie commented on a change in pull request #81: Add reconnect timeout after unload topic

2019-11-01 Thread GitBox

sijie commented on a change in pull request #81: Add reconnect timeout after 
unload topic
URL: https://github.com/apache/pulsar-client-go/pull/81#discussion_r341798895
 
 

 ##
 File path: pulsar/impl_partition_producer.go
 ##
 @@ -176,6 +176,8 @@ func (p *partitionProducer) ConnectionClosed() {
 func (p *partitionProducer) reconnectToBroker() {
p.log.Info("Reconnecting to broker")
backoff := internal.Backoff{}
+   // Delay one second to reconnect
+   time.Sleep(1 * time.Second)
 
 Review comment:
   Can we step back and figure out if the client handles `ServiceNotReady` 
correctly before fixing this by adding time before reconnecting?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] tuteng commented on a change in pull request #81: Add reconnect timeout after unload topic

2019-11-01 Thread GitBox

tuteng commented on a change in pull request #81: Add reconnect timeout after 
unload topic
URL: https://github.com/apache/pulsar-client-go/pull/81#discussion_r341798840
 
 

 ##
 File path: pulsar/impl_partition_producer.go
 ##
 @@ -176,6 +176,8 @@ func (p *partitionProducer) ConnectionClosed() {
 func (p *partitionProducer) reconnectToBroker() {
p.log.Info("Reconnecting to broker")
backoff := internal.Backoff{}
+   // Delay one second to reconnect
+   time.Sleep(1 * time.Second)
 
 Review comment:
   I'm not sure if there is logic for multiple retries after a failure on the 
go client-side. in my test, I found that after the topic was unload, the topic 
on the broker side was in the fence state, but the client did not seem to be 
performing multiple retries, the producer side was blocked.
   You may be right, correctly handling this error `ServiceNotReady` and then 
try again many times should be able to solve this problem.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] tuteng commented on issue #81: Add reconnect timeout after unload topic

2019-11-01 Thread GitBox

tuteng commented on issue #81: Add reconnect timeout after unload topic
URL: https://github.com/apache/pulsar-client-go/pull/81#issuecomment-549009567
 
 
   > Is it just printing the error message, right? The reconnection logic 
should go through after that.
   > 
   > I think the initial sleep is actually correct, in the sense that we don’t 
want to immediately aggressively reconnect. The chances of the topic being 
ready immediately after a failover are thin.
   > 
   > Though the 1sec sleep is too big. It will impose that latency spike on the 
produced messages. In the Java and C++ clients we’re actually using a 100ms 
initial back off delay, and then double it after that.
   > 
   > One other option is to convert the loop into first sleeping and then 
trying, rather than the reverse
   
   What I have observed now is that after the topic unload, it tries to 
reconnect. As a result, the broker-side Topic is in a fence state, causing the 
reconnection to fail. Then produce hang lives and do not send data. I did not 
have this problem using the java client


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] merlimat commented on issue #81: Add reconnect timeout after unload topic

2019-11-01 Thread GitBox

merlimat commented on issue #81: Add reconnect timeout after unload topic
URL: https://github.com/apache/pulsar-client-go/pull/81#issuecomment-549007052
 
 
   Is it just printing the error message, right? The reconnection logic should 
go through after that. 
   
   I think the initial sleep is actually correct, in the sense that we don’t 
want to immediately aggressively reconnect. The chances of the topic being 
ready immediately after a failover are thin. 
   
   Though the 1sec sleep is too big. It will impose that latency spike on the 
produced messages. In the Java and C++ clients we’re actually using a 100ms 
initial back off delay, and then double it after that. 
   
   One other option is to convert the loop into first sleeping and then trying, 
rather than the reverse


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] sijie commented on a change in pull request #81: Add reconnect timeout after unload topic

2019-11-01 Thread GitBox

sijie commented on a change in pull request #81: Add reconnect timeout after 
unload topic
URL: https://github.com/apache/pulsar-client-go/pull/81#discussion_r341797384
 
 

 ##
 File path: pulsar/impl_partition_producer.go
 ##
 @@ -176,6 +176,8 @@ func (p *partitionProducer) ConnectionClosed() {
 func (p *partitionProducer) reconnectToBroker() {
p.log.Info("Reconnecting to broker")
backoff := internal.Backoff{}
+   // Delay one second to reconnect
+   time.Sleep(1 * time.Second)
 
 Review comment:
   @tuteng I am not sure this is a right fix. It seems to me that we didn't 
handle the exception `ServiceNotReady` well. I think we need to add logic to 
handle `ServiceNotReady` and reconnect.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: Function runtime pluggable (#5463)

2019-11-01 Thread aahmed

This is an automated email from the ASF dual-hosted git repository.

aahmed pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new 9728d54  Function runtime pluggable (#5463)
9728d54 is described below

commit 9728d5429bfe359dfc3de4b277b9fbe108e0bd8e
Author: Boyang Jerry Peng 
AuthorDate: Fri Nov 1 20:12:24 2019 -0700

Function runtime pluggable (#5463)

* Allow Pulsar Function Runtimes to be pluggable

* cleaning up
---
 conf/functions_worker.yml  |  38 ++-
 .../worker/PulsarFunctionE2ESecurityTest.java  |   8 +-
 .../worker/PulsarFunctionLocalRunTest.java |   7 +-
 .../worker/PulsarFunctionPublishTest.java  |  10 +-
 .../worker/PulsarWorkerAssignmentTest.java |   7 +-
 .../apache/pulsar/io/PulsarFunctionAdminTest.java  |   7 +-
 .../apache/pulsar/io/PulsarFunctionE2ETest.java|  11 +-
 .../apache/pulsar/io/PulsarFunctionTlsTest.java|   7 +-
 .../org/apache/pulsar/functions/LocalRunner.java   |   4 +-
 .../functions/runtime/JavaInstanceStarter.java |   1 +
 .../pulsar/functions/runtime/RuntimeFactory.java   |  13 +
 .../pulsar/functions/runtime/RuntimeUtils.java |   7 +-
 .../{ => kubernetes}/KubernetesRuntime.java|   4 +-
 .../{ => kubernetes}/KubernetesRuntimeFactory.java | 277 ++---
 .../kubernetes/KubernetesRuntimeFactoryConfig.java | 128 ++
 .../runtime/{ => process}/ProcessRuntime.java  |   4 +-
 .../{ => process}/ProcessRuntimeFactory.java   |  68 -
 .../process/ProcessRuntimeFactoryConfig.java   |  46 
 .../runtime/{ => thread}/ThreadRuntime.java|   5 +-
 .../runtime/{ => thread}/ThreadRuntimeFactory.java |  80 +++---
 .../runtime/thread/ThreadRuntimeFactoryConfig.java |  34 +++
 .../pulsar/functions/worker/WorkerConfig.java  | 212 +---
 .../KubernetesRuntimeFactoryTest.java  | 109 ++--
 .../{ => kubernetes}/KubernetesRuntimeTest.java|  62 ++---
 .../runtime/{ => process}/ProcessRuntimeTest.java  |  33 ++-
 .../apache/pulsar/functions/utils/Reflections.java |   2 +-
 .../pulsar/functions/utils/ReflectionsTest.java|   2 +-
 .../functions/worker/FunctionRuntimeManager.java   |  80 ++
 .../functions/worker/FunctionsStatsGenerator.java  |   2 +-
 .../functions/worker/FunctionActionerTest.java |  21 +-
 .../worker/FunctionRuntimeManagerTest.java | 216 +++-
 .../functions/worker/MembershipManagerTest.java|  19 +-
 .../functions/worker/SchedulerManagerTest.java |   9 +-
 33 files changed, 1029 insertions(+), 504 deletions(-)

diff --git a/conf/functions_worker.yml b/conf/functions_worker.yml
index ed8e18e..88aeaa7 100644
--- a/conf/functions_worker.yml
+++ b/conf/functions_worker.yml
@@ -92,17 +92,33 @@ topicCompactionFrequencySec: 1800
 # Function Runtime Management
 ###
 
-#threadContainerFactory:
+ Process Runtime  
+# Pulsar function instances are launched as processes
+
+functionRuntimeFactoryClassName: 
org.apache.pulsar.functions.runtime.process.ProcessRuntimeFactory
+functionRuntimeFactoryConfigs:
+# location of log files for functions
+logDirectory: /tmp
+# change the jar location only when you put the java instance jar in a 
different location
+javaInstanceJarLocation:
+# change the python instance location only when you put the python 
instance jar in a different location
+pythonInstanceLocation:
+# change the extra dependencies location:
+extraFunctionDependenciesDir:
+
+ Thread Runtime 
+# Pulsar function instances are run as threads
+
+#functionRuntimeFactoryClassName: 
org.apache.pulsar.functions.runtime.thread.ThreadRuntimeFactory
+#functionRuntimeFactoryConfigs:
+#  # thread group name
 #  threadGroupName: "Thread Function Container Group"
-processContainerFactory:
-  logDirectory:
-  # change the jar location only when you put the java instance jar in a 
different location
-  javaInstanceJarLocation:
-  # change the python instance location only when you put the python instance 
jar in a different location
-  pythonInstanceLocation:
-  # change the extra dependencies location:
-  extraFunctionDependenciesDir:
-#kubernetesContainerFactory:
+
+ Kubernetes Runtime 
+# Pulsar function are deployed to Kubernetes
+
+#functionRuntimeFactoryClassName: 
org.apache.pulsar.functions.runtime.kubernetes.KubernetesRuntimeFactory
+#functionRuntimeFactoryConfigs:
 #  # uri to kubernetes cluster, leave it to empty and it will use the 
kubernetes settings in function worker
 #  k8Uri:
 #  # the kubernetes namespace to run the function instances. it is `default`, 
if this setting is left to be empty
@@ -116,7 +132,7 @@ processContainerFactory:
 #  # setting this to true is let function worker to submit functions to the 
same k8s cluster as function worker
 #  # is running. setting this

[GitHub] [pulsar] aahmed-se merged pull request #5463: Function runtime pluggable

2019-11-01 Thread GitBox

aahmed-se merged pull request #5463: Function runtime pluggable
URL: https://github.com/apache/pulsar/pull/5463
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] tuteng opened a new pull request #81: Add reconnect timeout after unload topic

2019-11-01 Thread GitBox

tuteng opened a new pull request #81: Add reconnect timeout after unload topic
URL: https://github.com/apache/pulsar-client-go/pull/81
 
 
   
   Currently, for a producer that is sending data, if the topic is unload, the 
following exception will be thrown, resulting in reconnection failure
   ```
   ERRO[0094] Error: ServiceNotReady, Error Message: Topic is temporarily 
unavailable  local_addr="127.0.0.1:55800" remote_addr="pulsar://localhost:6650"
   ```
   
   Reproduce:
   ```
   1. pulsar-perf produce -s 10 test-topic
   2. ./bin/pulsar-admin topics unload test-topic
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar-client-go] branch fix/topic-unload-reconnect updated: Fix reconnect after unload topic

2019-11-01 Thread guangning

This is an automated email from the ASF dual-hosted git repository.

guangning pushed a commit to branch fix/topic-unload-reconnect
in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git


The following commit(s) were added to refs/heads/fix/topic-unload-reconnect by 
this push:
 new b853799  Fix reconnect after unload topic
b853799 is described below

commit b85379902a3f378e1d95d4c50fdd37291c8b4011
Author: tuteng 
AuthorDate: Sat Nov 2 10:57:54 2019 +0800

Fix reconnect after unload topic
---
 pulsar/impl_partition_producer.go | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/pulsar/impl_partition_producer.go 
b/pulsar/impl_partition_producer.go
index 196dbbc..c06b47e 100644
--- a/pulsar/impl_partition_producer.go
+++ b/pulsar/impl_partition_producer.go
@@ -176,6 +176,8 @@ func (p *partitionProducer) ConnectionClosed() {
 func (p *partitionProducer) reconnectToBroker() {
p.log.Info("Reconnecting to broker")
backoff := internal.Backoff{}
+   // Delay one secnd to reconnect
+   time.Sleep(1 * time.Second)
for {
if p.state != producerReady {
// Producer is already closing

[pulsar-client-go] branch fix/topic-unload-reconnect created (now d645f94)

2019-11-01 Thread guangning

This is an automated email from the ASF dual-hosted git repository.

guangning pushed a change to branch fix/topic-unload-reconnect
in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git.


  at d645f94  Remove usage of global consumer vars. (#79)

No new revisions were added by this update.

[GitHub] [pulsar] sijie commented on issue #5373: Pulsar Standalone is shutdown before complete starting

2019-11-01 Thread GitBox

sijie commented on issue #5373: Pulsar Standalone is shutdown before complete 
starting
URL: https://github.com/apache/pulsar/issues/5373#issuecomment-549003857
 
 
   @ronaldofreitas standalone for development purpose. so we enable all the 
components and new features in standalone. The one you had problems with is 
from function state. It is irrelative to Pulsar pub/sub. You can use "-nss" to 
disable this component.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #5504: [Transaction coordinator]Add transaction metadata store service

2019-11-01 Thread GitBox

sijie commented on issue #5504: [Transaction coordinator]Add transaction 
metadata store service
URL: https://github.com/apache/pulsar/pull/5504#issuecomment-549003393
 
 
   @codelipenghui can you rebase this pull request? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ronaldofreitas commented on issue #5373: Pulsar Standalone is shutdown before complete starting

2019-11-01 Thread GitBox

ronaldofreitas commented on issue #5373: Pulsar Standalone is shutdown before 
complete starting
URL: https://github.com/apache/pulsar/issues/5373#issuecomment-548998966
 
 
   I haven't run bin/pulsar standalone -nss yet but even if that solved my 
problem, I would still be unsure of putting it into production. I am currently 
testing NATS. Thanks for the answer.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar-client-go] cckellogg opened a new pull request #80: Improve and fix message parsing.

2019-11-01 Thread GitBox

cckellogg opened a new pull request #80: Improve and fix message parsing.
URL: https://github.com/apache/pulsar-client-go/pull/80
 
 
   Fix and improve message parsing. This handles batch and single messages.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on issue #5463: Function runtime pluggable

2019-11-01 Thread GitBox

jerrypeng commented on issue #5463: Function runtime pluggable
URL: https://github.com/apache/pulsar/pull/5463#issuecomment-548900595
 
 
   @sijie please take a look again.  I have replied to your comments.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se edited a comment on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

aahmed-se edited a comment on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-54888
 
 
   conf file is a better option, makes it more explicit


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se edited a comment on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

aahmed-se edited a comment on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-54888
 
 
   conf file is a better option, makes it more explicit.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

aahmed-se commented on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-54888
 
 
   conf file is a better options makes it more explicit


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on issue #5463: Function runtime pluggable

2019-11-01 Thread GitBox

jerrypeng commented on issue #5463: Function runtime pluggable
URL: https://github.com/apache/pulsar/pull/5463#issuecomment-548878945
 
 
   rerun integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb opened a new pull request #5536: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb opened a new pull request #5536: Documentation updates, mainly for 
Python API
URL: https://github.com/apache/pulsar/pull/5536
 
 
   Replacement for #5531 
   
   This one also removes functions-api.md and functions-state.md (most of the 
latter moved to functions-develop.md)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on issue #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on issue #5531: Documentation updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#issuecomment-548833634
 
 
   Never mind, I will open new PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on issue #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on issue #5531: Documentation updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#issuecomment-548832962
 
 
   I have force-pushed new version; this also removes functions-api.md and 
functions-state.md


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb closed pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb closed pull request #5531: Documentation updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341617688
 
 

 ##
 File path: site2/docs/functions-api.md
 ##
 @@ -265,6 +265,7 @@ public interface Context {
 void incrCounter(String key, long amount);
 long getCounter(String key);
 void putState(String key, ByteBuffer value);
+void deleteState(String key);
 
 Review comment:
   yes, it should be removed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics

2019-11-01 Thread GitBox

digi691 commented on issue #5417: Broker hangs and crashes when listing 
non-persistent topics
URL: https://github.com/apache/pulsar/issues/5417#issuecomment-548826391
 
 
   Currently `numHttpServerThreads=8` is set to what #3776 suggests.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] addisonj commented on issue #5400: [functions] Allow functions to pass runtime specific options

2019-11-01 Thread GitBox

addisonj commented on issue #5400: [functions] Allow functions to pass runtime 
specific options
URL: https://github.com/apache/pulsar/pull/5400#issuecomment-548826149
 
 
   @jerrypeng @sijie I reworked this to account for the changes from #5404, I 
think it seems pretty reasonable.
   
   If you could give another review and let me know if there is anything else 
you want to see here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on issue #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on issue #5531: Documentation updates, mainly for 
Python API
URL: https://github.com/apache/pulsar/pull/5531#issuecomment-548825768
 
 
   > > We've adopted the new structure for Pulsar Functions in #4554. It's not 
complete at the moment.
   > 
   > Do you think I should wait until the structure is finalized before 
reworking this patch?
   
   You do not need to wait, just go ahead with your patch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341616186

##
File path: site2/docs/functions-develop.md
##
@@ -702,3 +716,7 @@ To access metrics created by Pulsar Functions, refer to
[Monitoring](deploy-moni
Pulsar Functions use [Apache BookKeeper](https://bookkeeper.apache.org) as a
state storage interface. Pulsar installation, including the local standalone
installation, includes deployment of BookKeeper bookies.

Review comment:
"Though", sorry for the typo.
If you think the conjunction is useful, you can remove it.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341614819
 
 

 ##
 File path: site2/docs/functions-develop.md
 ##
 @@ -35,6 +35,20 @@ def process(input):
 ```
 For complete code, see 
[here](https://github.com/apache/pulsar/blob/master/pulsar-functions/python-examples/native_exclamation_function.py).
 
+Note that functions can be written in python2 or python3, but pulsar
+currently only looks for "python" as the interpreter to execute them.
+
+A recent Ubuntu system may have only "python3" but not "python", in which
+case functions will fail to start.  As a workaround you can create a symlink, 
but beware this has some
+[risks](https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3#answer-475815):
+
+```bash
+sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
+```
+
+If you choose to do this, be careful not to install any other package which
+depends on "python" (2.x)
 
 Review comment:
   yes, sure.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on a change in pull request #5531: Documentation updates, 
mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341614553
 
 

 ##
 File path: site2/docs/functions-overview.md
 ##
 @@ -123,6 +123,18 @@ class RoutingFunction(Function):
 context.get_logger().warn(warning)
 ```
 
+If this code is stored in `~/router.py` then you would deploy it in your 
Pulsar cluster using the [command 
line](functions-deploy.md#command-line-interface) as follows.
 
 Review comment:
   OK


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on a change in pull request #5531: Documentation updates, 
mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341614040
 
 

 ##
 File path: site2/docs/functions-develop.md
 ##
 @@ -702,3 +716,7 @@ To access metrics created by Pulsar Functions, refer to 
[Monitoring](deploy-moni
 Pulsar Functions use [Apache BookKeeper](https://bookkeeper.apache.org) as a 
state storage interface. Pulsar installation, including the local standalone 
installation, includes deployment of BookKeeper bookies.
 
 Since Pulsar 2.1.0 release, Pulsar integrates with Apache BookKeeper [table 
service](https://docs.google.com/document/d/155xAwWv5IdOitHh1NVMEwCMGgB28M3FyMiQSxEpjE-Y/edit#heading=h.56rbh52koe3f)
 to store the `State` for functions. For example, a `WordCount` function can 
store its `counters` state into BookKeeper table service via Pulsar Functions 
State API.
+
+States are key-value pairs, where the key is a string and the value is 
arbitrary binary data - counters are stored as 64-bit big-endian binary values. 
 Keys are scoped to an individual pulsar function, but shared between all 
instances of that function.
 
 Review comment:
   s/Thought/Though/ ?
   
   I think the conjunction isn't useful here, as it implies the user already 
knows the scoping of keys.  I don't think it's mentioned earlier, and without 
this knowledge I might have guessed they were scoped differently (to the pulsar 
"namespace" that the function executes within, for example)
   
   So as a user, I just want a statement which answers the question: "What's 
the scope of the key?"
   
   I would be happy with "Keys are scoped to the pulsar function".  I thought 
it worth clarifying functions versus function instances, but maybe that's 
unnecessary.  State storage which *wasn't* shared between function instances 
wouldn't be very useful.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on a change in pull request #5531: Documentation updates,
mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341614407

Since Pulsar 2.1.0 release, Pulsar integrates with Apache BookKeeper [table
service](https://docs.google.com/document/d/155xAwWv5IdOitHh1NVMEwCMGgB28M3FyMiQSxEpjE-Y/edit#heading=h.56rbh52koe3f)
to store the `State` for functions. For example, a `WordCount` function can
store its `counters` state into BookKeeper table service via Pulsar Functions
State API.
+
+States are key-value pairs, where the key is a string and the value is
arbitrary binary data - counters are stored as 64-bit big-endian binary values.
Keys are scoped to an individual pulsar function, but shared between all
instances of that function.
+
+States are accessed within pulsar functions using the `putState`, `getState`,
`incrCounter`, `getCounter` and `deleteState` calls on the context object.
They can also be managed using the [querystate](pulsar-admin.md#querystate) and
[putstate](pulsar-admin.md#putstate) options to `pulsar-admin functions`.

Review comment:
I'll accept that - active rather than passive.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on a change in pull request #5531: Documentation updates, 
mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341610766
 
 

 ##
 File path: site2/docs/functions-develop.md
 ##
 @@ -35,6 +35,20 @@ def process(input):
 ```
 For complete code, see 
[here](https://github.com/apache/pulsar/blob/master/pulsar-functions/python-examples/native_exclamation_function.py).
 
+Note that functions can be written in python2 or python3, but pulsar
+currently only looks for "python" as the interpreter to execute them.
+
+A recent Ubuntu system may have only "python3" but not "python", in which
+case functions will fail to start.  As a workaround you can create a symlink, 
but beware this has some
+[risks](https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3#answer-475815):
+
+```bash
+sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
+```
+
+If you choose to do this, be careful not to install any other package which
+depends on "python" (2.x)
 
 Review comment:
   It could link to #5518 on github instead.  Actually, I'm hoping #5518 will 
accept a new config option for setting the path to the python interpreter, in 
which case we would document that instead.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on a change in pull request #5531: Documentation updates, 
mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341609828
 
 

 ##
 File path: site2/docs/functions-api.md
 ##
 @@ -265,6 +265,7 @@ public interface Context {
 void incrCounter(String key, long amount);
 long getCounter(String key);
 void putState(String key, ByteBuffer value);
+void deleteState(String key);
 
 Review comment:
   I already added it to functions-develop in bcce54d09
   
   This was just aligning functions-api, but if it's not needed then it should 
be rm'd.  (It will still exist in repo history).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on issue #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb commented on issue #5531: Documentation updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#issuecomment-548818868
 
 
   > We've adopted the new structure for Pulsar Functions in #4554. It's not 
complete at the moment.
   
   Do you think I should wait until the structure is finalized before reworking 
this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341593357
 
 

 ##
 File path: site2/docs/functions-develop.md
 ##
 @@ -35,6 +35,20 @@ def process(input):
 ```
 For complete code, see 
[here](https://github.com/apache/pulsar/blob/master/pulsar-functions/python-examples/native_exclamation_function.py).
 
+Note that functions can be written in python2 or python3, but pulsar
+currently only looks for "python" as the interpreter to execute them.
+
+A recent Ubuntu system may have only "python3" but not "python", in which
+case functions will fail to start.  As a workaround you can create a symlink, 
but beware this has some
+[risks](https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3#answer-475815):
+
+```bash
+sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
+```
+
+If you choose to do this, be careful not to install any other package which
+depends on "python" (2.x)
 
 Review comment:
   > Note 
   > You can write Pulsar Functions in python2 or python3. However, Pulsar only 
looks for `python` as the interpreter.
   
   > If you're running Pulsar Functions on Ubuntu system that only supports 
`python3`, you might fail to
   start the functions. In this case, you can create a symlink. However, such 
action
   has potential 
[risks](https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3#answer-475815).
   
   ```bash
   sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
   ```
   > If you create a symlink, you'd better not install any other package that 
depends on "python" (2.x).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341587037
 
 

 ##
 File path: site2/docs/functions-api.md
 ##
 @@ -265,6 +265,7 @@ public interface Context {
 void incrCounter(String key, long amount);
 long getCounter(String key);
 void putState(String key, ByteBuffer value);
+void deleteState(String key);
 
 Review comment:
   Since the `functions-api.md` file is deprecated, we can add this in the 
`functions-develop.md` file.
   In the new structure, Functions develop contains information related to 
available APIs, examples, and other info used for develop Functions. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341601671
 
 

 ##
 File path: site2/docs/functions-state.md
 ##
 @@ -9,9 +9,11 @@ for storing the `State` for functions. For example, A 
`WordCount` function can s
 
 ## API
 
+
 
 Review comment:
   Thank you very much for adding those valuable info. Could we add those info 
in the [functions-develop.md#state-storage] section?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341595137
 
 

 ##
 File path: site2/docs/functions-develop.md
 ##
 @@ -35,6 +35,20 @@ def process(input):
 ```
 For complete code, see 
[here](https://github.com/apache/pulsar/blob/master/pulsar-functions/python-examples/native_exclamation_function.py).
 
+Note that functions can be written in python2 or python3, but pulsar
+currently only looks for "python" as the interpreter to execute them.
+
+A recent Ubuntu system may have only "python3" but not "python", in which
+case functions will fail to start.  As a workaround you can create a symlink, 
but beware this has some
+[risks](https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3#answer-475815):
+
+```bash
+sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
+```
+
+If you choose to do this, be careful not to install any other package which
+depends on "python" (2.x)
 
 Review comment:
   A tiny suggestion on this: If we can summarize the risks briefly, we can use 
a brief summary, and not use external links.
   Reason: The external link might change or someone might delete the Q in 
this link, then the link in our docs will be broken, it's hard to maintain 
external links.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation 
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341597266
 
 

 ##
 File path: site2/docs/functions-overview.md
 ##
 @@ -123,6 +123,18 @@ class RoutingFunction(Function):
 context.get_logger().warn(warning)
 ```
 
+If this code is stored in `~/router.py` then you would deploy it in your 
Pulsar cluster using the [command 
line](functions-deploy.md#command-line-interface) as follows.
 
 Review comment:
   ```suggestion
   If this code is stored in `~/router.py`, then you can deploy it in your 
Pulsar cluster using the [command 
line](functions-deploy.md#command-line-interface) as follows.
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341595947

Review comment:
```suggestion
States are key-value pairs, where the key is a string and the value is
arbitrary binary data - counters are stored as 64-bit big-endian binary values.
Thought keys are scoped to an individual Pulsar Function, the keys are shared
among all instances of that function.
```

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [pulsar] Jennifer88huang commented on a change in pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

Jennifer88huang commented on a change in pull request #5531: Documentation
updates, mainly for Python API
URL: https://github.com/apache/pulsar/pull/5531#discussion_r341596826

Review comment:
```suggestion
You can access states within Pulsar Functions using the `putState`,
`getState`, `incrCounter`, `getCounter` and `deleteState` calls on the context
object. You can also manage states using the
[querystate](pulsar-admin.md#querystate) and
[putstate](pulsar-admin.md#putstate) options to `pulsar-admin functions`.
```

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5491: Fix message deduplicate issue while using external sequence id with batch produce

2019-11-01 Thread GitBox

codelipenghui commented on issue #5491: Fix message deduplicate issue while 
using external sequence id with batch produce
URL: https://github.com/apache/pulsar/pull/5491#issuecomment-548814960
 
 
   > 
org.apache.pulsar.broker.service.PeerReplicatorTest.testPeerClusterInReplicationClusterListChang
   
   run java8 tests
   run integration tests
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5504: [Transaction coordinator]Add transaction metadata store service

2019-11-01 Thread GitBox

codelipenghui commented on issue #5504: [Transaction coordinator]Add 
transaction metadata store service
URL: https://github.com/apache/pulsar/pull/5504#issuecomment-548814173
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated (89a9aaa -> f4d954d)

2019-11-01 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


from 89a9aaa  [zookeeper] bump zookeeper to version 3.5.6 (#5043)
 add f4d954d  [Transaction Coordinator] Bootstrap pulsar system namespace 
and create TC assign topic. (#5515)

No new revisions were added by this update.

Summary of changes:
 bin/pulsar |   3 +
 conf/broker.conf   |   5 +
 .../apache/pulsar/broker/ServiceConfiguration.java |   9 ++
 .../apache/pulsar/PulsarClusterMetadataSetup.java  | 111 -
 .../PulsarTransactionCoordinatorMetadataSetup.java | 100 +++
 .../org/apache/pulsar/broker/PulsarService.java|   5 +
 .../pulsar/broker/namespace/NamespaceService.java  |   2 +-
 .../apache/pulsar/broker/SLAMonitoringTest.java|   2 +-
 .../apache/pulsar/broker/admin/AdminApiTest.java   |   4 +-
 .../pulsar/broker/admin/v1/V1_AdminApiTest.java|   4 +-
 .../TransactionCoordinatorAssignTest.java  |  82 +++
 .../apache/pulsar/common/naming/NamespaceName.java |   2 +
 .../org/apache/pulsar/common/naming/TopicName.java |   3 +
 13 files changed, 303 insertions(+), 29 deletions(-)
 create mode 100644 
pulsar-broker/src/main/java/org/apache/pulsar/PulsarTransactionCoordinatorMetadataSetup.java
 create mode 100644 
pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/coordinator/TransactionCoordinatorAssignTest.java

[GitHub] [pulsar] sijie merged pull request #5515: [Transaction Coordinator] Bootstrap pulsar system namespace and create TC assign topic.

2019-11-01 Thread GitBox

sijie merged pull request #5515: [Transaction Coordinator] Bootstrap pulsar 
system namespace and create TC assign topic.
URL: https://github.com/apache/pulsar/pull/5515
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie closed issue #4448: Upgrade ZooKeeper to 3.5.5 (stable)

2019-11-01 Thread GitBox

sijie closed issue #4448: Upgrade ZooKeeper to 3.5.5 (stable)
URL: https://github.com/apache/pulsar/issues/4448
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie merged pull request #5043: [zookeeper] bump zookeeper to version 3.5.6

2019-11-01 Thread GitBox

sijie merged pull request #5043: [zookeeper] bump zookeeper to version 3.5.6
URL: https://github.com/apache/pulsar/pull/5043
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated (dd58baf -> 89a9aaa)

2019-11-01 Thread sijie

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git.


from dd58baf  fix: Functions unnecessarily restart during 
FunctionRuntimeManager init phase (#5527)
 add 89a9aaa  [zookeeper] bump zookeeper to version 3.5.6 (#5043)

No new revisions were added by this update.

Summary of changes:
 .../org/apache/bookkeeper/test/PortManager.java|   7 --
 pom.xml|   2 +-
 pulsar-io/flume/pom.xml|   4 +-
 ...TestAbstractZooKeeperConfigurationProvider.java |   5 +-
 .../org/apache/pulsar/metadata/TestZKServer.java   |   6 +-
 pulsar-sql/presto-distribution/LICENSE |   2 +-
 .../pulsar/zookeeper/SerializeUtilsAspect.java | 130 -
 .../src/main/resources/META-INF/aop.xml|   2 -
 .../pulsar/zookeeper/ZookeeperServerTest.java  |   2 +
 9 files changed, 15 insertions(+), 145 deletions(-)
 delete mode 100644 
pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/SerializeUtilsAspect.java

[GitHub] [pulsar] KannarFr commented on issue #5519: Create producer issues using function worker as separate node

2019-11-01 Thread GitBox

KannarFr commented on issue #5519: Create producer issues using function worker 
as separate node
URL: https://github.com/apache/pulsar/issues/5519#issuecomment-548778847
 
 
   In separate function worker, the log is:
   
   ```
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]: 
10:46:08.603 [pulsar-client-io-16-1] WARN  
org.apache.pulsar.client.impl.PulsarClientImpl - 
[persistent://clevercloud/functions/assignments] Failed to get partitioned 
topic metadata: org.apache.pulsar.client.api.PulsarClientException$LookupExcepti
   on: java.util.concurrent.ExecutionException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]: 
10:46:08.603 [pulsar-client-io-16-1] INFO  
org.apache.pulsar.client.impl.ClientCnx - [id: 0x307f84dd, L:/10.2.1.4:54488 ! 
R:c1-pulsar-clevercloud-customers.services.clever-cloud.com/185.42.117.147:2001]
 Disconnected
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]: 
10:46:08.603 [main] ERROR org.apache.pulsar.functions.worker.SchedulerManager - 
Exception while at creating producer to topic 
persistent://clevercloud/functions/assignments
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]: 
java.util.concurrent.ExecutionException: 
org.apache.pulsar.client.api.PulsarClientException$LookupException: 
java.util.concurrent.ExecutionException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) 
~[?:1.8.0_192]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) 
~[?:1.8.0_192]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.functions.worker.SchedulerManager.lambda$createProducer$0(SchedulerManager.java:109)
 ~[org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at org.apache.pulsar.functions.utils.Actions.runAction(Actions.java:110) 
[org.apache.pulsar-pulsar-functions-utils-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at org.apache.pulsar.functions.utils.Actions.run(Actions.java:92) 
[org.apache.pulsar-pulsar-functions-utils-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.functions.worker.SchedulerManager.createProducer(SchedulerManager.java:125)
 [org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.functions.worker.SchedulerManager.(SchedulerManager.java:91)
 [org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.functions.worker.WorkerService.start(WorkerService.java:152) 
[org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at org.apache.pulsar.functions.worker.Worker.start(Worker.java:68) 
[org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.functions.worker.FunctionWorkerStarter.main(FunctionWorkerStarter.java:67)
 [org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]: 
Caused by: org.apache.pulsar.client.api.PulsarClientException$LookupException: 
java.util.concurrent.ExecutionException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:895)
 ~[org.apache.pulsar-pulsar-client-original-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.client.impl.ClientCnx.handlePartitionResponse(ClientCnx.java:497)
 ~[org.apache.pulsar-pulsar-client-original-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:103)
 ~[org.apache.pulsar-pulsar-common-2.4.0.jar:2.4.0]
   Oct 31 10:46:08 clevercloud-pulsar-c1-n7 pulsar-functions-worker[5646]:  
   at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
 ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]

[GitHub] [pulsar] KannarFr commented on issue #5519: Zookeeper issue

2019-11-01 Thread GitBox

KannarFr commented on issue #5519: Zookeeper issue
URL: https://github.com/apache/pulsar/issues/5519#issuecomment-548778320
 
 
   After multiple try on restart, clean things. The only way to fix it, is to 
restart brokers. More interesting problem during this one: the functions are 
for admins things 404 not found but the messages are processed. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] KannarFr commented on issue #5519: Zookeeper issue

2019-11-01 Thread GitBox

KannarFr commented on issue #5519: Zookeeper issue
URL: https://github.com/apache/pulsar/issues/5519#issuecomment-548773023
 
 
   And the first exception comes from:
   
   ```Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]: 
12:36:37.911 [pulsar-client-io-16-1] INFO  
org.apache.pulsar.client.impl.ClientCnx - [id: 0x581702af, L:/10.2.1.4:33530 ! 
R:c1-pulsar-clevercloud-customers.services.clever-cloud.com/185.42.117.147:2001]
 Disconnected
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]: 
12:36:37.914 [main] ERROR org.apache.pulsar.functions.worker.SchedulerManager - 
Exception while at creating producer to topic 
persistent://clevercloud/functions/assignments
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]: 
java.util.concurrent.ExecutionException: 
org.apache.pulsar.client.api.PulsarClientException$LookupException: 
java.util.concurrent.ExecutionException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) 
~[?:1.8.0_192]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) 
~[?:1.8.0_192]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.functions.worker.SchedulerManager.lambda$createProducer$0(SchedulerManager.java:109)
 ~[org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at org.apache.pulsar.functions.utils.Actions.runAction(Actions.java:110) 
[org.apache.pulsar-pulsar-functions-utils-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at org.apache.pulsar.functions.utils.Actions.run(Actions.java:92) 
[org.apache.pulsar-pulsar-functions-utils-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.functions.worker.SchedulerManager.createProducer(SchedulerManager.java:125)
 [org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.functions.worker.SchedulerManager.(SchedulerManager.java:91)
 [org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.functions.worker.WorkerService.start(WorkerService.java:152) 
[org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at org.apache.pulsar.functions.worker.Worker.start(Worker.java:68) 
[org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.functions.worker.FunctionWorkerStarter.main(FunctionWorkerStarter.java:67)
 [org.apache.pulsar-pulsar-functions-worker-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]: 
Caused by: org.apache.pulsar.client.api.PulsarClientException$LookupException: 
java.util.concurrent.ExecutionException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:895)
 ~[org.apache.pulsar-pulsar-client-original-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.client.impl.ClientCnx.handlePartitionResponse(ClientCnx.java:497)
 ~[org.apache.pulsar-pulsar-client-original-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:103)
 ~[org.apache.pulsar-pulsar-common-2.4.0.jar:2.4.0]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
 ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
 ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
   Nov 01 12:36:37 clevercloud-pulsar-c1-n7 pulsar-functions-worker[3456]:  
   at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
 ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
   Nov 01 12:36:37

[GitHub] [pulsar] KannarFr commented on issue #5519: Zookeeper issue

2019-11-01 Thread GitBox

KannarFr commented on issue #5519: Zookeeper issue
URL: https://github.com/apache/pulsar/issues/5519#issuecomment-548768940
 
 
   Same issue today:
   
   ```
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.482 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x1054963969b type:delete 
cxid:0xfdf6 zxid:0x496cb3 txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.503 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x305496394510002 type:delete 
cxid:0xf94d zxid:0x496cb5 txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.530 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x305496394510002 type:delete 
cxid:0xf950 zxid:0x496cb8 txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.554 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x305496394510002 type:delete 
cxid:0xf953 zxid:0x496cbb txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.580 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x1054963969b type:delete 
cxid:0xfdfa zxid:0x496cbf txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.600 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x1054963969b type:delete 
cxid:0xfdfc zxid:0x496cc1 txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:12:57 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:12:57.623 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x305496394510002 type:delete 
cxid:0xf956 zxid:0x496cc4 txntype:-1 reqpath:n/a Error 
Path:/ledgers/00/0008 Error:KeeperErrorCode = Directory not empty for 
/ledgers/00/0008
   Nov 01 12:14:53 clevercloud-zookeeper-c2-n3 pulsar-zookeeper[8351]: 
12:14:53.305 [ProcessThread(sid:3 cport:-1):] INFO  
org.apache.zookeeper.server.PrepRequestProcessor - Got user-level 
KeeperException when processing sessionid:0x1054b6c97d10006 type:create 
cxid:0x1 zxid:0x496cda txntype:-1 reqpath:n/a Error Path:/pulsar/functions 
Error:KeeperErrorCode = NodeExists for /pulsar/functions
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on issue #5345: [Doc] Fix doc issues related to BookKeeper, Python Client, and so on

2019-11-01 Thread GitBox

candlerb commented on issue #5345: [Doc] Fix doc issues related to BookKeeper, 
Python Client, and so on
URL: https://github.com/apache/pulsar/issues/5345#issuecomment-548764098
 
 
   The python version message is from 
[`pulsar-client-cpp/python/pulsar/__init__.py`](https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/python/pulsar/__init__.py#L24)
   
   Therefore I think it should be sufficient to update it there, and then 
somehow republish the API docs (maybe this won't happen until a new version of 
the package is pushed into pip?)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] disanji opened a new issue #5535: failed to add producer to topic after connection closed

2019-11-01 Thread GitBox

disanji opened a new issue #5535: failed to add producer to topic after 
connection closed
URL: https://github.com/apache/pulsar/issues/5535
 
 
   Client timed out  while trying to send message to brokers, and failed to add 
producer to topic. Below are related logs.
   
   connection closed log: 
   
![WeChatWorkScreenshot_9cd78c74-bd86-4bbd-bafa-1662679f50f7](https://user-images.githubusercontent.com/8382020/68020781-85d66080-fcda-11e9-8787-aed7ad57f9ef.png)
   
   
   client log:
   
![企业微信截图_2065c1f1-3a3c-4703-b437-c6d35e9de62f](https://user-images.githubusercontent.com/8382020/68020689-398b2080-fcda-11e9-9537-6fcec8ba8cf2.png)
   
![企业微信截图_008d5a89-4c15-4dc6-ae21-1cd9db278c5b](https://user-images.githubusercontent.com/8382020/68020941-072df300-fcdb-11e9-8b93-7c914d4547ee.png)
   
   
   broker log:
   
![企业微信截图_d3451b70-f576-418f-91e6-bbd1ea6662de](https://user-images.githubusercontent.com/8382020/68020691-3abc4d80-fcda-11e9-943d-c4ad07910dd7.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sky-big opened a new issue #5534: Pulsar Operator On Kubernetes

2019-11-01 Thread GitBox

sky-big opened a new issue #5534: Pulsar Operator On Kubernetes
URL: https://github.com/apache/pulsar/issues/5534
 
 
   Add pulsar operator for 
kubernetes(https://github.com/sky-big/pulsar-operator) based on 
deployment/kubernetes/*.yaml
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] wolfstudy commented on issue #5532: Add subscribe position param for consumer of sink

2019-11-01 Thread GitBox

wolfstudy commented on issue #5532: Add subscribe position param for consumer 
of sink
URL: https://github.com/apache/pulsar/pull/5532#issuecomment-548727331
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5532: Add subscribe position param for consumer of sink

2019-11-01 Thread GitBox

codelipenghui commented on issue #5532: Add subscribe position param for 
consumer of sink
URL: https://github.com/apache/pulsar/pull/5532#issuecomment-548723048
 
 
   @wolfstudy please take a look the tests, seems some files are not committed


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb opened a new issue #5533: Missing CLI error handling in pulsar-admin

2019-11-01 Thread GitBox

candlerb opened a new issue #5533: Missing CLI error handling in pulsar-admin
URL: https://github.com/apache/pulsar/issues/5533
 
 
   **Describe the bug**
   pulsar-admin can raise Java exceptions in certain conditions
   
   **To Reproduce**
   
   (1) Giving an invalid command - e.g. "functions getstate" (should be 
"functions querystate"), with or without `--help` added
   
   ```
   $ apache-pulsar-2.4.1/bin/pulsar-admin functions getstate
   Expected a command, got getstate
   
   Exception in thread "main" com.beust.jcommander.ParameterException: Asking 
description for unknown command: null
at 
com.beust.jcommander.JCommander.getCommandDescription(JCommander.java:1003)
at com.beust.jcommander.JCommander.usage(JCommander.java:988)
at com.beust.jcommander.JCommander.usage(JCommander.java:980)
at com.beust.jcommander.JCommander.usage(JCommander.java:972)
at org.apache.pulsar.admin.cli.CmdBase.run(CmdBase.java:49)
at 
org.apache.pulsar.admin.cli.PulsarAdminTool.run(PulsarAdminTool.java:218)
at 
org.apache.pulsar.admin.cli.PulsarAdminTool.main(PulsarAdminTool.java:260)
   ```
   (Hangs here, have to press `^C`)
   
   (2) Giving a querystate command without the "--key" option
   
   ```
   $ apache-pulsar-2.4.1/bin/pulsar-admin functions querystate --name womble
   null
   
   Reason: java.lang.NullPointerException: path is 'null'.
   ```
   
   tcpdump shows no traffic on port 8080, so this appears to be a local CLI 
issue.
   
   **Expected behavior**
   Error message to be reported
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb edited a comment on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

candlerb edited a comment on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-548721196
 
 
   How about adding a setting to broker.conf / standalone.conf? e.g.
   
   ```
   python_interpreter=/usr/bin/python
   ```
   
   However I notice at least one place where searching in  `$PATH` is currently 
used:
   
   ```
   ./pulsar-client-cpp/docker/Dockerfile:ENV 
PATH="/opt/python/${PYTHON_SPEC}/bin:${PATH}"
   ```
   
   So I suggest:
   
   1. Default is `python_interpreter=python`
   2. If the setting starts with slash, it's an absolute path (avoids some 
edge-case security issues)
   3. Otherwise search for it in `$PATH` as today
   
   If you don't want a conf setting, then an environment variable would work 
for me too (`PYTHON=/usr/bin/python3`)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb edited a comment on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

candlerb edited a comment on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-548721196
 
 
   How about adding a setting to broker.conf / standalone.conf? e.g.
   
   ```
   python_interpreter=/usr/bin/python
   ```
   
   However I notice at least one place where searching in  `$PATH` is currently 
used:
   
   ```
   ./pulsar-client-cpp/docker/Dockerfile:ENV 
PATH="/opt/python/${PYTHON_SPEC}/bin:${PATH}"
   ```
   
   So I suggest:
   
   1. Default is `python_interpreter=python`
   2. If the setting starts with slash, it's an absolute path (avoids some 
edge-case security issues)
   3. Otherwise search for it in `$PATH` as today


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb edited a comment on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

candlerb edited a comment on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-548721196
 
 
   How about adding a setting to broker.conf / standalone.conf? e.g.
   
   ```
   python_interpreter=/usr/bin/python
   ```
   
   However I notice some places where searching in  `$PATH` is currently 
required:
   
   ```
   ./pulsar-client-cpp/docker/Dockerfile:ENV 
PATH="/opt/python/${PYTHON_SPEC}/bin:${PATH}"
   ./build/docker/Dockerfile:ENV PATH "$PATH:/usr/local/rvm/bin"
   ```
   
   So I suggest:
   
   1. Default is `python_interpreter=python`
   2. If the setting starts with slash, it's an absolute path (avoids some 
edge-case security issues)
   3. Otherwise search for it in `$PATH` as today


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb commented on issue #5518: python functions cannot find python3 interpreter

2019-11-01 Thread GitBox

candlerb commented on issue #5518: python functions cannot find python3 
interpreter
URL: https://github.com/apache/pulsar/issues/5518#issuecomment-548721196
 
 
   How about adding a setting to broker.conf / standalone.conf? e.g.
   
   ```
   python_interpreter=/usr/bin/python
   ```
   
   However I notice some places where searching in  `$PATH` search is currently 
required:
   
   ```
   ./pulsar-client-cpp/docker/Dockerfile:ENV 
PATH="/opt/python/${PYTHON_SPEC}/bin:${PATH}"
   ./build/docker/Dockerfile:ENV PATH "$PATH:/usr/local/rvm/bin"
   ```
   
   So I suggest:
   
   1. Default is `python_interpreter=python`
   2. If the setting starts with slash, it's an absolute path (avoids some 
edge-case security issues)
   3. Otherwise search for it in `$PATH` as today


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] wolfstudy opened a new pull request #5532: Add subscribe position param for consumer of sink

2019-11-01 Thread GitBox

wolfstudy opened a new pull request #5532: Add subscribe position param for 
consumer of sink
URL: https://github.com/apache/pulsar/pull/5532
 
 
   Signed-off-by: xiaolong.ran 
   
   ### Motivation
   
   In some case, user wants to use a custom subscription position.
   
   ### Modifications
   
   Add `--subs-position` for sink.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] candlerb opened a new pull request #5531: Documentation updates, mainly for Python API

2019-11-01 Thread GitBox

candlerb opened a new pull request #5531: Documentation updates, mainly for 
Python API
URL: https://github.com/apache/pulsar/pull/5531
 
 
   ### Motivation
   
   Documentation improvements
   
   ### Modifications
   
   * functions-debug: show where function stderr logs are written
   * functions-develop: note that pulsar only starts "python", you may need a 
frig to use "python3" (#5518)
   * functions-overview: give example of deploying python function with `--py` 
and `--classname`
   * functions-state: document Python state API (currently says it's not 
implemented)
   * functions-api, functions-state: add deleteState to Java API
   
   Note: `functions-state` and `functions-api` are currently unlinked from the 
navbar.  Need to decide what to do about these - e.g. move valuable parts into 
"functions-develop" and then "git rm" them.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on a change in pull request #5515: [Transaction Coordinator] Bootstrap pulsar system namespace and create TC assign topic.

2019-11-01 Thread GitBox

codelipenghui commented on a change in pull request #5515: [Transaction 
Coordinator] Bootstrap pulsar system namespace and create TC assign topic.
URL: https://github.com/apache/pulsar/pull/5515#discussion_r341478382
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
 ##
 @@ -472,6 +472,11 @@ public Boolean get() {
 // Register heartbeat and bootstrap namespaces.
 this.nsService.registerBootstrapNamespaces();
 
+// Register pulsar system namespaces
+if (config.isTransactionCoordinatorEnabled()) {
+   
this.nsService.registerNamespace(NamespaceName.SYSTEM_NAMESPACE.toString(), 
false);
 
 Review comment:
   Here is the document task tracker 
https://github.com/apache/pulsar/issues/5530


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui opened a new issue #5530: Task tracker for transaction documentation

2019-11-01 Thread GitBox

codelipenghui opened a new issue #5530: Task tracker for transaction 
documentation
URL: https://github.com/apache/pulsar/issues/5530
 
 
   ### This is the task tracker for transaction documentation
   
   - [ ] how to upgrade exists cluster support transaction streaming
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on a change in pull request #5515: [Transaction Coordinator] Bootstrap pulsar system namespace and create TC assign topic.

2019-11-01 Thread GitBox

codelipenghui commented on a change in pull request #5515: [Transaction 
Coordinator] Bootstrap pulsar system namespace and create TC assign topic.
URL: https://github.com/apache/pulsar/pull/5515#discussion_r341477246
 
 

 ##
 File path: 
pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
 ##
 @@ -472,6 +472,11 @@ public Boolean get() {
 // Register heartbeat and bootstrap namespaces.
 this.nsService.registerBootstrapNamespaces();
 
+// Register pulsar system namespaces
+if (config.isTransactionCoordinatorEnabled()) {
+   
this.nsService.registerNamespace(NamespaceName.SYSTEM_NAMESPACE.toString(), 
false);
 
 Review comment:
   Added a new PulsarTransactionCoordinatorMetadataSetup,  use this setup can 
create pulsar/system namespace and tc assign partitioned topic.
   
   Considering we don't have documentation for transaction now, i would like to 
create a task to track the document.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5504: [Transaction coordinator]Add transaction metadata store service

2019-11-01 Thread GitBox

codelipenghui commented on issue #5504: [Transaction coordinator]Add 
transaction metadata store service
URL: https://github.com/apache/pulsar/pull/5504#issuecomment-548688457
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] addisonj commented on issue #5400: [functions] Allow functions to pass runtime specific options

2019-11-01 Thread GitBox

addisonj commented on issue #5400: [functions] Allow functions to pass runtime 
specific options
URL: https://github.com/apache/pulsar/pull/5400#issuecomment-548686861
 
 
   rerun integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #5528: [Doc] Add pulsar-admin command and fix inconsistencies

2019-11-01 Thread GitBox

Anonymitaet commented on issue #5528: [Doc] Add pulsar-admin command and fix 
inconsistencies
URL: https://github.com/apache/pulsar/pull/5528#issuecomment-548686227
 
 
   @codelipenghui thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui commented on issue #5528: [Doc] Add pulsar-admin command and fix inconsistencies

2019-11-01 Thread GitBox

codelipenghui commented on issue #5528: [Doc] Add pulsar-admin command and fix 
inconsistencies
URL: https://github.com/apache/pulsar/pull/5528#issuecomment-548686108
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] Anonymitaet commented on issue #5528: [Doc] Add pulsar-admin command and fix inconsistencies

2019-11-01 Thread GitBox

Anonymitaet commented on issue #5528: [Doc] Add pulsar-admin command and fix 
inconsistencies
URL: https://github.com/apache/pulsar/pull/5528#issuecomment-548686103
 
 
   @Jennifer88huang could you please help review? Thank you


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5463: Function runtime pluggable

2019-11-01 Thread GitBox

aahmed-se commented on issue #5463: Function runtime pluggable
URL: https://github.com/apache/pulsar/pull/5463#issuecomment-548680331
 
 
   @jerrypeng 
   ```
   org/apache/pulsar/client/impl/ReaderBuilderImpl
   Stacktrace
   java.lang.NoClassDefFoundError: 
org/apache/pulsar/client/impl/ReaderBuilderImpl
at 
org.apache.pulsar.client.impl.PulsarClientImpl.newReader(PulsarClientImpl.java:192)
at 
org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.testWindowFunction(PulsarFunctionsTest.java:1019)
at 
org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.testSlidingCountWindowTest(PulsarFunctionsTest.java:1084)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at 
org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
at 
org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] aahmed-se commented on issue #5514: use pulsar get error in mac os

2019-11-01 Thread GitBox

aahmed-se commented on issue #5514: use pulsar get error in mac os
URL: https://github.com/apache/pulsar/issues/5514#issuecomment-548676718
 
 
   @nathan-zhu you have python cache files still interfering with the execution 
remove all *.pyc files.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

80 matches

Mail list logo