[GitHub] [pulsar] codelipenghui commented on a change in pull request #3830: Fix read batching message by pulsar reader

[GitBox]

codelipenghui commented on a change in pull request #3830: Fix read batching 
message by pulsar reader
URL: https://github.com/apache/pulsar/pull/3830#discussion_r266190359
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
 ##
 @@ -1431,6 +1428,15 @@ public boolean hasMessageAvailable() throws 
PulsarClientException {
 return booleanFuture;
 }
 
+private boolean hasMoreMessages(MessageId lastMessageIdInBroker, MessageId 
lastDequeuedMessage) {
+return (lastMessageIdInBroker.compareTo(lastDequeuedMessage) > 0 ||
+// Make sure batch message can be read completely.
+lastDequeuedMessage instanceof BatchMessageIdImpl
+&& 
lastMessageIdInBroker.compareTo(lastDequeuedMessage) >= 0
+&& incomingMessages.size() > 0)
+&& ((MessageIdImpl)lastMessageIdInBroker).getEntryId() != -1;
 
 Review comment:
   @merlimat Thanks, more clearer. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 commented on issue #3752: revise the schema default type not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473502631
 
 
   run Integration Tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 commented on issue #3752: revise the schema default type not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473502619
 
 
   run cpp tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 removed a comment on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 removed a comment on issue #3752: revise the schema default type 
not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473501497
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 removed a comment on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 removed a comment on issue #3752: revise the schema default type 
not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473300193
 
 
   run java8 tests
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 commented on issue #3752: revise the schema default type not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473501497
 
 
   run java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] srkukarni merged pull request #3827: Expand add env functionality to add variables if not present

[GitBox]

srkukarni merged pull request #3827: Expand add env functionality to add 
variables if not present
URL: https://github.com/apache/pulsar/pull/3827
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: Expand add env functionality to add variables if not present (#3827)

[sanjeevrk]

This is an automated email from the ASF dual-hosted git repository.

sanjeevrk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new da2bb66  Expand add env functionality to add variables if not present 
(#3827)
da2bb66 is described below

commit da2bb660461345f11446635298c2fdd35955ac32
Author: Sanjeev Kulkarni 
AuthorDate: Fri Mar 15 22:07:11 2019 -0700

Expand add env functionality to add variables if not present (#3827)

* Add config variables if absent

* Took feedback into account
---
 docker/pulsar/scripts/apply-config-from-env.py | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/docker/pulsar/scripts/apply-config-from-env.py 
b/docker/pulsar/scripts/apply-config-from-env.py
index 0ad9b2b..94a8b4b 100755
--- a/docker/pulsar/scripts/apply-config-from-env.py
+++ b/docker/pulsar/scripts/apply-config-from-env.py
@@ -34,6 +34,9 @@ if len(sys.argv) < 2:
 # Always apply env config to env scripts as well
 conf_files = ['conf/pulsar_env.sh', 'conf/bkenv.sh'] + sys.argv[1:]
 
+PF_ENV_PREFIX = 'PULSAR_'
+
+
 for conf_filename in conf_files:
 lines = []  # List of config file lines
 keys = {} # Map a key to its line number in the file
@@ -56,6 +59,17 @@ for conf_filename in conf_files:
 idx = keys[k]
 lines[idx] = '%s=%s\n' % (k, v)
 
+
+# Add new keys from Env
+for k in sorted(os.environ.keys()):
+v = os.environ[k]
+if not k.startswith(PF_ENV_PREFIX):
+continue
+k = k[len(PF_ENV_PREFIX):]
+if k not in keys:
+print('[%s] Adding config %s = %s' % (conf_filename, k, v))
+lines.append('%s=%s\n' % (k, v))
+
 # Store back the updated config in the same file
 f = open(conf_filename, 'w')
 for line in lines:

[GitHub] [pulsar] srkukarni commented on issue #3827: Expand add env functionality to add variables if not present

[GitBox]

srkukarni commented on issue #3827: Expand add env functionality to add 
variables if not present
URL: https://github.com/apache/pulsar/pull/3827#issuecomment-473495130
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: Use correct number of messages in batch for publish rate stats during replication (#3834)

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new ce13251  Use correct number of messages in batch for publish rate 
stats during replication (#3834)
ce13251 is described below

commit ce13251caf8649f74e3f5196784eec24c062046c
Author: Matteo Merli 
AuthorDate: Fri Mar 15 18:28:47 2019 -0700

Use correct number of messages in batch for publish rate stats during 
replication (#3834)
---
 .../main/java/org/apache/pulsar/client/impl/ProducerImpl.java  | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
index 6dcaca0..0793a50 100644
--- 
a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
+++ 
b/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java
@@ -372,12 +372,18 @@ public class ProducerImpl extends ProducerBase 
implements TimerTask, Conne
 ByteBuf encryptedPayload = 
encryptMessage(msgMetadataBuilder, compressedPayload);
 
 MessageMetadata msgMetadata = msgMetadataBuilder.build();
-ByteBufPair cmd = sendMessage(producerId, sequenceId, 1, 
msgMetadata, encryptedPayload);
+
+// When publishing during replication, we need to set the 
correct number of message in batch
+// This is only used in tracking the publish rate stats
+int numMessages = 
msg.getMessageBuilder().hasNumMessagesInBatch()
+? msg.getMessageBuilder().getNumMessagesInBatch()
+: 1;
+ByteBufPair cmd = sendMessage(producerId, sequenceId, 
numMessages, msgMetadata, encryptedPayload);
 msgMetadataBuilder.recycle();
 msgMetadata.recycle();
 
 final OpSendMsg op = OpSendMsg.create(msg, cmd, 
sequenceId, callback);
-op.setNumMessagesInBatch(1);
+op.setNumMessagesInBatch(numMessages);
 op.setBatchSizeByte(encryptedPayload.readableBytes());
 pendingMessages.put(op);
 lastSendFuture = callback.getFuture();

[GitHub] [pulsar] merlimat merged pull request #3834: Use correct number of messages in batch for publish rate stats during replication

[GitBox]

merlimat merged pull request #3834: Use correct number of messages in batch for 
publish rate stats during replication
URL: https://github.com/apache/pulsar/pull/3834
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] tcbuzor commented on issue #3832: io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 5242880: 369295620 - discarded

[GitBox]

tcbuzor commented on issue #3832: io.netty.handler.codec.TooLongFrameException: 
Adjusted frame length exceeds 5242880: 369295620 - discarded
URL: https://github.com/apache/pulsar/issues/3832#issuecomment-473468295
 
 
   I got the same exception with the url option. B/W, I already added the 
following to the CLI:
   webServiceUrl=https://broker.example.com:8443/
   brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
   useTls=true
   tlsAllowInsecureConnection=false
   tlsTrustCertsFilePath=/path/to/ca.cert.pem
   tlsEnableHostnameVerification=false


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rdhabalia commented on a change in pull request #3786: [pulsar-function] fix deadlock in pulsar-source

[GitBox]

rdhabalia commented on a change in pull request #3786: [pulsar-function] fix 
deadlock in pulsar-source
URL: https://github.com/apache/pulsar/pull/3786#discussion_r266169774
 
 

 ##
 File path: 
pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/source/PulsarSource.java
 ##
 @@ -100,7 +98,15 @@ public void open(Map config, SourceContext 
sourceContext) throws
 }
 
 return cb.subscribeAsync();
-
}).collect(Collectors.toList()).stream().map(CompletableFuture::join).collect(Collectors.toList());
+}).collect(Collectors.toList()).stream().map(consumerFuture -> {
+try {
+return consumerFuture.get(consumerOpTimeoutMs, 
TimeUnit.MILLISECONDS);
 
 Review comment:
   yes..good point..fixed it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] merlimat opened a new pull request #3834: Use correct number of messages in batch for publish rate stats during replication

[GitBox]

merlimat opened a new pull request #3834: Use correct number of messages in 
batch for publish rate stats during replication
URL: https://github.com/apache/pulsar/pull/3834
 
 
   ### Motivation
   
   When replicator is repackaging the messages for replication, it's always 
passing 1 for messages count in the `CommandSend` header. That header is used 
to compute the publish rate stats in the target cluster. It should be set to 
the actual number of messages in the batch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch master updated: Fix topic name logic for partitioned topics (#3693)

[jerrypeng]

This is an automated email from the ASF dual-hosted git repository.

jerrypeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
 new fdaa9e3  Fix topic name logic for partitioned topics (#3693)
fdaa9e3 is described below

commit fdaa9e3728e463bc67f5e946833d1dac392412e2
Author: Sanjeev Kulkarni 
AuthorDate: Fri Mar 15 15:05:53 2019 -0700

Fix topic name logic for partitioned topics (#3693)

* Since partitioned topics have a -partition- affixed to the 
topic name,
when doing explicit acking, check for the case to determine the right topic 
name

* added unittests
---
 .../instance/src/main/python/contextimpl.py| 21 +++--
 .../src/scripts/run_python_instance_tests.sh   |  3 ++-
 .../src/test/python/test_python_instance.py| 22 +-
 3 files changed, 38 insertions(+), 8 deletions(-)

diff --git a/pulsar-functions/instance/src/main/python/contextimpl.py 
b/pulsar-functions/instance/src/main/python/contextimpl.py
index 638e64f..f3a9710 100644
--- a/pulsar-functions/instance/src/main/python/contextimpl.py
+++ b/pulsar-functions/instance/src/main/python/contextimpl.py
@@ -23,6 +23,7 @@
 """contextimpl.py: ContextImpl class that implements the Context interface
 """
 
+import re
 import time
 import os
 import json
@@ -54,7 +55,6 @@ class ContextImpl(pulsar.Context):
 self.publish_producers = {}
 self.publish_serializers = {}
 self.message = None
-self.current_input_topic_name = None
 self.current_start_time = None
 self.user_config = json.loads(instance_config.function_details.userConfig) 
\
   if instance_config.function_details.userConfig \
@@ -73,7 +73,6 @@ class ContextImpl(pulsar.Context):
   # Called on a per message basis to set the context for the current message
   def set_current_message_context(self, message, topic):
 self.message = message
-self.current_input_topic_name = topic
 self.current_start_time = time.time()
 
   def get_message_id(self):
@@ -89,7 +88,7 @@ class ContextImpl(pulsar.Context):
 return self.message.properties()
 
   def get_current_message_topic_name(self):
-return self.current_input_topic_name
+return self.message.topic_name()
 
   def get_function_name(self):
 return self.instance_config.function_details.name
@@ -176,9 +175,19 @@ class ContextImpl(pulsar.Context):
 self.publish_producers[topic_name].send_async(output_bytes, 
partial(self.callback_wrapper, callback, topic_name, self.get_message_id()), 
properties=properties)
 
   def ack(self, msgid, topic):
-if topic not in self.consumers:
-  raise ValueError('Invalid topicname %s' % topic)
-self.consumers[topic].acknowledge(msgid)
+topic_consumer = None
+if topic in self.consumers:
+  topic_consumer = self.consumers[topic]
+else:
+  # if this topic is a partitioned topic
+  m = re.search('(.+)-partition-(\d+)', topic)
+  if not m:
+raise ValueError('Invalid topicname %s' % topic)
+  elif m.group(1) in self.consumers:
+topic_consumer = self.consumers[m.group(1)]
+  else:
+raise ValueError('Invalid topicname %s' % topic)
+topic_consumer.acknowledge(msgid)
 
   def get_and_reset_metrics(self):
 metrics = self.get_metrics()
diff --git a/pulsar-functions/instance/src/scripts/run_python_instance_tests.sh 
b/pulsar-functions/instance/src/scripts/run_python_instance_tests.sh
index 9b33c24..7005b9b 100644
--- a/pulsar-functions/instance/src/scripts/run_python_instance_tests.sh
+++ b/pulsar-functions/instance/src/scripts/run_python_instance_tests.sh
@@ -22,9 +22,10 @@
 # Make sure dependencies are installed
 pip install mock --user
 pip install protobuf --user
+pip install fastavro --user
 
 CUR_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
 PULSAR_HOME=$CUR_DIR/../../../../
 
 # run instance tests
-PULSAR_HOME=${PULSAR_HOME} 
PYTHONPATH=${PULSAR_HOME}/pulsar-functions/instance/target/python-instance 
python -m unittest discover -v 
${PULSAR_HOME}/pulsar-functions/instance/target/python-instance/tests
\ No newline at end of file
+PULSAR_HOME=${PULSAR_HOME} 
PYTHONPATH=${PULSAR_HOME}/pulsar-functions/instance/target/python-instance 
python -m unittest discover -v 
${PULSAR_HOME}/pulsar-functions/instance/target/python-instance/tests
diff --git a/pulsar-functions/instance/src/test/python/test_python_instance.py 
b/pulsar-functions/instance/src/test/python/test_python_instance.py
index 0071e2f..8b92fa8 100644
--- a/pulsar-functions/instance/src/test/python/test_python_instance.py
+++ b/pulsar-functions/instance/src/test/python/test_python_instance.py
@@ -20,9 +20,12 @@
 
 # DEPENDENCIES:  unittest2,mock
 
+from mock import Mock
+import sys
+sys.modules['prometheus_client'] = Mock()
+
 from contextimpl import ContextImpl
 from python_instance import InstanceConfig
-from mock import Mock
 fr

[GitHub] [pulsar] jerrypeng merged pull request #3693: Fix topic name logic for partitioned topics

[GitBox]

jerrypeng merged pull request #3693: Fix topic name logic for partitioned topics
URL: https://github.com/apache/pulsar/pull/3693
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] merlimat commented on a change in pull request #3830: Fix read batching message by pulsar reader

[GitBox]

merlimat commented on a change in pull request #3830: Fix read batching message 
by pulsar reader
URL: https://github.com/apache/pulsar/pull/3830#discussion_r266159040
 
 

 ##
 File path: 
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java
 ##
 @@ -1431,6 +1428,15 @@ public boolean hasMessageAvailable() throws 
PulsarClientException {
 return booleanFuture;
 }
 
+private boolean hasMoreMessages(MessageId lastMessageIdInBroker, MessageId 
lastDequeuedMessage) {
+return (lastMessageIdInBroker.compareTo(lastDequeuedMessage) > 0 ||
+// Make sure batch message can be read completely.
+lastDequeuedMessage instanceof BatchMessageIdImpl
+&& 
lastMessageIdInBroker.compareTo(lastDequeuedMessage) >= 0
+&& incomingMessages.size() > 0)
+&& ((MessageIdImpl)lastMessageIdInBroker).getEntryId() != -1;
 
 Review comment:
   I think the condition is not correct in current form. I believe it should be 
like : 
   
   ```java
   if (lastMessageIdInBroker.compareTo(lastDequeuedMessage) > 0 
   ((MessageIdImpl)lastMessageIdInBroker).getEntryId() != -1) {
   return true;
   } else if (lastMessageIdInBroker.compareTo(lastDequeuedMessage) == 0
  && incomingMessages.size() > 0 ) {
return true;
   } else {
  return false;
   }
   ``` 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on issue #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on issue #3735: Implementing authentication for Pulsar 
Functions
URL: https://github.com/apache/pulsar/pull/3735#issuecomment-473453206
 
 
   @ivankelly you are correct service accounts are not needed to mount secrets 
in pods.  I have remove the service account related code


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266157897
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/ClearTextFunctionTokenAuthProvider.java
 ##
 @@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.client.impl.auth.AuthenticationToken;
+import org.apache.pulsar.functions.instance.AuthenticationConfig;
+
+import java.util.Optional;
+
+import static 
org.apache.pulsar.broker.authentication.AuthenticationProviderToken.getToken;
 
 Review comment:
   @sijie I think pulsar-function-runtime is going to have to depend on 
pulsar-broker-common for the time being since  AuthenticationDataSource 
interface in part of pulsar-broker-common and pulsar-function-worker already 
depends on pulsar-broker-common anyways


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266155384
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/ClearTextFunctionTokenAuthProvider.java
 ##
 @@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.client.impl.auth.AuthenticationToken;
+import org.apache.pulsar.functions.instance.AuthenticationConfig;
+
+import java.util.Optional;
+
+import static 
org.apache.pulsar.broker.authentication.AuthenticationProviderToken.getToken;
 
 Review comment:
   It would also be a big move since AuthenticationDataSource would also need 
to move the pulsar-common and all the things that AuthenticationDataSource 
depends on. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266153604
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/ClearTextFunctionTokenAuthProvider.java
 ##
 @@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.client.impl.auth.AuthenticationToken;
+import org.apache.pulsar.functions.instance.AuthenticationConfig;
+
+import java.util.Optional;
+
+import static 
org.apache.pulsar.broker.authentication.AuthenticationProviderToken.getToken;
 
 Review comment:
   @ivankelly its kind of messy extracting the getToken method from 
AuthenticationProviderToken since it relies on the on many package local String 
declared in the class.  
   
   I think the cleanest way is just duplicate some of the code


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rdhabalia removed a comment on issue #3818: [pulsar-common] add open Concurrent LongPair RangeSet

[GitBox]

rdhabalia removed a comment on issue #3818: [pulsar-common] add open Concurrent 
LongPair RangeSet
URL: https://github.com/apache/pulsar/pull/3818#issuecomment-473435018
 
 
   rerun integration tests
   rerun java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rdhabalia commented on issue #3818: [pulsar-common] add open Concurrent LongPair RangeSet

[GitBox]

rdhabalia commented on issue #3818: [pulsar-common] add open Concurrent 
LongPair RangeSet
URL: https://github.com/apache/pulsar/pull/3818#issuecomment-473435018
 
 
   rerun integration tests
   rerun java8 tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rdhabalia commented on issue #3818: [pulsar-common] add open Concurrent LongPair RangeSet

[GitBox]

rdhabalia commented on issue #3818: [pulsar-common] add open Concurrent 
LongPair RangeSet
URL: https://github.com/apache/pulsar/pull/3818#issuecomment-473409151
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] rdhabalia removed a comment on issue #3818: [pulsar-common] add open Concurrent LongPair RangeSet

[GitBox]

rdhabalia removed a comment on issue #3818: [pulsar-common] add open Concurrent 
LongPair RangeSet
URL: https://github.com/apache/pulsar/pull/3818#issuecomment-473409151
 
 
   retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] geyrix57 opened a new issue #3833: Error offloading:

[GitBox]

geyrix57 opened a new issue #3833: Error offloading:
URL: https://github.com/apache/pulsar/issues/3833
 
 
   I'm getting this error when trying to manually offload a topic to s3 bucket.
   
    Broker config
   managedLedgerOffloadDriver=aws-s3
   s3ManagedLedgerOffloadRegion=us-west-2
   s3ManagedLedgerOffloadBucket=mybucket
   s3ManagedLedgerOffloadServiceEndpoint=https://mybucket.blabla.blabla.com/
   
    Error
   Reason: Error offloading: 
org.apache.bookkeeper.mledger.ManagedLedgerException: 
java.util.concurrent.CompletionException: org.jclouds.aws.AWSResponseException: 
request POST 
https://mybucket.blabla.blabla.com/mybucket/3aab7c02-81ab-470b-badb-7e4e5c74bb2d-ledger-19?uploads
 HTTP/1.1 failed with code 400, error: AWSError{requestId='', 
requestToken='', code='AuthorizationHeaderMalformed', message='The 
authorization header is malformed; the region 'us-east-1' is wrong; expecting 
'us-west-2'', context='{Region=us-west-2, HostId=}'}
   
   It doesn't matter what region a use in s3ManagedLedgerOffloadRegion I'm 
always getting the same error. (the right region is us-west-2 as shown on the 
config)  
   
    System configuration
   **Pulsar version**: 2.3.0
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on issue #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on issue #3735: Implementing authentication for Pulsar 
Functions
URL: https://github.com/apache/pulsar/pull/3735#issuecomment-473397683
 
 
   @jerrypeng 
   > I don't believe that is the correct design choice because now the runtime 
is making assumptions on what the data in FunctionAuthData is. 
   
   What you are doing now in the case of KubernetesFunctionAuthProvider is 
premature generalization. We only have one use case for it, and that's very 
clear in what methods are in the interface. Something for vault with dynamic 
tokens (i think i figured out the workload for this btw) would need something 
very different. 
   
   Generalization for this interface doesn't make sense even. The AuthProvider 
interface is a way for the runtime to allow common rest code code to pass 
information to the common instance code. But between these two points, 
everything is k8s specific. So we should treat it as such.
   
   > The current implementation KubernetesSecretsTokenAuthProvider stores the 
name of the secret (pointer to it) in the data field of FunctionAuthData. That 
is an workflow establish in KubernetesSecretsTokenAuthProvider because we cache 
the name via cacheAuthData in the function metadata topic. How can we make that 
assumption for another FunctionAuthProvider impl?
   
   We can make this assumption for all k8s based auth providers, because 
secrets is the way to pass around sensitive information in k8s.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266102584
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/FunctionAuthProvider.java
 ##
 @@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.functions.instance.AuthenticationConfig;
+
+import java.util.Optional;
+
+/**
+ * This is a generic interface that functions can use to cache and distribute 
appropriate authentication
+ * data that is needed to configure the runtime of functions to support 
appropriate authentication of function instances
+ */
+public interface FunctionAuthProvider {
+
+/**
+ * Set authentication configs for function instance based on the data in 
FunctionAuthenticationSpec
+ * @param authConfig authentication configs passed to the function instance
+ * @param functionAuthData function authentication data that is provider 
specific
+ */
+void configureAuthenticationConfig(AuthenticationConfig authConfig, 
FunctionAuthData functionAuthData);
 
 Review comment:
   sure I can do that but the reason I named the method 
```configureAuthenticationConfig ``` is because we are literally configuring 
```AuthenticationConfig``` based on ```FunctionAuthData```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266099274
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/KubernetesRuntimeFactory.java
 ##
 @@ -181,6 +193,12 @@ public KubernetesRuntime createContainer(InstanceConfig 
instanceConfig, String c
 default:
 throw new RuntimeException("Unsupported Runtime " + 
instanceConfig.getFunctionDetails().getRuntime());
 }
+
+// adjust the auth config to support auth
+if (instanceConfig.getFunctionAuthenticationSpec() != null) {
+getAuthProvider().configureAuthenticationConfig(authConfig, 
getFunctionAuthData(instanceConfig.getFunctionAuthenticationSpec()));
 
 Review comment:
   This call is to configure the authConfig appropriately to support auth 
because it will get passed into KubernetesRuntime and eventually in the the 
function instance. And the fields in the authConfig is what the function 
instance or the pulsar client in the instance will use for authentication


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on issue #3735: Implementing authentication for Pulsar Functions

[GitBox]

jerrypeng commented on issue #3735: Implementing authentication for Pulsar 
Functions
URL: https://github.com/apache/pulsar/pull/3735#issuecomment-473393406
 
 
   @ivankelly responding to your comments
   
   > First of all, service accounts are not needed, you can attached the secret 
directly to the stateful set. In fact, thats exactly what you are doing. 
service accounts are for pods to authenticate with k8s services. We're not 
doing that, and we don't need to do that for this implementation. It may be 
needed for vault at a later stage, but let's not make assumptions about how 
that'll be used until we have concretely worked out the flow for that.
   
   let me experiment with not having a service account and see if that works
   
   > Secondly, i don't think we need an interface for attaching the secret to 
the stateful set. Whatever the auth data is that we are passing in, we should 
assume it is secret, so if there is auth data, the k8s runtime should attach 
that as a secret. Then it is up to configureAuthenticationConfig to know how to 
do something with that authData. So we should move the attachment of the secret 
volume and the mount into the k8s runtime itself.
   
   I don't believe that is the correct design choice because now the runtime is 
making assumptions on what the data in FunctionAuthData is.  The current 
implementation ```KubernetesSecretsTokenAuthProvider``` stores the name of the 
secret (pointer to it) in the data field of FunctionAuthData.  That is an 
workflow establish in ```KubernetesSecretsTokenAuthProvider``` because we cache 
the name via ```cacheAuthData``` in the function metadata topic.  How can we 
make that assumption for another FunctionAuthProvider impl?  I could have 
implemented another version of a Kubernetes Function Authentication Provider 
that puts the actual token in the data field of FunctionAuthData and set it 
explicitly as a env variable in the pod spec.  While that approach might not be 
secure, I would like implementations to have that flexibility and not to pigeon 
hole implementation to a certain workflow.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] merlimat commented on issue #3832: io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 5242880: 369295620 - discarded

[GitBox]

merlimat commented on issue #3832: 
io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 
5242880: 369295620 - discarded
URL: https://github.com/apache/pulsar/issues/3832#issuecomment-473391618
 
 
   @tcbuzor this looks like a mismatch between client and server on whether TLS 
is enabled (eg: connecting with TLS on a non-TLS port or vice-versa)
   
   the regular client command (`pulsar-client -n 0 -s "TEST" TEST_TOPIC`) will 
connector to `pulsar://localhost:6650` without TLS.
   
   Use `pulsar-client --url pulsar+ssl://localhost:6651 ... ` to have the 
client use TLS when connecting.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] tcbuzor opened a new issue #3832: io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 5242880: 369295620 - discarded

[GitBox]

tcbuzor opened a new issue #3832: io.netty.handler.codec.TooLongFrameException: 
Adjusted frame length exceeds 5242880: 369295620 - discarded
URL: https://github.com/apache/pulsar/issues/3832
 
 
   **Describe the bug**
   I installed Apache pulsar 2.3.0 and set up TLS by following the steps 
described in https://pulsar.apache.org/docs/en/security-tls-transport/ 
   When I use the client in the pulsar distribution, I get the 
TooLonFrameException
   
   
   11:36:56.866 [pulsar-io-49-3] INFO  
org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - 
[public/functions/persistent/coordinate-participants] Rewind from 120:0 to 120:0
   11:37:01.309 [pulsar-io-49-4] INFO  
org.apache.pulsar.broker.service.ServerCnx - New connection from 
/10.107.48.30:43 
   11:37:01.329 [pulsar-io-49-4] WARN  
org.apache.pulsar.broker.service.ServerCnx - [IP Address:43754] Got exception 
TooLongFrameException : Adjusted frame length exceeds 5242880: 369295620 - 
discarded
   io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 
5242880: 369295620 - discarded
   at 
io.netty.handler.codec.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:522)
 ~[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
 
11:37:01.334 [pulsar-io-49-4] INFO  
org.apache.pulsar.broker.service.ServerCnx - Closed connection from 
/10.107.48.30:43754 
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. Download the Apache pulsar binary from 
https://pulsar.apache.org/en/download/
   Current version (Stable) 2.3.0
   2. Unzip the binary and follow the steps for setting up TLS with self signed 
certificate
   3. launch pulsar in standalone mode - ./pulsar standalone
   4. start pulsar consumer ./pulsar-client -n 0 -s "TEST" TEST_TOPIC
   
   **Expected behavior**
   I expect the consumer to start listening for messages on TEST_TOPIC and the 
producer should be able to send messages to the TEST_TOPIC
   
   **Screenshots**
   If applicable, add screenshots to help explain your problem.
   
   **Desktop (please complete the following information):**
- OS: Red Hat Enterprise Linux 7
   
   **Additional context**
   Add any other context about the problem here.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] srkukarni commented on issue #3827: Expand add env functionality to add variables if not present

[GitBox]

srkukarni commented on issue #3827: Expand add env functionality to add 
variables if not present
URL: https://github.com/apache/pulsar/pull/3827#issuecomment-473375339
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] srkukarni commented on issue #3827: Expand add env functionality to add variables if not present

[GitBox]

srkukarni commented on issue #3827: Expand add env functionality to add 
variables if not present
URL: https://github.com/apache/pulsar/pull/3827#issuecomment-473353067
 
 
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266050433
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/ClearTextFunctionTokenAuthProvider.java
 ##
 @@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.client.impl.auth.AuthenticationToken;
+import org.apache.pulsar.functions.instance.AuthenticationConfig;
+
+import java.util.Optional;
+
+import static 
org.apache.pulsar.broker.authentication.AuthenticationProviderToken.getToken;
 
 Review comment:
   ya, break it into common


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266040261
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/FunctionAuthProvider.java
 ##
 @@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import org.apache.pulsar.broker.authentication.AuthenticationDataSource;
+import org.apache.pulsar.functions.instance.AuthenticationConfig;
+
+import java.util.Optional;
+
+/**
+ * This is a generic interface that functions can use to cache and distribute 
appropriate authentication
+ * data that is needed to configure the runtime of functions to support 
appropriate authentication of function instances
+ */
+public interface FunctionAuthProvider {
+
+/**
+ * Set authentication configs for function instance based on the data in 
FunctionAuthenticationSpec
+ * @param authConfig authentication configs passed to the function instance
+ * @param functionAuthData function authentication data that is provider 
specific
+ */
+void configureAuthenticationConfig(AuthenticationConfig authConfig, 
FunctionAuthData functionAuthData);
 
 Review comment:
   configureAuthenticationConfig sounds weird, why not just configureAuth?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266050013
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/FunctionAuthData.java
 ##
 @@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import lombok.Builder;
+import lombok.Data;
+
+@Data
+@Builder
+/**
+ * A wrapper for authentication data for functions
+ */
+public class FunctionAuthData {
+private byte[] data;
 
 Review comment:
   Add a string member here to hold which functions auth provider created the 
auth data. this can be used by the k8s runtime provider at least validate the 
auth data is of the type expected. I know this will be configured at a cluster 
level, but people screw up configuration all the time. Better to be defensive.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266038911
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/KubernetesRuntimeFactory.java
 ##
 @@ -181,6 +193,12 @@ public KubernetesRuntime createContainer(InstanceConfig 
instanceConfig, String c
 default:
 throw new RuntimeException("Unsupported Runtime " + 
instanceConfig.getFunctionDetails().getRuntime());
 }
+
+// adjust the auth config to support auth
+if (instanceConfig.getFunctionAuthenticationSpec() != null) {
+getAuthProvider().configureAuthenticationConfig(authConfig, 
getFunctionAuthData(instanceConfig.getFunctionAuthenticationSpec()));
 
 Review comment:
   I don't understand what this call is for?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on issue #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on issue #3735: Implementing authentication for Pulsar 
Functions
URL: https://github.com/apache/pulsar/pull/3735#issuecomment-473344350
 
 
   @jerrypeng 
   
   So mapping out the interactions here, yields the following sequence diagram.
   
![image](https://user-images.githubusercontent.com/54955/54443615-3fc78f00-4741-11e9-90b9-c0370ca0112a.png)
   
   I think this can be similifed a fair bit. 
   
   First of all, service accounts are not needed, you can attached the secret 
directly to the stateful set. In fact, thats exactly what you are doing. 
service accounts are for pods to authenticate with k8s services. We're not 
doing that, and we don't need to do that for this implementation. It may be 
needed for vault at a later stage, but let's not make assumptions about how 
that'll be used until we have concretely worked out the flow for that.
   
   Secondly, i don't think we need an interface for attaching the secret to the 
stateful set. Whatever the auth data is that we are passing in, we should 
assume it is secret, so if there is auth data, the **k8s runtime** should 
attach that as a secret. Then it is up to configureAuthenticationConfig to know 
how to do something with that authData. So we should move the attachment of the 
secret volume and the mount into the k8s runtime itself.
   
   so the resulting sequence would look like
   
![image](https://user-images.githubusercontent.com/54955/5803-c54c3e80-4743-11e9-9ae6-057a48651f31.png)
   
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on a change in pull request #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on a change in pull request #3735: Implementing 
authentication for Pulsar Functions
URL: https://github.com/apache/pulsar/pull/3735#discussion_r266037227
 
 

 ##
 File path: 
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/KubernetesFunctionAuthProvider.java
 ##
 @@ -0,0 +1,42 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.functions.auth;
+
+import io.kubernetes.client.models.V1ServiceAccount;
+import io.kubernetes.client.models.V1StatefulSet;
+
+/**
+ * Kubernetes runtime specific functions authentication provider
+ */
+public interface KubernetesFunctionAuthProvider extends FunctionAuthProvider {
 
 Review comment:
   > I think there is some misunderstanding here. 
   
   Ya, I needed to map out the interactions. See my top level comment.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #3830: Fix read batching message by pulsar reader

[GitBox]

sijie commented on issue #3830: Fix read batching message by pulsar reader
URL: https://github.com/apache/pulsar/pull/3830#issuecomment-473329689
 
 
   run java8 tests
   run integration tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on a change in pull request #3825: hide kafka-connecter details for easy use debezium connector

[GitBox]

sijie commented on a change in pull request #3825: hide kafka-connecter details 
for easy use debezium connector
URL: https://github.com/apache/pulsar/pull/3825#discussion_r266027748
 
 

 ##
 File path: 
pulsar-io/debezium/mysql/src/main/java/org/apache/pulsar/io/debezium/mysql/DebeziumMysqlSource.java
 ##
 @@ -34,42 +34,60 @@
  */
 @Slf4j
 public class DebeziumMysqlSource extends KafkaConnectSource {
+static private final String OFFSET_TOPIC_PREFIX = "mysql-offset-topic-";
+static private final String HISTORY_TOPIC_PREFIX = "mysql-history-topic-";
 
-static private String OFFSET_TOPIC_PREFIX = "mysql-offset-topic-";
-static private String HISTORY_TOPIC_PREFIX = "mysql-history-topic-";
+static private final String DEFAULT_TASK = 
"io.debezium.connector.mysql.MySqlConnectorTask";
+static private final String DEFAULT_CONVERTER = 
"org.apache.kafka.connect.json.JsonConverter";
+static private final String DEFAULT_HISTORY = 
"org.apache.pulsar.io.debezium.PulsarDatabaseHistory";
 
 private final String uuid = UUID.randomUUID().toString();
 
+private static void throwExceptionIfConfigNotMatch(Map 
config,
+   String key,
+   String expect) throws 
IllegalArgumentException {
 
 Review comment:
   nit: it should be called `value`. `expect` is a confusing name.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #3829: [Issue 3806] Fix NPE while call PartitionedProducerImpl.getStats()

[GitBox]

sijie commented on issue #3829: [Issue 3806] Fix NPE while call 
PartitionedProducerImpl.getStats()
URL: https://github.com/apache/pulsar/pull/3829#issuecomment-473326758
 
 
   >  I got stuck about the unit test, can you give me some advice? Thanks!
   
   you can just add a new test file `PartitionedProducerImplTest` and then test 
- if you construct a `PartitionedProducerImpl` using the zero arg constructor, 
it will not get NPE when accessing the variables.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ambition119 commented on issue #3831: [schema] support timestamp/date schema

[GitBox]

ambition119 commented on issue #3831: [schema] support timestamp/date schema
URL: https://github.com/apache/pulsar/issues/3831#issuecomment-473320786
 
 
   is java.sql.Date/Time/Time/Timestamp?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] ivankelly commented on issue #3735: Implementing authentication for Pulsar Functions

[GitBox]

ivankelly commented on issue #3735: Implementing authentication for Pulsar 
Functions
URL: https://github.com/apache/pulsar/pull/3735#issuecomment-473315125
 
 
   @jerrypeng So, in the sequence you have, the service account bit is entirely 
unnecessary. Rather than specifying a service account for the StatefulSet(?), 
you can just attach the secret as a volumeMount or as a env variable. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 commented on issue #3752: revise the schema default type not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473300193
 
 
   run java8 tests
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 commented on issue #3752: revise the schema default type not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473251615
 
 
   run Integration Tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 removed a comment on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 removed a comment on issue #3752: revise the schema default type 
not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473251549
 
 
   run cpp tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 commented on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 commented on issue #3752: revise the schema default type not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473251549
 
 
   run cpp tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] congbobo184 removed a comment on issue #3752: revise the schema default type not null

[GitBox]

congbobo184 removed a comment on issue #3752: revise the schema default type 
not null
URL: https://github.com/apache/pulsar/pull/3752#issuecomment-473251615
 
 
   run Integration Tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie opened a new issue #3831: [schema] support timestamp/date schema

[GitBox]

sijie opened a new issue #3831: [schema] support timestamp/date schema
URL: https://github.com/apache/pulsar/issues/3831
 
 
   **Is your feature request related to a problem? Please describe.**
   
   `timestamp` or `date` is a typical type supported in other data systems. we 
need to support `timestamp` or `date` type in pulsar. so we can add this in 
schema builder.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] codelipenghui opened a new pull request #3830: Fix read batching message by pulsar reader

[GitBox]

codelipenghui opened a new pull request #3830: Fix read batching message by 
pulsar reader
URL: https://github.com/apache/pulsar/pull/3830
 
 
   ### Motivation
   
   ```java
   ```
   
   ### Modifications
   
   *Describe the modifications you've done.*
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   
   *(Please pick either of the following options)*
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   *(or)*
   
   This change is already covered by existing tests, such as *(please describe 
tests)*.
   
   *(or)*
   
   This change added tests and can be verified as follows:
   
   *(example:)*
 - *Added integration tests for end-to-end deployment with large payloads 
(10MB)*
 - *Extended integration test for recovery after broker failure*
   
   ### Does this pull request potentially affect one of the following parts:
   
   *If `yes` was chosen, please highlight the changes*
   
 - Dependencies (does it add or upgrade a dependency): (yes / no)
 - The public API: (yes / no)
 - The schema: (yes / no / don't know)
 - The default values of configurations: (yes / no)
 - The wire protocol: (yes / no)
 - The rest endpoints: (yes / no)
 - The admin cli options: (yes / no)
 - Anything that affects deployment: (yes / no / don't know)
   
   ### Documentation
   
 - Does this pull request introduce a new feature? (yes / no)
 - If yes, how is the feature documented? (not applicable / docs / JavaDocs 
/ not documented)
 - If a feature is not applicable for documentation, explain why?
 - If a feature is not documented yet in this PR, please create a followup 
issue for adding the documentation
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on a change in pull request #3825: hide kafka-connecter details for easy use debezium connector

[GitBox]

jiazhai commented on a change in pull request #3825: hide kafka-connecter 
details for easy use debezium connector
URL: https://github.com/apache/pulsar/pull/3825#discussion_r265909742
 
 

 ##
 File path: pulsar-io/debezium/core/pom.xml
 ##
 @@ -0,0 +1,110 @@
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+  4.0.0
+  
+org.apache.pulsar
+pulsar-io-debezium
+2.4.0-SNAPSHOT
+  
+
+  pulsar-io-debezium-core
+  Pulsar IO :: Debezium :: Core
+
+  
+
+
+  ${project.groupId}
+  pulsar-io-core
+  ${project.version}
+
+
+
+  io.debezium
+  debezium-core
+  ${debezium.version}
 
 Review comment:
   this is a reference of debezium: io.debezium.debezium-core, not the one in 
pulsar.io.debezium.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #3807: Ledger Offload error when use S3 type

[GitBox]

sijie commented on issue #3807: Ledger Offload error when use S3 type
URL: https://github.com/apache/pulsar/issues/3807#issuecomment-473211686
 
 
   @jiazhai are you working on this bug?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] murong00 commented on issue #3829: [Issue 3806] Fix NPE while call PartitionedProducerImpl.getStats()

[GitBox]

murong00 commented on issue #3829: [Issue 3806] Fix NPE while call 
PartitionedProducerImpl.getStats()
URL: https://github.com/apache/pulsar/pull/3829#issuecomment-473210167
 
 
   @sijie @jiazhai Not so sure about this issue, by the way, also found in 
`ConsumerStatsRecorderImpl`, I got stuck about the unit test, can you give me 
some advice? Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jiazhai commented on a change in pull request #3825: hide kafka-connecter details for easy use debezium connector

[GitBox]

jiazhai commented on a change in pull request #3825: hide kafka-connecter 
details for easy use debezium connector
URL: https://github.com/apache/pulsar/pull/3825#discussion_r265895743
 
 

 ##
 File path: 
pulsar-io/debezium/mysql/src/main/java/org/apache/pulsar/io/debezium/mysql/DebeziumMysqlSource.java
 ##
 @@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.io.debezium.mysql;
+
+import java.util.Map;
+import java.util.UUID;
+
+import io.debezium.connector.mysql.MySqlConnectorConfig;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.kafka.connect.runtime.TaskConfig;
+import org.apache.pulsar.io.core.SourceContext;
+import org.apache.pulsar.io.debezium.PulsarDatabaseHistory;
+import org.apache.pulsar.io.kafka.connect.KafkaConnectSource;
+import org.apache.pulsar.io.kafka.connect.PulsarKafkaWorkerConfig;
+
+/**
+ * A pulsar source that runs
+ */
+@Slf4j
+public class DebeziumMysqlSource extends KafkaConnectSource {
+
+static private String OFFSET_TOPIC_PREFIX = "mysql-offset-topic-";
+static private String HISTORY_TOPIC_PREFIX = "mysql-history-topic-";
+
+private final String uuid = UUID.randomUUID().toString();
+
+
+@Override
+public void open(Map config, SourceContext sourceContext) 
throws Exception {
+config.put(
+TaskConfig.TASK_CLASS_CONFIG,
+"io.debezium.connector.mysql.MySqlConnectorTask");
+config.put(
+PulsarKafkaWorkerConfig.KEY_CONVERTER_CLASS_CONFIG,
+"org.apache.kafka.connect.json.JsonConverter");
+config.put(
+PulsarKafkaWorkerConfig.VALUE_CONVERTER_CLASS_CONFIG,
+"org.apache.kafka.connect.json.JsonConverter");
+
+// database.history
+config.put(
+MySqlConnectorConfig.DATABASE_HISTORY.name(),
 
 Review comment:
   mysql is different


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[pulsar] branch asf-site updated: Updated site at revision fff02e2

[mmerli]

This is an automated email from the ASF dual-hosted git repository.

mmerli pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new f77740f  Updated site at revision fff02e2
f77740f is described below

commit f77740f9229dc64b21ff7275d83f3cd6992aece1
Author: jenkins 
AuthorDate: Fri Mar 15 08:52:01 2019 +

Updated site at revision fff02e2
---
 content/docs/en/next/reference-configuration.html  |  2 ++
 .../en/next/reference-configuration/index.html |  2 ++
 content/docs/fr/next/reference-configuration.html  |  2 ++
 .../fr/next/reference-configuration/index.html |  2 ++
 content/docs/ja/next/reference-configuration.html  |  2 ++
 .../ja/next/reference-configuration/index.html |  2 ++
 .../docs/zh-CN/next/reference-configuration.html   |  2 ++
 .../zh-CN/next/reference-configuration/index.html  |  2 ++
 content/swagger/swagger.json   | 34 +++---
 9 files changed, 33 insertions(+), 17 deletions(-)

diff --git a/content/docs/en/next/reference-configuration.html 
b/content/docs/en/next/reference-configuration.html
index 1606691..95052c1 100644
--- a/content/docs/en/next/reference-configuration.html
+++ b/content/docs/en/next/reference-configuration.html
@@ -206,6 +206,7 @@
 tlsCiphersSpecify the tls cipher the broker will use to 
negotiate during TLS Handshake. Multiple values can be specified, separated by 
commas. Example:- 
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 tokenSecretKeyConfigure the secret key to be used to validate 
auth tokens. The key can be specified like: 
tokenSecretKey=data:base64,x or 
tokenSecretKey=file:///my/secret.key
 tokenPublicKeyConfigure the public key to be used to validate 
auth tokens. The key can be specified like: 
tokenPublicKey=data:base64,x or 
tokenPublicKey=file:///my/secret.key
+tokenAuthClaimSpecify which of the token's claims will be 
used as the authentication "principal" or "role". The 
default "sub" claim will be used if this is left 
blank
 maxUnackedMessagesPerConsumerMax number of unacknowledged 
messages allowed to receive messages by a consumer on a shared subscription. 
Broker will stop sending messages to consumer once, this limit reaches until 
consumer starts acknowledging messages back. Using a value of 0, is disabling 
unackeMessage limit check and consumer can receive messages without any 
restriction5
 maxUnackedMessagesPerSubscriptionMax number of unacknowledged 
messages allowed per shared subscription. Broker will stop dispatching messages 
to all consumers of the subscription once this limit reaches until consumer 
starts acknowledging messages back and unack count reaches to limit/2. Using a 
value of 0, is disabling unackedMessage-limit check and dispatcher can dispatch 
messages without any restriction20
 subscriptionRedeliveryTrackerEnabledEnable subscription 
message redelivery trackertrue
@@ -514,6 +515,7 @@
 tlsCiphersSpecify the tls cipher the broker will use to 
negotiate during TLS Handshake. Multiple values can be specified, separated by 
commas. Example:- 
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 tokenSecretKeyConfigure the secret key to be used to validate 
auth tokens. The key can be specified like: 
tokenSecretKey=data:base64,x or 
tokenSecretKey=file:///my/secret.key
 tokenPublicKeyConfigure the public key to be used to validate 
auth tokens. The key can be specified like: 
tokenPublicKey=data:base64,x or 
tokenPublicKey=file:///my/secret.key
+tokenAuthClaimSpecify the token claim that will be used as 
the authentication "principal" or "role". The 
"subject" field will be used if this is left blank
 
 
 

[GitHub] [pulsar] jiazhai commented on a change in pull request #3825: hide kafka-connecter details for easy use debezium connector

[GitBox]

jiazhai commented on a change in pull request #3825: hide kafka-connecter 
details for easy use debezium connector
URL: https://github.com/apache/pulsar/pull/3825#discussion_r265888276
 
 

 ##
 File path: 
pulsar-io/debezium/mysql/src/main/resources/debezium-mysql-source-config.yaml
 ##
 @@ -19,17 +19,13 @@
 
 tenant: "test"
 namespace: "test-namespace"
-name: "debezium-kafka-source"
-topicName: "kafka-connect-topic"
-archive: "connectors/pulsar-io-kafka-connect-adaptor-2.3.0-SNAPSHOT.nar"
+name: "debezium-mysql-source"
+topicName: "debezium-mysql-topic"
+archive: "connectors/pulsar-io-debezium-mysql-2.4.0-SNAPSHOT.nar"
 
 Review comment:
   If we use local run, we should give the archive.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] jerrypeng commented on issue #3828: Pulsar SQL is not able to read the latest message

[GitBox]

jerrypeng commented on issue #3828: Pulsar SQL is not able to read the latest 
message
URL: https://github.com/apache/pulsar/issues/3828#issuecomment-473179814
 
 
   @sijie I totally agree we should add something like this.  I was just 
presenting some previous thoughts we have had on this problem.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services