2019-04-17 11:59:56 UTC - Ramesh Bobba: @Ramesh Bobba has joined the channel
----
2019-04-17 12:59:23 UTC - Laurent Chriqui: Hello. I have tried running my
pulsar function on Mac Os X with the latest pulsar-client 2.3.1 and I get the
following error :
Traceback (most recent call last):
File
“/Users/laurentchriqui/Downloads/apache-pulsar-2.3.1/instances/python-instance/python_instance_main.py”,
line 36, in <module>
import pulsar
File
“/Users/laurentchriqui/Downloads/apache-pulsar-2.3.1/instances/python-instance/pulsar/__init__.py”,
line 101, in <module>
from _pulsar import Result, CompressionType, ConsumerType, InitialPosition,
PartitionsRoutingMode # noqa: F401
ImportError: cannot import name InitialPosition
Indeed, the class is not present in my _pulsar.cpython-37m-darwin.so
@Matteo Merli
----
2019-04-17 13:02:53 UTC - Matteo Merli: It’s seems like you have the older
client in your mac. Did you upgrade the client lib? `pip3 install
pulsar-client==2.3.1`
----
2019-04-17 13:04:43 UTC - Alexandre DUVAL: Hi, considering this quote, there is
a missing field on functions_worker.yml, something like tokenPublicKey?
----
2019-04-17 13:05:05 UTC - Alexandre DUVAL: And considering yaml, could it be
`tokenPublicKey: file:///pulsar/conf/keys/my-public.key` ?
----
2019-04-17 13:09:21 UTC - Alexandre DUVAL: here
----
2019-04-17 13:15:15 UTC - Alexandre DUVAL: or maybe i dont get the right
informations from the error logs :stuck_out_tongue:
----
2019-04-17 13:15:49 UTC - Laurent Chriqui: yes I did
----
2019-04-17 13:16:24 UTC - Laurent Chriqui: I think there may be a problem with
the compiled files for Mac Os
----
2019-04-17 13:19:06 UTC - Matteo Merli: definitely it’s missing the public
key.. though I’m not sure how to pass that in that yaml file
----
2019-04-17 13:19:08 UTC - Matteo Merli: :smile:
----
2019-04-17 13:19:32 UTC - Matteo Merli: let me try to pull them from PyPI
----
2019-04-17 13:21:17 UTC - Alexandre DUVAL: `tokenPublicKey:
file:///pulsar/conf/keys/my-public.key` this doesn't work, exactly the same
error
----
2019-04-17 13:21:25 UTC - Matteo Merli: Seems to work here:
```
$ pip3 install pulsar-client
Collecting pulsar-client
Downloading
<https://files.pythonhosted.org/packages/38/74/01281eedf5134fc26b9763611b364f90be36b2f3f6d601cf6e31ce988924/pulsar_client-2.3.1-cp37-cp37m-macosx_10_14_x86_64.whl>
(15.7MB)
100% |████████████████████████████████| 15.7MB 1.8MB/s
Requirement already satisfied: grpcio in
/usr/local/lib/python3.7/site-packages/grpcio-1.18.0rc1-py3.7-macosx-10.14-x86_64.egg
(from pulsar-client) (1.18.0rc1)
Requirement already satisfied: six in
/usr/local/Cellar/protobuf/3.6.1.3_1/libexec/lib/python3.7/site-packages (from
pulsar-client) (1.11.0)
Requirement already satisfied: ratelimit in
/usr/local/lib/python3.7/site-packages/ratelimit-2.2.1-py3.7.egg (from
pulsar-client) (2.2.1)
Requirement already satisfied: fastavro in
/usr/local/lib/python3.7/site-packages (from pulsar-client) (0.21.16)
Requirement already satisfied: protobuf>=3.6.1 in
/usr/local/Cellar/protobuf/3.6.1.3_1/libexec/lib/python3.7/site-packages (from
pulsar-client) (3.6.1)
Requirement already satisfied: apache-bookkeeper-client in
/usr/local/lib/python3.7/site-packages (from pulsar-client) (4.9.1)
Requirement already satisfied: prometheus-client in
/usr/local/lib/python3.7/site-packages/prometheus_client-0.5.0-py3.7.egg (from
pulsar-client) (0.5.0)
Requirement already satisfied: setuptools in
/usr/local/lib/python3.7/site-packages (from
protobuf>=3.6.1->pulsar-client) (40.8.0)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in
/usr/local/lib/python3.7/site-packages (from
apache-bookkeeper-client->pulsar-client) (2.21.0)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/site-packages
(from apache-bookkeeper-client->pulsar-client) (2019.1)
Requirement already satisfied: mmh3>=2.5.1 in
/usr/local/lib/python3.7/site-packages (from
apache-bookkeeper-client->pulsar-client) (2.5.1)
Requirement already satisfied: idna<2.9,>=2.5 in
/usr/local/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
/usr/local/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(3.0.4)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in
/usr/local/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(1.24.1)
Requirement already satisfied: certifi>=2017.4.17 in
/usr/local/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(2019.3.9)
Installing collected packages: pulsar-client
Successfully installed pulsar-client-2.3.1
[mmerli@pepito] ~ $ python3
Python 3.7.2 (default, Feb 12 2019, 08:15:36)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pulsar
>>>
```
----
2019-04-17 13:26:53 UTC - Laurent Chriqui: weird :
Looking in indexes: <https://pypi.org/simple>,
<http://ec2-54-77-49-109.eu-west-1.compute.amazonaws.com:6543/pypi/>
Collecting pulsar-client
Downloading
<https://files.pythonhosted.org/packages/38/74/01281eedf5134fc26b9763611b364f90be36b2f3f6d601cf6e31ce988924/pulsar_client-2.3.1-cp37-cp37m-macosx_10_14_x86_64.whl>
(15.7MB)
100% |████████████████████████████████| 15.7MB 25.8MB/s
Requirement already satisfied: apache-bookkeeper-client in
./venv/lib/python3.7/site-packages (from pulsar-client) (4.9.1)
Requirement already satisfied: protobuf>=3.6.1 in
./venv/lib/python3.7/site-packages (from pulsar-client) (3.7.1)
Requirement already satisfied: ratelimit in ./venv/lib/python3.7/site-packages
(from pulsar-client) (2.2.1)
Requirement already satisfied: grpcio in ./venv/lib/python3.7/site-packages
(from pulsar-client) (1.20.0)
Requirement already satisfied: six in ./venv/lib/python3.7/site-packages (from
pulsar-client) (1.12.0)
Requirement already satisfied: prometheus-client in
./venv/lib/python3.7/site-packages (from pulsar-client) (0.6.0)
Requirement already satisfied: fastavro in ./venv/lib/python3.7/site-packages
(from pulsar-client) (0.21.20)
Requirement already satisfied: mmh3>=2.5.1 in
./venv/lib/python3.7/site-packages (from
apache-bookkeeper-client->pulsar-client) (2.5.1)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in
./venv/lib/python3.7/site-packages (from
apache-bookkeeper-client->pulsar-client) (2.21.0)
Requirement already satisfied: pytz in ./venv/lib/python3.7/site-packages (from
apache-bookkeeper-client->pulsar-client) (2019.1)
Requirement already satisfied: setuptools>=34.0.0 in
./venv/lib/python3.7/site-packages/setuptools-39.1.0-py3.7.egg (from
apache-bookkeeper-client->pulsar-client) (39.1.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
./venv/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in
./venv/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(2019.3.9)
Requirement already satisfied: idna<2.9,>=2.5 in
./venv/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(2.8)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in
./venv/lib/python3.7/site-packages (from
requests<3.0.0dev,>=2.18.0->apache-bookkeeper-client->pulsar-client)
(1.24.1)
Installing collected packages: pulsar-client
Successfully installed pulsar-client-2.3.1
You are using pip version 10.0.1, however version 19.0.3 is available.
You should consider upgrading via the ‘pip install --upgrade pip’ command.
(venv) MacBook-Pro-de-Laurent:python-instance laurentchriqui$ python
Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43)
[Clang 6.0 (clang-600.0.57)] on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import pulsar
----
2019-04-17 13:27:04 UTC - Laurent Chriqui: Segmentation fault: 11
----
2019-04-17 13:39:49 UTC - Matteo Merli: That looks weird :confused:
----
2019-04-17 13:41:04 UTC - Matteo Merli: Can you double check a couple of things.
```
ulimit -c unlimited
python -v -c 'import pulsar'
```
----
2019-04-17 13:41:22 UTC - Matteo Merli: that should print from where it’s
trying to load it
----
2019-04-17 13:41:32 UTC - Matteo Merli: and leave a core file for the segfault
----
2019-04-17 13:42:57 UTC - Matteo Merli: then,
```
lldb `which python` -c /cores/core.XYZ
```
Should be able to load the core dump and `bt` should get a stack trace
----
2019-04-17 13:55:20 UTC - Alexandre DUVAL: @Matteo Merli who knows how to pass
that? :stuck_out_tongue:
----
2019-04-17 14:33:12 UTC - Kai Levy: @Alexandre DUVAL not positive, but try
passing it as `clientAuthenticationParameters:
"{\"tokenPublicKey\":\"<path>\"}"`
----
2019-04-17 14:35:18 UTC - Jianfeng Qiao: I start a broker by running
'bin/pulsar-daemon start broker' on a bare-metal server and use the default
configuration, I see the following messages outputted on the screen
'starting broker, logging to /opt/broker/logs/pulsar-broker-dfv.log', but why I
cannot find the log file under logs?
----
2019-04-17 14:35:18 UTC - Alexandre DUVAL: @Kai Levy <path> should
contains file:// as prefix?
----
2019-04-17 14:36:27 UTC - Kai Levy: Most likely? Or you could also try pasting
in the key raw
----
2019-04-17 14:40:02 UTC - Alexandre DUVAL: @Kai Levy im sure to understand, but
it's not clientauthparam, it's the tokenpublickey for functions_worker
----
2019-04-17 14:45:50 UTC - Kai Levy: I believe that the functions worker just
uses that one parameter and passes that along:
<https://github.com/apache/pulsar/blob/859c914a7a3f3ddb9f89c62eba3c048039bd852a/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/ThreadRuntimeFactory.java#L77>
----
2019-04-17 14:47:15 UTC - Laurent Chriqui: I get the following stack trace, I
don’t understand why it comes from a python2 file...
```Traceback (most recent call last):
File "<input>", line 1, in <module>
File
"/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy.py",
line 52, in <module>
import weakref
File
"/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/weakref.py",
line 14, in <module>
from _weakref import (
ImportError: cannot import name _remove_dead_weakref````
----
2019-04-17 14:49:27 UTC - Alexandre DUVAL: Yes, but it's not the
functions_worker as client of pulsar problem, it's related to the auth system
of the functions worker itself.
----
2019-04-17 14:49:45 UTC - Alexandre DUVAL: Something is missing on the
configuration of it:
----
2019-04-17 14:49:48 UTC - Alexandre DUVAL:
```############################################
# security settings for worker service
############################################
# Enforce authentication
authenticationEnabled: true
# Enforce authorization on accessing functions api
authorizationEnabled: true
# Set of autentication provider name list, which is a list of class names
authenticationProviders:
- org.apache.pulsar.broker.authentication.AuthenticationProviderToken
# Set of role names that are treated as "super-user", meaning they will be able
to access any admin-api
superUserRoles:
- yolo```
----
2019-04-17 14:53:01 UTC - Kai Levy: That's functions_worker.yml?
----
2019-04-17 14:53:57 UTC - Kai Levy: Look at this:
<https://github.com/apache/pulsar/blob/8b2929b3e5403fb44653976dc74be287666f7b96/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/AuthenticationConfig.java>
----
2019-04-17 14:57:24 UTC - Kai Levy: oh i think i understand what you're saying
----
2019-04-17 14:57:29 UTC - Kai Levy: nevermind
----
2019-04-17 14:58:11 UTC - Alexandre DUVAL: :slightly_smiling_face:
----
2019-04-17 15:33:57 UTC - Alexandre DUVAL: So anyone has an answer? :confused:
----
2019-04-17 15:36:42 UTC - Devin G. Bost: @Jerry Peng Any updates on this?
----
2019-04-17 15:39:52 UTC - Laurent Chriqui: I cleaned my python installations
and it seems to have fixed the problem, thanks for your help
:slightly_smiling_face:
----
2019-04-17 15:54:36 UTC - David Kjerrumgaard: @Alexandre DUVAL The stacktrace
indicates that the issue is on the broker,
`org.apache.pulsar.broker.PulsarServerException: Failed to load an
authentication provider.`. Have you configured the brokers to authenticate
clients?
----
2019-04-17 15:54:58 UTC - David Kjerrumgaard: by adding the following
properties to the broker.conf?
----
2019-04-17 15:55:04 UTC - David Kjerrumgaard:
----
2019-04-17 15:55:23 UTC - Alexandre DUVAL: yes, and i query the brokers with
auth without problems
----
2019-04-17 15:55:45 UTC - David Kjerrumgaard: which provider did you specify?
----
2019-04-17 15:56:10 UTC - Alexandre DUVAL:
`authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken`
----
2019-04-17 15:56:39 UTC - David Kjerrumgaard: `Caused by: java.io.IOException:
No secret key was provided for token authentication
at
org.apache.pulsar.broker.authentication.AuthenticationProviderToken.getValidationKey(AuthenticationProviderToken.java:135)
~[org.apache.pulsar-pulsar-broker-common-2.3.1.jar:2.3.1]
at `
----
2019-04-17 15:56:47 UTC - David Kjerrumgaard: did you specify the private key?
----
2019-04-17 15:57:19 UTC - David Kjerrumgaard:
`tokenSecretKey=file:///path/to/secret.key`
----
2019-04-17 15:57:25 UTC - Alexandre DUVAL: no, the public
----
2019-04-17 15:57:37 UTC - David Kjerrumgaard: you need the above in your
broker.conf
----
2019-04-17 15:57:47 UTC - Alexandre DUVAL: oh, ok
----
2019-04-17 16:02:09 UTC - Alexandre DUVAL: The private key and public key? Not
the secret, it's asym auth, no ?
----
2019-04-17 16:03:02 UTC - David Kjerrumgaard:
<http://pulsar.apache.org/docs/en/security-token-admin/>
----
2019-04-17 16:04:11 UTC - David Kjerrumgaard: How did you create your key pair?
----
2019-04-17 17:46:11 UTC - Jerry Peng: sorry for the late reply
----
2019-04-17 17:46:48 UTC - Jerry Peng: You just have to specify in the jar field
“builtin://<name of connector>”
----
2019-04-17 17:47:13 UTC - Jerry Peng: e.g. “<builtin://kafka>”
----
2019-04-17 18:03:03 UTC - Devin G. Bost: How do I specify the `sinkType` or
`sourceType` (for a kafka sink or source) when using the Java Admin API? I'm
not finding any helpful unit tests in the source code, and I don't see any
applicable properties on the `SinkConfig` or `SourceConfig` class objects that
are expected by `adminClient.source().createSource(..)`,
`adminClient.source().updateSource()`, `adminClient.sink().createSink(..)`, and
`adminClient.sink().updateSink(..)`.
I was able to locate the KafkaSourceConfig and KafkaSinkConfig classes, but
they don't have public properties like the SourceConfig and SinkConfig classes
(such as for `tenant`, `namespace`, etc.).
----
2019-04-17 18:03:53 UTC - Jerry Peng: @Devin G. Bost I replied in the thread we
were discussing on
----
2019-04-17 18:04:18 UTC - Devin G. Bost: Oh, sorry, I didn't see your response.
Thanks for getting back to me.
----
2019-04-17 18:08:16 UTC - Devin G. Bost: @Jerry Peng That's for the archive
parameter?
----
2019-04-17 18:12:55 UTC - Devin G. Bost: It worked!!!
----
2019-04-17 18:53:46 UTC - Adam P.: is there a way to watch pulsar logs for
functions being created? I'm trying to create a python function but keep
getting this error, unless i use the `localrun` mode
`Reason: javax.ws.rs.ProcessingException: Java heap space`
I'm not able to see anything go through the `logs/pulsar.log` or
`logs/pulsar-standalone.log`
----
2019-04-17 18:54:39 UTC - Ali Ahmed: is that from the pulasr client ?
----
2019-04-17 18:54:52 UTC - Devin G. Bost: @Jerry Peng When I run update on the
kafka sinks and sources, I get this 500 error:
```
org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException:
HTTP 500 Internal Server Error
at
org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:163)
at
org.apache.pulsar.client.admin.internal.SinkImpl.updateSink(SinkImpl.java:172)
at deployment.manifest.Sink.upsert(Sink.java:88)
at
deployment.manifest.Topology.lambda$upsertComponents$2(Topology.java:73)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at deployment.manifest.Topology.upsertComponents(Topology.java:73)
at
deployment.test.DeploymentTest.topology_components_upsertComponents(DeploymentTest.java:118)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.apache.pulsar.shade.javax.ws.rs.InternalServerErrorException:
HTTP 500 Internal Server Error
at
org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1098)
at
org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:883)
at
org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$1(JerseyInvocation.java:767)
at
org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:316)
at
org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:298)
at
org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:229)
at
org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:414)
at
org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:765)
at
org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:456)
at
org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$Builder.put(JerseyInvocation.java:340)
at
org.apache.pulsar.client.admin.internal.SinkImpl.updateSink(SinkImpl.java:170)
... 27 more```
----
2019-04-17 18:55:26 UTC - Adam P.: it's the `bin/pulsar-admin functions create
...` command.
----
2019-04-17 18:55:47 UTC - Ali Ahmed: you need to increase the heap size for the
pulsar-admin
----
2019-04-17 18:57:00 UTC - Adam P.: so, maybe i'm doing that wrong but when i
run this from docker-compose, i set these env vars to try to do that
```
- PULSAR_EXTRA_OPTS=-XX:+UnlockExperimentalVMOptions
- PULSAR_MEM=-Xms2g -Xmx4g -XX:MaxDirectMemorySize=6g```
am i missing something? i don't think i should be running out of space with
those numbers. my python package is 46M
----
2019-04-17 18:57:35 UTC - Thor Sigurjonsson: @Ali Ahmed Should that be in
conf/pulsar_tools_env.sh ?
----
2019-04-17 18:57:42 UTC - Ali Ahmed: yes
----
2019-04-17 18:58:20 UTC - Thor Sigurjonsson: @Adam Williams Might have been
doing that in conf/pulsar_env.sh instead of the tools one.
----
2019-04-17 18:59:44 UTC - Thor Sigurjonsson: Quick question: Is there a way to
change parallelism on a function once it's deployed without doing an update
with the `--jar` upload and all -- full redeploy.
----
2019-04-17 19:00:24 UTC - Matteo Merli: Yes, if you use the update command, you
only specify what to overwrite
----
2019-04-17 19:01:18 UTC - Thor Sigurjonsson: @Matteo Merli Very good, thanks!
----
2019-04-17 19:20:17 UTC - Jerry Peng: are there any errors in the broker logs?
----
2019-04-17 19:21:10 UTC - Devin G. Bost: Good question. I'll check.
----
2019-04-17 19:22:11 UTC - Devin G. Bost: If I'm running standalone, will the
broker logs output to the console?
----
2019-04-17 19:22:56 UTC - Jerry Peng: yes
----
2019-04-17 19:23:12 UTC - Devin G. Bost: I'll send you the output. There are
definitely errors.
----
2019-04-17 19:24:13 UTC - Devin G. Bost:
----
2019-04-17 19:25:56 UTC - Jerry Peng: taking a look
----
2019-04-17 19:26:08 UTC - Devin G. Bost: Thanks!
----
2019-04-17 19:34:16 UTC - Devin G. Bost: I'm wondering if it could be blowing
up at the `. . . ` in the `sasl.jaas.config` config.
----
2019-04-17 19:36:28 UTC - Kai Levy: Hey @Alexandre DUVAL I dug a little more
and it looks like the WorkerConfig never has it's properties set. So when the
auth plugin calls `getProperty` then it won't turn up anything.
----
2019-04-17 19:39:42 UTC - Devin G. Bost: That's not it. I tried passing in a
valid `sasl.jaas.config` string, and I still got the same error.
----
2019-04-17 19:39:58 UTC - Devin G. Bost: I only get the error with the kafka
sinks and sources.
----
2019-04-17 19:41:39 UTC - Jerry Peng: ya it does seem there is some thing wrong
when trying to deserialize the config. Let me investigate further
+1 : Devin G. Bost
----
2019-04-17 19:59:30 UTC - Devin G. Bost: I'm passing the configs as
`Map<String, Object>`.
For the log file that I sent you, here's what the `sinkConfig` values look like.
----
2019-04-17 20:00:47 UTC - Devin G. Bost: I tried using this as the
sasl.jaas.config value instead, but I still got the same errors:
`"com.sun.security.auth.module.Krb5LoginModule required doNotPrompt=true
useTicketCache=false serviceName=\"kafka\"
principal=\"[email protected]\" useKeyTab=true
keyTab=\"/pulsar/conf/auth/pulsar_runtime_test.keytab\" client=true;"`
----
2019-04-17 20:02:54 UTC - Jerry Peng: ok I can reproduce this error as well
----
2019-04-17 20:03:03 UTC - Jerry Peng: let me see what is wrong
----
2019-04-17 20:03:19 UTC - Jerry Peng: I think it has to do with nested jsons
because of the config
----
2019-04-17 20:03:30 UTC - Jerry Peng: but only happens on update
----
2019-04-17 20:04:08 UTC - Devin G. Bost: The fact that it only happens on
update is the thing that is most interesting to me.
----
2019-04-17 20:07:09 UTC - Jerry Peng: The error happens when trying to
deserialize the config json string that is stored internally via a protobuf
object.
----
2019-04-17 20:07:56 UTC - Jerry Peng: It happens on update because we retrieve
the old metadata for the function and merge it with the new one the user has
submitted
----
2019-04-17 20:08:21 UTC - Devin G. Bost: Oh, good to know.
----
2019-04-17 20:10:53 UTC - Adam Williams: I'm not sure I understand. Could you
provide more context?
----
2019-04-17 20:14:37 UTC - Jerry Peng: i know the issue. I will have a fix up
soon for this issue
----
2019-04-17 20:20:54 UTC - Devin G. Bost: Thanks!!
----
2019-04-17 21:46:19 UTC - Jerry Peng:
<https://github.com/apache/pulsar/pull/4067>
----
2019-04-17 21:46:21 UTC - Jerry Peng: is the fix
----
2019-04-17 22:21:28 UTC - Alexandre DUVAL: Okay
----
2019-04-17 23:09:25 UTC - Matteo Merli: I think the wrong handle was tagged :)
----
2019-04-18 08:30:04 UTC - Sébastien de Melo: Hello, we have a function that
sometimes stops working and cannot even be deleted. The other subcommands (get,
status, stats) respond though. Do you know why it happens and what we can do
to delete it?
----
2019-04-18 09:09:44 UTC - Alexandre DUVAL: Hi, if a Pulsar fonctions push
message to inexistent tenant/namespace/topic is it working? Or i have to create
it before ?
----
2019-04-18 09:10:41 UTC - Ali Ahmed: topics can be created on demand tenant and
namespace have to exist before hand
----
