2020-02-11 09:26:38 UTC - Eric Feliksik: I am running the standalone docker
version of Pulsar, using image `apachepulsar/pulsar:2.5.0`.
```tenant: public
namespace: default
name: exclaimer
jar:
/files/pulsar-functions/com.my.test.pulsarfunctions/target/pulsar-functions-0.2.jar
className: com.my.test.pulsarfunctions.exclaim.PostFixer
inputs:
- <persistent://public/default/incoming>
output: messages_exclaimed
logTopic: <persistent://public/default/my_logs>
parallellism: 1
userConfig:
postfix: '!!!'
autoAck: true```
My function implementation is:
```public class PostFixer implements Function<String, String> {
@Override
public String process(String input, Context context) throws
PulsarClientException {
if (input.equals("log")) {
System.out.println(">>>>>>>>>>>
logging on [stdout]");
context.getLogger().error(">>>>>>>>>>> logging
with [logger]");
context.newOutputMessage("my_logs",
Schema.STRING).value(">>>>>> logging explicitly to
topic").send();
throw new
RuntimeException(">>>>>>>>>>>> logging by
throwing [Exception]");
}
return input + "!!!";
}
}```
Now I would expect some logs to end up in the `my_logs` topic. However, Only
the “logging explicitly to topic” ends up there. The logging to stdout appears
in the docker-compose logs, and the logging via the context.getLogger() and the
Exception are completely untraceable.
How can I make sure that I find the thrown exceptions? I thought they should
end up in the `my_logs` topic; is that correct?
----
2020-02-11 10:07:27 UTC - Andres Garagiola: @Andres Garagiola has joined the
channel
----
2020-02-11 10:10:34 UTC - Younes: @Younes has joined the channel
----
2020-02-11 11:46:11 UTC - Yannis Weishaupt: @Yannis Weishaupt has joined the
channel
----
2020-02-11 12:45:39 UTC - Igor Vaisman: @Igor Vaisman has joined the channel
----
2020-02-11 12:58:12 UTC - Pradeep Mishra: i am getting this error when i run
Pulsar Broker (2.5.0) with websocket service on, i am not running it on
standalone mode
----
2020-02-11 12:58:14 UTC - Pradeep Mishra: ```09:10:31.170 [pulsar-web-32-4]
WARN org.apache.pulsar.websocket.ConsumerHandler - [10.1.1.194:53608] Failed
in creating subscription sub-checkpoint-logs on topic
<persistent://public/default/checkpoint_logs>
java.lang.IllegalArgumentException: Param serviceUrl must not be blank.```
----
2020-02-11 16:26:21 UTC - Yoav Cohen: Looks like this fixed the issue @Sijie
Guo and @Shahar Fermon, thank you!
----
2020-02-11 17:15:03 UTC - Naby: I am trying to remember if a python function as
Pulsar Function can depends on another custom module or it has to be a single
file. I’m having a hard time finding this fact on pulsar documents. Can anyone
help?
----
2020-02-11 19:19:30 UTC - Ming: Python code for function has to be in the same
file. If you have another Python library dependency, one option is you can pip
install in function's docker image because Python function worker is started
within a docker container.
----
2020-02-11 19:24:09 UTC - Ming: What is your wss or ws url? It might be just
the error code indicating `Param serviceUrl must not be blank.` It should look
like this
`wss://{pulsar-host-fqdn}:8001/ws/v2/consumer/persistent/public/default/foobar-topic/my-subscription`
----
2020-02-11 19:51:38 UTC - Ming: Essentially you are asking a high performance
cluster. ~10,000 msgs/sec with avg msg size of ~5MB is like 400gbps (50 GBytes
per second) You need run partitioned topic on multiple brokers so that to
spread network load cross multiple NIC. Data also hav eto be tripped across
multiple bookkeepers for disk I/O to keep up. This is not just Python limit.
The system has to be spec-ed properly.
----
2020-02-11 19:54:47 UTC - Ming: `maxMessageSize` should be tweaked over 5MB
which is your average. By default it is 5MB. You need shared subscription to
handle the rate, unless you already have a number of topics. So this is more
than a question of choice of Python. If it is designed properly, Python is not
a problem.
----
2020-02-11 19:56:13 UTC - Ming: Here are just a few concerns top of my mind.
There must be a few things I missed. @Aditya Vishwakarma
----
2020-02-11 20:00:49 UTC - Sijie Guo: @Pradeep Mishra there was a regression
regarding websocket in 2.5.0. I have fixed it in master. planning a 2.5.1
bugfix release soon.
----
2020-02-11 21:37:41 UTC - Kelvin Sajere: Apache pulsar 2.4.2 works fine for now
@Pradeep Mishra
----
2020-02-11 22:15:31 UTC - Jordan Pilat: @Jordan Pilat has joined the channel
----
2020-02-11 23:44:52 UTC - Sijie Guo: @Naby the python function support multiple
files via wheel and zip package. When the python function downloads a wheel
package, it will install the wheel package. The feature is already there since
2.3.0. It might not be well documented.
----
2020-02-12 00:11:57 UTC - rmb: Hi all, I'm trying to set up authentication via
JWT (I'm running pulsar in standalone mode). I've generated tokens and edited
conf/client.conf and conf/broker.conf as described at
<https://pulsar.apache.org/docs/en/security-jwt/>, but when I try to run
'bin/pulsar-admin namespaces grant-permission', it says
> Authorization is not enabled
>
> Reason: HTTP 501 Not Implemented
is there something else I have to configure?
----
2020-02-12 02:04:13 UTC - Sijie Guo: For stand-alone, you need to configure
conf/standalone.conf
----
2020-02-12 06:04:41 UTC - Abhilash Mandaliya: Hi
Is it mandatory to create a pulsar-io.yaml file under
src/main/resources/META-INF/services/
?
----
2020-02-12 06:05:16 UTC - Ali Ahmed: yes
----
2020-02-12 06:05:34 UTC - Ali Ahmed: it’s part of the packaging format
----
2020-02-12 06:08:21 UTC - Abhilash Mandaliya: ok, thanks :slightly_smiling_face:
But I didn’t find this anywhere in site documentation. Shouldn’t we highlight
this somewhere?
----
2020-02-12 06:14:33 UTC - Pradeep Mishra: Yes, it works for 2.4.2.
----
2020-02-12 06:15:12 UTC - Pradeep Mishra: Thanks @Sijie Guo
----
2020-02-12 06:18:45 UTC - Sijie Guo: ah, we should clarify this in
<http://pulsar.apache.org/docs/en/io-develop/> . @Abhilash Mandaliya do you
want to contribute a documentation change? /cc @Anonymitaet
----
2020-02-12 06:24:32 UTC - Abhilash Mandaliya: Okay. I would like to give it a
try.
----
2020-02-12 07:18:35 UTC - Abhilash Mandaliya: has anyone built connector using
gradle instead of maven?
or any good blog post on the same?
I am unable to create .nar file using gradle
----
2020-02-12 07:19:28 UTC - Abhilash Mandaliya: I am using the following
dependency in my build.gradle.kts
```compile("org.apache.nifi:nifi-nar-maven-plugin:1.3.1")```
----
2020-02-12 08:15:12 UTC - Ali Ahmed: you can’t use a maven plugin in gradle.
----
2020-02-12 08:15:41 UTC - Ali Ahmed: try one of the gradle nar plugins
<https://github.com/search?utf8=%E2%9C%93&q=gradle+nar&type=>
----
2020-02-12 08:39:02 UTC - rmb: any lines besides authenticationEnabled,
authorizationEnabled, and authenticationProviders? when I set those options,
pulsar just shuts down, before I have a chance to run any pulsar-admin commands
----
2020-02-12 08:50:36 UTC - Abhilash Mandaliya: thanks. it worked
:slightly_smiling_face:
----
2020-02-12 08:55:07 UTC - rmb: I'm having a new problem: if I try to run a
freshly unzipped 2.5.0 binary in docker, trying to run a pulsar-admin command
returns
> null
>
> Reason: javax.ws.rs.ProcessingException: Remotely closed
no matter how long I wait.
----
