2020-01-10 12:45:16 UTC - Eira S: Trying to connect to pulsar client using pulsar+ssl (using pulsar-client-go), I followed all the steps <https://pulsar.apache.org/docs/en/security-tls-authentication/> But I get this error: ```INFO[0000] Connecting to broker remote_addr="<pulsar+ssl://localhost:6651>" WARN[0000] Failed to connect to broker. error="x509: certificate signed by unknown authority" remote_addr="<pulsar+ssl://localhost:6651>" INFO[0000] Connection closed ``` ---- 2020-01-10 13:19:40 UTC - Fabien LD: Not an error, a warn because it is a self signed certificate ---- 2020-01-10 13:20:19 UTC - Fabien LD: It should work with that log in my understanding ---- 2020-01-10 16:38:27 UTC - Eira S: Thanks @Fabien LD . It still didn't connect with Broker, I used the certs provided with pulsar-client-go instead of using self-signed CA. Works for now! +1 : Fabien LD ---- 2020-01-10 17:38:02 UTC - Fernando: does anyone know if there’s a `kafkacat` equivalent for pulsar? ---- 2020-01-10 17:38:51 UTC - Jerry Peng: What does kafkacat do? Tail a topic? ---- 2020-01-10 17:39:15 UTC - Jerry Peng: If so you can use the pulsar-client cli to consume from a topic ---- 2020-01-10 17:39:21 UTC - Fernando: <https://github.com/edenhill/kafkacat> ---- 2020-01-10 17:39:47 UTC - Jerry Peng: Yup the equivalent is pulsar-client cli tool ---- 2020-01-10 17:40:13 UTC - Jerry Peng: Found in the bin/ folder ---- 2020-01-10 17:40:16 UTC - Fernando: this is `bin/pulsar-client`? ---- 2020-01-10 17:40:18 UTC - Fernando: ok cool ---- 2020-01-10 17:40:22 UTC - Jerry Peng: Yes ---- 2020-01-10 17:40:57 UTC - Jerry Peng: There are also examples of usage on the Apache pulsar website ---- 2020-01-10 17:44:41 UTC - Fernando: For future reference: <https://pulsar.apache.org/docs/en/reference-cli-tools/> ---- 2020-01-10 17:45:52 UTC - Fernando: Thanks! I ended up doing that ---- 2020-01-10 18:56:03 UTC - Eira S: pulsar-client-go: Is it possible to have two pulsar instances running on 2 different docker containers of same docker image? This is how I have two containers set up: ```bash-4.2# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8c3128d34f4d pulsar-client-go-test:latest "/pulsar/bin/pulsar …" 41 seconds ago Up 40 seconds 0.0.0.0:6650-6651->6650-6651/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8843/tcp pulsar-client-go-test f896860278a7 a698de63982b "/pulsar/bin/pulsar …" 8 minutes ago Up 8 minutes 0.0.0.0:7650-7651->7650-7651/tcp, 0.0.0.0:8081->8081/tcp, 0.0.0.0:8444->8844/tcp pulsar-client-go-2nd``` I get these errors on both docker containers while trying to produce messages using CLI
```18:54:38.080 [pulsar-client-io-1-1] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xbb0539f5, L:/127.0.0.1:36174 - R:localhost/127.0.0.1:6651] Received error from server: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available 18:54:38.083 [pulsar-client-io-1-1] ERROR org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [null] Failed to create producer: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available 18:54:38.084 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - Error while producing messages 18:54:38.084 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available``` ---- 2020-01-10 19:47:33 UTC - Piotr Jander: @Piotr Jander has joined the channel ---- 2020-01-10 20:17:41 UTC - Jordan Widdison: @Jordan Widdison has joined the channel ---- 2020-01-10 20:18:13 UTC - Daniel: @Daniel has joined the channel ---- 2020-01-10 20:21:44 UTC - Roman Popenov: so bastion is like the admin pod if I understand it correctly? ---- 2020-01-10 20:37:41 UTC - Jake: @Jake has joined the channel ---- 2020-01-10 21:08:57 UTC - Roman Popenov: I keep getting out of heap exception thrown in local run function ---- 2020-01-10 21:12:41 UTC - Roman Popenov: Is a function processing one message at a time in localrun or are there many threads that are spawn to try and process al the messages at once? ---- 2020-01-10 23:38:51 UTC - Jerry Peng: > Is a function processing one message at a time in localrun yes but the pulsar client might be running multiple threads ---- 2020-01-10 23:39:07 UTC - Jerry Peng: try increasing the direct memory set for the JVM ---- 2020-01-10 23:41:16 UTC - Roman Popenov: Hm… I have an IO Connectors, it is streaming an archived log file from S3, archived the file is roughly ~300 Mbs, and unarchived the log file is about 9 Gs ---- 2020-01-10 23:41:54 UTC - Roman Popenov: Basically I stream the archived file through a buffered reader and write a message that is a string of roughly 100 lines onto a topic. ---- 2020-01-10 23:42:40 UTC - Roman Popenov: It seems like Pulsar doesn’t have any issues reading and streaming it to a topic ---- 2020-01-10 23:43:09 UTC - Roman Popenov: But when I launch the function, which reads the message, splits it into 100 lines and does some ETL on each ---- 2020-01-10 23:43:22 UTC - Roman Popenov: and pushes it back to S3 ---- 2020-01-10 23:43:27 UTC - Roman Popenov: that is when the heap issue arrives ---- 2020-01-10 23:45:18 UTC - Roman Popenov: really weird ---- 2020-01-10 23:48:17 UTC - Jerry Peng: Can do a heap dump? ---- 2020-01-10 23:50:48 UTC - Jerry Peng: how are you starting localrun? ---- 2020-01-10 23:51:38 UTC - Roman Popenov: `/pulsar# /pulsar/bin/pulsar-admin functions localrun --jar /tmp/Pulsar-Demo-1.0-SNAPSHOT-uberjar.jar --classname com.fp.gem.pulsar.functions.PulsarSysmonTopicETL --namespace test-ns --tenant fp-tenant --inputs <persistent://fp-tenant/test-ns/demo-topic-sysmon> --output <persistent://fp-tenant/test-ns/demo-topic-sysmon-cim> --subs-name my-function-subscription` ---- 2020-01-10 23:51:42 UTC - Roman Popenov: Running that command ---- 2020-01-10 23:52:50 UTC - Jerry Peng: did you adjust the memory settings accordingly here: <https://github.com/apache/pulsar/blob/master/conf/pulsar_tools_env.sh#L45> ---- 2020-01-10 23:53:46 UTC - Roman Popenov: Where can I find those? ---- 2020-01-10 23:54:25 UTC - Roman Popenov: I only adjusted the memory and cpu values in the heml values file ---- 2020-01-10 23:55:33 UTC - Roman Popenov: ```# Extra options to be passed to the jvm PULSAR_MEM="-Xmx128m -XX:MaxDirectMemorySize=128m"``` ---- 2020-01-10 23:55:53 UTC - Roman Popenov: Those are the values I find in `/pulsar/conf` on the pod ---- 2020-01-10 23:59:58 UTC - Jerry Peng: Can you check what the heap or direct memory is set for the jvm process spawned from your localrun cmd ---- 2020-01-11 00:01:03 UTC - Roman Popenov: Not sure how to do it for localrun ---- 2020-01-11 00:01:13 UTC - Jerry Peng: jps -v ---- 2020-01-11 00:04:22 UTC - Roman Popenov: ```root@good-koala-pulsar-broker-5f56c84c4d-tgm4z:/pulsar# jps -v 389 LocalRunner -Dlog4j.configurationFile=log4j2.yaml -<http://Djava.net|Djava.net>.preferIPv4Stack=true -Xmx128m -XX:MaxDirectMemorySize=128m -Dio.netty.leakDetectionLevel=disabled -Dpulsar.log.appender=RoutingAppender -Dpulsar.log.dir=/pulsar/logs -Dpulsar.log.level=info -Dpulsar.routing.appender.default=Console -Dpulsar.functions.java.instance.jar=/pulsar/instances/java-instance.jar -Dpulsar.functions.python.instance.file=/pulsar/instances/python-instance/python_instance_main.py 441 Jps -Dapplication.home=/usr/local/openjdk-8 -Xms8m 364 PulsarAdminTool -Dlog4j.configurationFile=log4j2.yaml -<http://Djava.net|Djava.net>.preferIPv4Stack=true -Xmx128m -XX:MaxDirectMemorySize=128m -Dio.netty.leakDetectionLevel=disabled -Dpulsar.log.appender=RoutingAppender -Dpulsar.log.dir=/pulsar/logs -Dpulsar.log.level=info -Dpulsar.routing.appender.default=Console``` ---- 2020-01-11 00:06:13 UTC - Jerry Peng: are you running a python function? ---- 2020-01-11 00:06:18 UTC - Roman Popenov: no ---- 2020-01-11 00:06:20 UTC - Roman Popenov: Java ---- 2020-01-11 00:06:48 UTC - Jerry Peng: nvm I miss read ---- 2020-01-11 00:07:03 UTC - Jerry Peng: try setting Xmx128m -XX:MaxDirectMemorySize=128m to be higher ---- 2020-01-11 00:07:27 UTC - Roman Popenov: Ok. I can try ---- 2020-01-11 00:10:46 UTC - Roman Popenov: I am also thinking that perhaps it has something to do with the env settings: ```# Extra options to be passed to the jvm PULSAR_MEM="-Xms512m -Xmx1g -XX:MaxDirectMemorySize=512m -Dio.netty.leakDetectionLevel=disabled -Dio.netty.recycler.linkCapacity=1024 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+ExitOnOutOfMemoryError -XX:+PerfDisableSharedMem" # Garbage collection options PULSAR_GC="-XX:+UseG1GC -XX:MaxGCPauseMillis=10"``` ---- 2020-01-11 00:12:44 UTC - Jerry Peng: Thats the broker configurations? ---- 2020-01-11 00:12:51 UTC - Roman Popenov: yes ---- 2020-01-11 00:13:32 UTC - Jerry Peng: The pulsar-admin tool will be different and will use the settings in conf/pulsar_tools_env.sh ---- 2020-01-11 00:16:02 UTC - Roman Popenov: ok, I will try to double the settings there and see what happens ---- 2020-01-11 00:21:40 UTC - Roman Popenov: Yeah, looks like it did solve the issue ---- 2020-01-11 00:22:32 UTC - Roman Popenov: hopefully I can process the whole archived stream :crossed_fingers: ---- 2020-01-11 00:23:18 UTC - Jerry Peng: :+1: ---- 2020-01-11 01:34:45 UTC - ravi satya durga prasad Yenugula: @ravi satya durga prasad Yenugula has joined the channel ---- 2020-01-11 01:39:02 UTC - ravi satya durga prasad Yenugula: Hi, I am trying to install Pulsar on a Ec2 instance (centos) and I am getting memory error ---- 2020-01-11 01:39:28 UTC - ravi satya durga prasad Yenugula: ```OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000080000000, 2147483648, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 2147483648 bytes for committing reserved memory. # An error report file with more information is saved as:``` ---- 2020-01-11 01:44:18 UTC - ravi satya durga prasad Yenugula: Error Logs ---- 2020-01-11 01:44:43 UTC - ravi satya durga prasad Yenugula: any Idea? ---- 2020-01-11 02:39:34 UTC - Alexandre DUVAL: Hi, you need to set the $PULSAR_MEM env var. ---- 2020-01-11 03:10:55 UTC - Sijie Guo: is the event sequence external sequence or pulsar sequence id? ----
