2020-07-12 15:51:28 UTC - Asaf Mesika: I saw this, on recent blog post comparing Kafka to Pulsar: > Pulsar’s consumption model takes a *streaming-pull* approach. This is an enhanced version of long-polling as it eliminates the wait time between individual calls and requests and provides bi-directional message streaming. The streaming-pull model enables Pulsar to achieve lower end-to-end latency than any other existing long-polling-based messaging solutions, such as Kafka. Lower end-to-end latency helps enterprises gain business insights faster. I would really love some more in-depth explanation to this, especially comparing Kafka client behaviour to Pulsar one. From my understanding, Kafka utilizes, poll based approach: Thread is done, it takes what ever it has from previous fetch, if not, it goes to broker, and wait until receives records (long poll). Pulsar on the other hand, also has a buffer, but it’s async in nature, you consume from the buffer, which buffer keeps getting filled up by async thread. if buffer is empty, it blocks. Is that true? ---- 2020-07-12 17:43:18 UTC - Zhenhao Li: hi, I'm trying to deploy a Pulsar cluster. but bookies cannot start with the following error ```Jul 12 19:25:36 server1 systemd[1]: Started Pulsar's Bookkeeper Daemon. Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: 19:25:43.286 [main] ERROR org.apache.bookkeeper.bookie.Bookie - There are directories without a cookie, and this is neither a new environment, nor is storage expansion enabled. Empty directories are [/var/lib/pulsar-bookie/journal> Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: 19:25:43.308 [main] ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: org.apache.bookkeeper.bookie.BookieException$InvalidCookieException: Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.bookie.Bookie.checkEnvironmentWithStorageExpansion(Bookie.java:468) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.bookie.Bookie.checkEnvironment(Bookie.java:250) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.bookie.Bookie.<init>(Bookie.java:688) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:136) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:105) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:41) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:301) ~[org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.server.Main.doMain(Main.java:221) [org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.server.Main.main(Main.java:203) [org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 pulsar-bookie-start[5229]: at org.apache.bookkeeper.proto.BookieServer.main(BookieServer.java:313) [org.apache.bookkeeper-bookkeeper-server-4.10.0.jar:4.10.0] Jul 12 19:25:43 server1 systemd[1]: pulsar-bookie.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Jul 12 19:25:43 server1 systemd[1]: pulsar-bookie.service: Failed with result 'exit-code'. Jul 12 19:25:43 server1 systemd[1]: pulsar-bookie.service: Consumed 12.151s CPU time, received 7.5K IP traffic, sent 5.8K IP traffic.``` what could be the possible cause? ---- 2020-07-12 18:03:05 UTC - Zhenhao Li: in 3-node deployment, this occurs on 2 nodes. it worked on one node. I am surprised by the inconsistency. ---- 2020-07-12 18:29:36 UTC - Zhenhao Li: tried a few times. it seems that exactly one node works the other two fail. the working node is random ---- 2020-07-13 00:33:07 UTC - Joshua Decosta: Are people using Pulsar generally using the JWT or TLS authN/authZ setup? I’m trying to find a way to securely use the default setup with a truststore. I’m getting lost in how to properly manage those certificates once generated. Does cert-manager allow for truststores to be dynamically update with new certs or are people taking to using their own certificate authority to generate certs? ---- 2020-07-13 04:41:03 UTC - Hiroyuki Yamada: @Penghui Li <https://github.com/apache/pulsar/issues/7455#issuecomment-655883058> Can you give me the detail of a broker dump ? Once given, I will prepare logs. ---- 2020-07-13 04:47:46 UTC - Mindaugas: @Mindaugas has joined the channel ---- 2020-07-13 06:45:56 UTC - Penghui Li: The broker heap dump ---- 2020-07-13 06:46:25 UTC - Penghui Li: I want to check the consumer state under the key_shared subscription ---- 2020-07-13 07:20:14 UTC - Hiroyuki Yamada: @Penghui Li Sorry, how can I get that ? ---- 2020-07-13 07:22:00 UTC - Penghui Li: You can use `jmap -dump` to get the heap dump ---- 2020-07-13 07:22:58 UTC - Hiroyuki Yamada: OK, so is it fine ? • Start pulsar • Run the test • jmap -dump ---- 2020-07-13 07:23:40 UTC - Penghui Li: yes +1 : Hiroyuki Yamada ---- 2020-07-13 07:24:34 UTC - Penghui Li: If the problem happens, dump the broker +1 : Hiroyuki Yamada ---- 2020-07-13 07:56:26 UTC - Hiroyuki Yamada: @Penghui Li Can you see this? <https://www.dropbox.com/sh/ah4ton8latx1yua/AAA6STJopAqKQQje9m21g0Dga?dl=0> ---- 2020-07-13 07:57:31 UTC - zlql1: @zlql1 has joined the channel ---- 2020-07-13 07:58:11 UTC - Hiroyuki Yamada: before-produce, after-produce and after-consume are the dump files from `jmap -dump:file=xxx` . sub1 and sub2 are the logs of consumers for each subscription. ---- 2020-07-13 07:58:39 UTC - Hiroyuki Yamada: Since I use the standalone for this, no way to dump only broker. Is it OK ? ---- 2020-07-13 08:26:01 UTC - Penghui Li: Standalone is ok ---- 2020-07-13 08:28:03 UTC - Penghui Li: Do you have a tool to highlight the inconsistent messages? ---- 2020-07-13 08:28:38 UTC - Hiroyuki Yamada: <https://github.com/feeblefakie/misc/blob/master/pulsar/check.sh> ---- 2020-07-13 08:29:03 UTC - Hiroyuki Yamada: put the sub1 and sub2 to /tmp , then you can run it ---- 2020-07-13 08:37:48 UTC - geal: hey all! I'm pleased to announce that we released the 1.0 version of the Rust Pulsar client: <https://crates.io/crates/pulsar> It supports most of the features of other clients: • plaintext and TLS connections • normal and partitioned topics • consumers can subscribe on one topic, or a list, or a regex of topic names • message batching • compression • automatic reconnection with exponential backoff +1 : Manuel Mueller ---- 2020-07-13 08:39:10 UTC - geal: missing features as of now: • integration with the schema registry: adding all of the serialization libraries in the project was not practical, and they are not all of good quality, but the API gives an easy way to handle serialization and deserialization • encryption ----
