2019-07-26 12:16:09 UTC - Ben: Ah, the issue was message retention time. ---- 2019-07-26 12:29:21 UTC - David Lucas: @David Lucas has joined the channel ---- 2019-07-26 13:53:27 UTC - Mahesh: Hi, I have created a tenant through pulsar-admin command, It got created successfully and I can see the same when I list it. But on pulsar dashboard webUI, I dont see that. What could have gone wrong here ? ---- 2019-07-26 13:56:55 UTC - Richard Sherman: A couple of scenarios where this has happened to me 1. Dashboard didn't actually have network access to a broker/proxy 2. Tenant didn't have the cluster set correctly ---- 2019-07-26 14:02:52 UTC - Richard Sherman: To check the second run `pulsar-admin tenants get tenant_name` ---- 2019-07-26 14:03:01 UTC - Richard Sherman: To check the first one run the collector.sh script in a terminal and see if you get any errors logged. You'll need to ctrl-c out of the script as it loops and reruns the collection code every 60 seconds ---- 2019-07-26 14:34:26 UTC - Alex Lepsa: @Alex Lepsa has joined the channel ---- 2019-07-26 14:43:10 UTC - Alex Lepsa: Hi, I'd like to use a `protoc` generated class to set the schema for a topic but trying to do that via something like `JSONSchema.of(TaskProtos.Task.class)` is not working. How do you guys (or don't you) use protobuf generated classes to set a schema? ---- 2019-07-26 14:49:00 UTC - Mahesh: I think I found the issue, I gave wrong cluster name. After fixing the cluster name, now i get this error ---- 2019-07-26 14:49:01 UTC - Mahesh: Namespace missing local cluster name in clusters list: local_cluster=bpulsar ns=public/functions clusters=[apulsar] ---- 2019-07-26 14:49:21 UTC - Alex Lepsa: or is a .proto based schema registration only supported server side? ---- 2019-07-26 14:49:37 UTC - Mahesh: ``` [zk: localhost:2181(CONNECTED) 17] ls /admin/policies/public/functions [] ``` ---- 2019-07-26 14:49:37 UTC - Mahesh: what could be wrong here ? ---- 2019-07-26 14:53:08 UTC - Thomas Cothran: @Thomas Cothran has joined the channel ---- 2019-07-26 14:54:30 UTC - Richard Sherman: Does the following command show the correct cluster name? pulsar-admin namespaces get-clusters tenant_name/namespace_name ---- 2019-07-26 14:56:24 UTC - Ryan Samo: Hey guys, Does Pulsar SQL work in a cluster that is setup with TLS authentication and authorization? It couldn’t find an example setup for that in the docs or source code ---- 2019-07-26 14:59:15 UTC - Mahesh: yes, it returns, ``` <mailto:[email protected]|[email protected]>:/usr/local/apache-pulsar/bin# ./pulsar-admin namespaces get-clusters public/default apulsar bpulsar ``` ---- 2019-07-26 14:59:31 UTC - Mahesh: after sometime broker exits ---- 2019-07-26 15:26:46 UTC - David Kjerrumgaard: @Ryan Samo I wouldn't see why not, although I am unaware of any tests conducted in such an environment. Are you experiencing issues with such a configuration? ---- 2019-07-26 15:28:53 UTC - Ryan Samo: @David Kjerrumgaard yeah I tried it a while back and was receiving an authorization error. I need to try it once more ---- 2019-07-26 15:30:36 UTC - Ryan Samo: Just wondering if you all had an example of a cluster config that worked ---- 2019-07-26 15:32:52 UTC - Ryan Samo: A secure cluster :) ---- 2019-07-26 15:35:53 UTC - David Kjerrumgaard: @Ryan Samo At which level were you getting the auth error? And what are you using to secure the cluster? TLS, JWT? ---- 2019-07-26 15:37:48 UTC - Ryan Samo: TLS ---- 2019-07-26 15:38:24 UTC - Ryan Samo: Let me start a worker and retry it to see. It was when I tried to perform a sql query ---- 2019-07-26 15:59:09 UTC - Ryan Samo: @David Kjerrumgaard Error: Query 20190726_155712_00001_mntr3 failed: Failed to get schemas from pulsar: error:10000412:SSL routines:OPENSSL_internal:SSLV3_ALERT_BAD_CERTIFICATE ---- 2019-07-26 16:06:19 UTC - Jerry Peng: @Ryan Samo auth in sql was just added and is not currently in any official release ---- 2019-07-26 16:06:42 UTC - Jerry Peng: Feel free to compile from master to try it out ---- 2019-07-26 16:07:12 UTC - Ryan Samo: @Jerry Peng that’s awesome! Thanks for the tip, I sure will! ---- 2019-07-26 23:15:09 UTC - Devin G. Bost: @Jerry Peng What does it mean when there is a backlog on a topic without any un-acked messages? ---- 2019-07-26 23:15:33 UTC - Devin G. Bost: We have a producer/consumer that's not processing messages, and we're trying to figure out what's wrong. ---- 2019-07-26 23:36:46 UTC - Jerry Peng: @Devin G. Bost that means your consumers for a particular subscriptions is not consuming messages as fast as messages are getting produced to the topic ---- 2019-07-26 23:42:03 UTC - Jerry Peng: @Devin G. Bost is the consumer msgRateOut for the consumer 0? ---- 2019-07-26 23:42:10 UTC - Jerry Peng: check availablePermits ---- 2019-07-27 03:07:55 UTC - Sijie Guo: @Devin G. Bost :
a few things you can check: 1) try unload the topic to see if the consumer can resume. if it resumes, it might be related to any race conditions in the permits flow; if it resumes and falls back again, check the stats-internal to see if there is any outstanding read requests. It can be related to too small a block cache configured in bookies, which results in index pages swap in-and-out when consuming large backlog. if you expect to have large backlog, consider configuring dbStorage_rocksDB_blockCacheSize to be 1~2GB or even more. I have seen a lot of similar cases related to this setting. 2) if unload a topic doesn’t help, check if you are using 2.3.0 or earlier versions. If so, check if `bookkeeperEnableStickyReads` is set to true. If it is set to true, disable it. There is a problem of sticky reads. I have also seen similar cases related to this setting. 3) if none above addressed your problem, it would be good to grep the `stats-internal` every a few minutes and compare the stats to see if there is any pattern. ----
