2020-10-16 12:32:11 UTC - Rolf Arne Corneliussen: I do not think that is possible. Using just two data centers, you will end up with a majority of ZK nodes in one of them, and if that data center becomes unavailable, the ZK cluster as a whole becomes unavailable. You will need at least three data centers if you want the ZK cluster to be available in the event one of the data centers fails. ---- 2020-10-16 15:03:11 UTC - Kenan Dalley: It looks like the server & client need to be at least 2.6.x for this to work. I built my client with it, but the server is still 2.4.x and basically just ignored the api command. ---- 2020-10-16 15:03:28 UTC - Kenan Dalley: Thanks for the help, though ---- 2020-10-16 15:04:07 UTC - ckdarby: Ah, I'm in the land of 2.5 didn't know about this nice feature. Thanks @Addison Higham ---- 2020-10-16 15:05:30 UTC - ckdarby: @Addison Higham Does the PrestoSQL connector also use this if I search by timestamp? :thinking_face: ---- 2020-10-16 15:17:50 UTC - Jeff Schneller: I am unable to unsubscribe a subscription. I have done it in the past without issue. I am receiving "Subscription has active connected consumers" I am trying to pass a -f to unsubscribe but it is not forcing it and says -f is an unknown option. any ideas? Need to unsubscribe to change a subscription from exclusive to shared ---- 2020-10-16 15:38:26 UTC - Evan Furman: Hi! Is there formal documentation regarding configuring zookeeper with TLS outside of what is documented here? <https://github.com/apache/pulsar/issues/6236#issuecomment-630613528> ---- 2020-10-16 15:39:06 UTC - Evan Furman: Do I need both the conf and pulsar.sh arguments? ---- 2020-10-16 15:46:18 UTC - Addison Higham: @Jeff Schneller `--force` is a recently new option in the CLI and also requires a new version of pulsar, what version are you running for client and server?
If you are running a version before force, obviously, the easiest thing to do is just to stop your consumers, but if that is difficult and you are using authorization, you can temporarily remove auth for the client, `unload` the topic to force the consumer to try and reconnect (and won't be able to), unsubscribe, then change the subscription. If not running auth a similar thing can work at the network layer ---- 2020-10-16 15:46:47 UTC - chiru: @chiru has joined the channel ---- 2020-10-16 15:50:48 UTC - chiru: <!here>, we are trying to install Pulsar in AKS (Azure Kubernetes) with Azure file share as persistent data storage for Pulsar Engine (Broker).. can't find any literature available online.. while we could see articles for S3, GCS and File Storage (Hadoop).. with current version 2.6.1, is it possible to use either Azure blob or Azure file share as persistent data storage (literally speaking, backend of pulsar to store the data instead its local file storage). kindly guide us or share any articles related to it.. thanks in advance.. face_with_head_bandage : Frank Kelly, ckdarby ---- 2020-10-16 15:51:38 UTC - ckdarby: @chiru There are 2300 people in this channel please don't use the [AT]here tag. ---- 2020-10-16 15:52:49 UTC - chiru: my apologies.. sorry .. wont use it again ---- 2020-10-16 15:55:25 UTC - Ebere Abanonu: Try use this <https://github.com/eaba/charts> ---- 2020-10-16 15:56:24 UTC - Ebere Abanonu: You can set the StorageClassName to default as Azure create three StorageClassName by default eyes : ckdarby +1 : chiru star-struck : Frank Kelly ---- 2020-10-16 15:56:36 UTC - Jeff Schneller: is there a way to determine where the consumers are coming from? We are using authorization. I can remove an auth easily enough. Not sure what you are referring to with the unload the topic. If I remove the auth won't the consumer fail. ---- 2020-10-16 16:02:14 UTC - Addison Higham: Yes, that will cause the consumer to not be able to connect, but that is essentially what happens when you do `force`, it will kill the connection, but once the consumer reconnects, it would just recreate. Backing up a bit, if your consumer is specifying it wants an exclusive subscription, it will error out if you re-create the subscription as shared ---- 2020-10-16 16:03:12 UTC - ckdarby: Never thought of doing it this way @Ebere Abanonu very clever :slightly_smiling_face: ---- 2020-10-16 16:03:36 UTC - Jeff Schneller: ok. I will just revoke permissions and run the unload. It is all in a dev environment anyhow. Then remove the subscription and finally add the permission back in ---- 2020-10-16 16:03:44 UTC - Addison Higham: There is two parts to schema, one is being able to use the serializers and deserialziers, the other part is enforcement, you can still use the serde framework btu disable the topic validating the schema ---- 2020-10-16 16:05:27 UTC - Addison Higham: See the `pulsar-admin namespaces set-schema-validation-enforce` and `pulsar-admin namespaces set-schema-compatibility-strategy`. The one thing I am not sure of is if you can distribute schema the same way, @Sijie Guo ^^ can you add any details to that? ---- 2020-10-16 16:08:51 UTC - Jeff Schneller: @Addison Higham Thanks that worked. ---- 2020-10-16 16:12:26 UTC - Addison Higham: @Evan Furman are you referring to client auth from pulsar? To answer your first question, no, clientTls auth in zookeeper is relatively new and we haven't yet created a doc for it (but it should work) The reason for the pulsar.sh arguments is that I don't believe we expose all the parameters for zookeeper client tls in `broker.conf` , but you can use the java properties (the `-Dzookeeper`) arguments to configure it instead ---- 2020-10-16 16:27:47 UTC - Evan Furman: Ok cool. I’m looking to get TLS setup clusterwide — for brokers, bookies, and zk ---- 2020-10-16 16:27:54 UTC - Evan Furman: Trying it out now ---- 2020-10-16 16:42:31 UTC - Raghav: Hi I want to deploy just the broker component in kubernetes. But I don’t see any deployment file for brokers in the helm charts shared here <https://github.com/apache/pulsar-helm-chart/tree/master/charts/pulsar/templates> Can someone help me understand how the broker pods gets created from these templates? ---- 2020-10-16 16:44:03 UTC - Addison Higham: <https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/templates/broker-statefulset.yaml> <- we use a statefulset ---- 2020-10-16 16:45:40 UTC - Raghav: Thanks Addison will check it ---- 2020-10-16 16:47:28 UTC - Addison Higham: I believe it should yes ---- 2020-10-16 16:48:33 UTC - Sijie Guo: There is a WIP on support multiple different schema subjects on one topic. @Shivji Kumar Jha has a PIP for that. ---- 2020-10-16 17:00:04 UTC - Olivier Chicha: Hello, We use pulsar with kubernetes, we have some issues and we are trying to rebuild the image of the broker to be able to add some traces / try some patches I have forked the pulsar project and I am trying to build it on travis CI as I saw you have a .travis.yml at the root of the project unfortunately when I launched the build on the branch <http://2.6.it|2.6.0, >I get plenty of errors during the make : ```++ make Scanning dependencies of target gtest [ 25%] Building CXX object googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o In file included from /usr/include/c++/4.8/type_traits:35:0, from /home/travis/pulsar-dep/googletest/googletest/include/gtest/gtest.h:59, from /home/travis/pulsar-dep/googletest/googletest/src/gtest-all.cc:38: /usr/include/c++/4.8/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.``` what would be the simpliest way to modify some code in pulsar-zookeeper-utils and regenerate the docker image of pulsar and publish it on a docker repository ? ---- 2020-10-16 17:30:03 UTC - Addison Higham: @Olivier Chicha ```# from the project root mvn install -DskipTests # that will take a while... once finished cd docker mvn install -DskipTests -Pdocker``` That will build all of pulsar and new images (tagged like apachepulsar/pulsar-all) then you can just re-tag images ---- 2020-10-16 17:30:11 UTC - Evan Furman: @Addison Higham i don’t have a ton of experience with Java tls configuration — is there a good example for the `bk_jaas.conf` and `zk_jaas.conf` files? ---- 2020-10-16 17:32:37 UTC - Addison Higham: ah so some of that is also sasl stuff, which you shouldn't need ---- 2020-10-16 17:35:26 UTC - Evan Furman: I don’t know how to discern what is what :joy: ---- 2020-10-16 17:57:29 UTC - Olivier Chicha: thanks a lot ---- 2020-10-16 20:08:18 UTC - Addison Higham: for sure, I am trying to see if I can find a good example... the best place to start might be the zookeeper docs ---- 2020-10-16 20:09:45 UTC - Evan Furman: I kind of cobbled together a few different docs I read. ---- 2020-10-16 20:40:19 UTC - Evan Furman: ---- 2020-10-16 20:41:14 UTC - Evan Furman: ---- 2020-10-16 20:42:23 UTC - Evan Furman: ---- 2020-10-16 20:43:13 UTC - Toktok Rambo: `pulsar.Shared` ---- 2020-10-16 21:15:30 UTC - Addison Higham: @Evan Furman you might be able to ask @Rattanjot Singh questions as well as he just recently added TLS for bookkeeper side pray : Evan Furman ---- 2020-10-16 21:17:06 UTC - Addison Higham: @Evan Furman is your zookeeper.conf missing your trust store and what not? or is that being set elsewhere in your startup opts? If so, then it looks fairly sane to me ---- 2020-10-16 21:19:43 UTC - Evan Furman: I think it’s being set in pulsar_env.sh ---- 2020-10-17 05:29:34 UTC - hangc: <https://github.com/apache/pulsar/pull/8284> I have a PR for this problem, please help check the bug analysis, thanks a lot. @Sijie Guo @Lari Hotari @Penghui Li @jia zhai @Addison Higham +1 : jia zhai, Penghui Li heart : Lari Hotari ---- 2020-10-17 06:09:31 UTC - Lari Hotari: Good work @hangc! Thank you ---- 2020-10-17 06:25:01 UTC - hangc: @Lari Hotari Please help review the code, thanks a lot. ----
