2019-07-09 09:18:39 UTC - Mahesh: Looks like my message is lost, posting it for the 3rd time, Hi, I have a question regarding pulsar functions. 1) If the input topic is regex pattern, in what subscription mode does the function run ? 2) Can auto creation of topics be disabled in pulsar ? ---- 2019-07-09 09:24:26 UTC - Sijie Guo: 1) the subscription type is determined by `--retain-ordering` and `--processing-guarantees`. It is not determined by whether it is regex pattern. 2) yes. in broker.conf, you can set `allowAutoTopicCreation` to `false`. ---- 2019-07-09 09:43:43 UTC - Mahesh: @Sijie Guo Can you please let me know in which release of pulsar, its added. I have checked <https://pulsar.apache.org/docs/en/reference-configuration/#broker> 2.4.0 page, I dont find ---- 2019-07-09 09:44:54 UTC - Sijie Guo: it is in 2.4.0. I think this setting is not updated to the broker.conf and documentation. filling a github issue to fix it. ---- 2019-07-09 09:54:48 UTC - Mahesh: thanks ---- 2019-07-09 10:00:17 UTC - Sijie Guo: <https://github.com/apache/pulsar/issues/4693> ---- 2019-07-09 10:03:54 UTC - Mahesh: I have one quick question regarding geo replication. I have 2 DCs and have configured sync replication. If one DC is down, and write through a healthy DC. Will the message write fails ---- 2019-07-09 15:10:28 UTC - jia zhai: write to local healthy DC will success. but message could not be replicated to failed DC ---- 2019-07-09 16:27:53 UTC - Devin G. Bost: BTW, regarding the video I posted here <https://youtu.be/qsz-FruOGoo> , I noticed that most people were dropping off at the slide when I talked about how Overstock is hiring, so I removed that slide since we already talk about that at the end. +1 : Jerry Peng, Bogdan BUNECI ---- 2019-07-09 18:11:57 UTC - Thor Sigurjonsson: Quick question: When I want to get rid of a bookie from a cluster (I removed a server to be used in a 2nd Pulsar cluster) -- How do I make it so that other bookies stop trying to connect to it? ---- 2019-07-09 18:29:33 UTC - Jon Bock: @Mahesh How have you configured synchronous replication? Pulsar’s cluster-to-cluster replication is asynchronous as you probably saw. ---- 2019-07-09 18:35:33 UTC - David Kjerrumgaard: I think that would require editing the metadata in the ZK node and removing the bookie from the list of bookies ---- 2019-07-09 18:37:33 UTC - David Kjerrumgaard: I am trying to find the documentation on that..... :smiley: ---- 2019-07-09 18:40:48 UTC - Sijie Guo: 1) you can stop a bookie or turn a bookie to readonly 2) run `bin/bookkeeper shell recover` to manually replicated the data of that bookie to other bookies
(<http://bookkeeper.apache.org/docs/latest/admin/autorecovery/#manual-recovery>) alternatively, you can run `bin/bookkeeper shell decommission` to decommission a bookie. (<http://bookkeeper.apache.org/docs/latest/reference/cli/#decommissionbookie>) +1 : Thor Sigurjonsson ---- 2019-07-09 18:53:01 UTC - David Kjerrumgaard: @Thor Sigurjonsson I believe that you will need to use the BK shell tool to decommission a bookie <https://bookkeeper.apache.org/docs/4.8.2/reference/cli/#decommissionbookie> +1 : Thor Sigurjonsson ---- 2019-07-09 19:26:16 UTC - Thor Sigurjonsson: @David Kjerrumgaard Thanks, that makes sense. The server was down and so it didn't show up in the bk shell in list of bookies, but still other bookies want to connect. I'll see if I can get it up and decommissioned.. But that's kind of where I'm at. ---- 2019-07-09 19:26:47 UTC - Thor Sigurjonsson: Just wondering if there is a way if it was nuked completely (a hole in the ground) -- how I'd go about it. ---- 2019-07-09 19:33:29 UTC - David Kjerrumgaard: In that scenario, you would have to use the ZK shell and delete the bookie entries, but I don't know which zk node(s) contain that information. ---- 2019-07-09 20:07:12 UTC - Grant Wu: please ---- 2019-07-09 20:07:15 UTC - Grant Wu: no `@here` ---- 2019-07-09 20:54:58 UTC - Aaron: I am making the ackTimeout for the consumer an hour, and as a result, the memory in my JVM is being blown away with lots of ConcurrentOpenHashSets. Is this intended and are there ways to lower the footprint without lowering the ackTimeout too much (I need control of when the server redelivers messages, and do not want the server redelivering on timeout at any point)? ---- 2019-07-09 20:59:16 UTC - Matteo Merli: From 2.4 on, using negative acks is a more efficient way to trigger redeliveries ---- 2019-07-09 21:00:58 UTC - Aaron: That's what I am doing, and I want the ackTimeout to be large enough that it is never triggered. (I am also working with a multiple-partitioned topic, which causes the large allocation of ConcurrentOpenHashSets) ---- 2019-07-09 21:03:11 UTC - Matteo Merli: That should just contain the message ids of message not yet acked / neg-acked. It shouldn’t grow too big ---- 2019-07-09 21:55:19 UTC - Naveen Siddareddy: @Naveen Siddareddy has joined the channel ---- 2019-07-09 21:58:38 UTC - Naveen Siddareddy: Kstreams or Apache Beam search here didn't yield any result..wondering if pulsar had libraries that does this.. mainly taking care of data dependencies ---- 2019-07-09 22:00:08 UTC - Jerry Peng: @Naveen Siddareddy we have something better :smile:. Its called Pulsar Functions: <http://pulsar.apache.org/docs/en/functions-overview/> +1 : Matteo Merli, Naveen Siddareddy, Ali Ahmed, Sijie Guo ---- 2019-07-10 01:58:33 UTC - suenaga kengo: @suenaga kengo has joined the channel ----
