2020-07-09 12:43:21 UTC - Ermir Zaimi: hi. we want to integrate pulsar with our core banking system. ibm as400. and we want to use change data capture. any suggestions? ---- 2020-07-09 13:18:22 UTC - Ebere Abanonu: I am in a situation where the broker pushes two messages at most even when the flow permit is 1000 ---- 2020-07-09 13:22:24 UTC - Aaron: When running with a standalone should the batch index acknowledge enable be in the broker conf or standalone conf? ---- 2020-07-09 13:31:41 UTC - Tamer: What database will you use CDC with?
We had great success with PostgresSQL and CDC. The only advice I have from that experience (could be specific for Postgres) make sure to have proper monitoring and alerting for the CDC source. Otherwise the Postgres slot will keep accumulating data and grow in size if it is not being consumer by an active CDC source. ---- 2020-07-09 13:32:37 UTC - Ermir Zaimi: thanks for the reply. we have db2 database ---- 2020-07-09 13:41:59 UTC - Tamer: Never tried with DB2, but I see a potential debezium connector in incubating state, <https://debezium.io/documentation/reference/1.2/connectors/db2.html> ---- 2020-07-09 13:54:02 UTC - Ermir Zaimi: is debezium the only option? ---- 2020-07-09 14:10:19 UTC - Ermir Zaimi: i mean is there another cdc tool that we can use ---- 2020-07-09 15:10:35 UTC - Lagranmoon: @Lagranmoon has joined the channel ---- 2020-07-09 15:54:39 UTC - Addison Higham: @Ermir Zaimi I know DB2 has some built in CDC support as well as commercial offerings that do CDC for DB2, I would imagine that those should be fairly straight forward to connect in to via implementing a "pulsar source function", but I imagine Debezium is hooking into that API ---- 2020-07-09 16:04:44 UTC - Addison Higham: are you talking about a key shared subscription? or just in general with exclusive subscriptions? Theoretically a key_shared subscription could perhaps preserve order, but I don't believe that is the case right now. As far as in general, there really isn't a great way to guarantee it generally, but this article: <https://jack-vanlightly.com/blog/2019/9/4/a-look-at-multi-topic-subscriptions-with-apache-pulsar> might give you some ideas about how you can minimize it, with the main idea being that the real problem with re-ordering is not on the producer side as much, but more a problem of consumers needing to read sort of "in lockstep" from multiple topics. +1 : victor ---- 2020-07-09 16:48:55 UTC - Viktor: @Sijie Guo I am using the omb framework for benchmarking. It starts up the BookieServer by running the class above. Thats what I see if I do a ps on the box where broker and bookie are running.. ---- 2020-07-09 17:11:40 UTC - Sijie Guo: standalone is using standalone.conf ---- 2020-07-09 17:16:27 UTC - Sijie Guo: I see. I haven’t touched omb for a while. So I don’t know how exactly it starts bookie. Can you get the full copy of `/opt/pulsar/conf/bookkeeper.conf` ? We can look into it. ---- 2020-07-09 18:09:33 UTC - Viktor: ```$ cat /opt/pulsar/conf/bookkeeper.conf | grep -v "#" zkServers=x.x.x.x:x,y.y.y.y:y,z.z.z.z:z advertisedAddress=x.x.x.x:x journalDirectories=/mnt/journal/1,/mnt/journal/2,/mnt/journal/3,/mnt/journal/4 ledgerDirectories=/mnt/storage dbStorage_writeCacheMaxSizeMb=1024 dbStorage_readAheadCacheMaxSizeMb=1024 dbStorage_rocksDB_blockCacheSize=1073741824 journalSyncData=false bookiePort=3181 allowLoopback=false ledgerManagerType=hierarchical zkLedgersRootPath=/ledgers ledgerStorageClass=org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage entryLogFilePreallocationEnabled=true logSizeLimit=1073741824 minorCompactionThreshold=0.2 minorCompactionInterval=3600 majorCompactionThreshold=0.5 majorCompactionInterval=86400 compactionMaxOutstandingRequests=100000 compactionRate=1000 isThrottleByBytes=false compactionRateByEntries=1000 compactionRateByBytes=1000000 journalMaxSizeMB=2048 journalMaxBackups=5 journalPreAllocSizeMB=16 journalWriteBufferSizeKB=256 journalRemoveFromPageCache=true journalAdaptiveGroupWrites=true journalMaxGroupWaitMSec=1 journalAlignmentSize=4096 journalBufferedWritesThreshold=524288 journalFlushWhenQueueEmpty=false numJournalCallbackThreads=8 rereplicationEntryBatchSize=5000 gcWaitTime=900000 gcOverreplicatedLedgerWaitTime=86400000 flushInterval=60000 bookieDeathWatchInterval=1000 zkServers=localhost:2181 zkTimeout=30000 serverTcpNoDelay=true openFileLimit=0 pageLimit=0 readOnlyModeEnabled=true diskUsageThreshold=0.95 diskCheckInterval=10000 auditorPeriodicCheckInterval=604800 auditorPeriodicBookieCheckInterval=86400 numAddWorkerThreads=8 maxPendingAddRequestsPerThread=1000000 numReadWorkerThreads=8 maxPendingReadRequestsPerThread=1000000 readBufferSizeBytes=4096 writeBufferSizeBytes=524288 useHostNameAsBookieID=false prometheusStatsHttpPort=8000 dbStorage_readAheadCacheBatchSize=1000 dbStorage_rocksDB_writeBufferSizeMB=64 dbStorage_rocksDB_sstSizeInMB=64 dbStorage_rocksDB_blockSize=65536 dbStorage_rocksDB_bloomFilterBitsPerKey=10 dbStorage_rocksDB_numLevels=-1 dbStorage_rocksDB_numFilesInLevel0=4 dbStorage_rocksDB_maxSizeInLevel1MB=256``` ---- 2020-07-09 18:09:55 UTC - Viktor: cc @Sijie Guo thanks. I pasted the file without the comments ---- 2020-07-09 18:19:57 UTC - Sijie Guo: I believe you need to configure the following setting in order to enable Prometheus statsProviderClass=org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider ---- 2020-07-09 18:36:11 UTC - Viktor: I see.. will give it a shot. ---- 2020-07-09 18:43:42 UTC - Viktor: ```Jul 09 18:40:33 pulsar[23956]: org.apache.commons.configuration.ConfigurationException: org.apache.commons.configuration.ConfigurationException: java.lang.ClassNotFoundException: org.apache.bookkeeper.stats.PrometheusMetricsProvider``` Getting this when starting up the bookie now. Will keep looking. Please let me know if you have ideas ---- 2020-07-09 19:18:41 UTC - Sijie Guo: Okay you need to include PrometheusMetricsProvider ---- 2020-07-09 19:19:48 UTC - Viktor: Seems related to <https://github.com/apache/bookkeeper/commit/20b360a3530d8a377169bf0740785d422d6e5181> ? ---- 2020-07-09 19:20:48 UTC - Viktor: OMB yaml seems to just grab `<https://archive.apache.org/dist/pulsar/pulsar-{{> pulsarVersion }}/apache-pulsar-{{ pulsarVersion }}-bin.tar.gz` does this not include this class. ---- 2020-07-09 19:22:08 UTC - Sijie Guo: the pulsar binary distribution includes that. ---- 2020-07-09 19:26:24 UTC - Viktor: Hmmm. do I need to any environment variable for monitoring jars to be included in class path following is the class path of the process. is it sufficient ? ```-cp /opt/pulsar/conf:::/opt/pulsar/lib/*:``` ---- 2020-07-09 19:33:17 UTC - Alan Hoffmeister: @Alan Hoffmeister has joined the channel ---- 2020-07-09 19:33:53 UTC - Sijie Guo: I think so ---- 2020-07-09 19:39:19 UTC - Alan Hoffmeister: Hello there, I'm trying to use the websocket API but I'm not sure about how I should implement a deduplication mechanism for my producer. I'm basically trying to avoid a message from being queued twice in case my producer needs to reconnect for some reason ---- 2020-07-09 19:49:45 UTC - Alan Hoffmeister: I guess I should wait for PIP-31? ---- 2020-07-09 20:25:28 UTC - Viktor: @Sijie Guo do you know the jar that will contain this class ---- 2020-07-09 20:30:33 UTC - Viktor: Okay. the class name was/is wrong in OMB ---- 2020-07-09 20:49:13 UTC - Viktor: Works now.. :smile: ---- 2020-07-09 23:47:07 UTC - victor: Nice posts thanks! ---- 2020-07-10 02:18:12 UTC - Joshua Decosta: Im having trouble getting the documented MTLS setup working with JKS. Here is all the information: client.conf: ```# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # <http://www.apache.org/licenses/LICENSE-2.0> # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # # Configuration for pulsar-client and pulsar-admin CLI tools # URL for Pulsar REST API (for admin operations) # For TLS: webServiceUrl=<https://localhost:8443/> # webServiceUrl=<http://localhost:8080/> # URL for Pulsar Binary Protocol (for produce and consume operations) # For TLS: brokerServiceUrl=<pulsar+ssl://localhost:6651/> # brokerServiceUrl=<pulsar://localhost:6650/> # Authentication plugin to authenticate with servers # e.g. for TLS authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls authParams=keyStoreType:JKS,keyStorePath:/broker.keystore.jks,keyStorePassword:changeme # Parameters passed to authentication plugin. # A comma separated list of key:value pairs. # Keys depend on the configured authPlugin. # e.g. for TLS # authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem # authParams= # Allow TLS connections to servers whose certificate cannot be # be verified to have been signed by a trusted certificate # authority. tlsAllowInsecureConnection=false # Whether server hostname must match the common name of the certificate # the server is using. tlsEnableHostnameVerification=false # Path for the trusted TLS certificate file. # This cert is used to verify that any cert presented by a server # is signed by a certificate authority. If this verification # fails, then the cert is untrusted and the connection is dropped. tlsTrustCertsFilePath= # Enable TLS with KeyStore type configuration in broker. useKeyStoreTls=true # TLS KeyStore type configuration: JKS, PKCS12 tlsTrustStoreType=JKS # TLS TrustStore path tlsTrustStorePath=/broker.truststore.jks # TLS TrustStore password tlsTrustStorePassword=changeme tlsClientAuthentication=true``` ---- 2020-07-10 02:20:07 UTC - Joshua Decosta: Ive attached the standalone.conf file cause its too big. ---- 2020-07-10 02:21:23 UTC - Joshua Decosta: here is what my broker keystore contains: broker.keystore.jks: ``` Keystore type: jks Keystore provider: SUN Your keystore contains 2 entries Alias name: caroot Creation date: Jul 8, 2020 Entry type: trustedCertEntry Owner: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Issuer: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Serial number: 2c02429ecdb1902ae3a4a28bd95247b45f2835d4 Valid from: Wed Jul 08 17:25:33 EDT 2020 until: Thu Jul 08 17:25:33 EDT 2021 Certificate fingerprints: MD5: 25:19:46:58:1E:cd:1D:44:ED:A6:59:95:1C:8C:28:7D SHA1: 9E:C7:82:C8:59:24:0F:C1:5D:9C:6D:CB:80:14:AF:BF:F0:F4:51:BC SHA256: 8B:F2:6C:de:5C:B6:20:C5:33:97:C6:48:91:87:8C:57:90:24:8E:1F:7F:72:3E:EE:AD:57:55:93:3C:E4:AE:90 Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit RSA key Version: 3 Extensions: #1: ObjectId: 2.5.29.35 Criticality=false AuthorityKeyIdentifier [ KeyIdentifier [ 0000: 4D 54 D1 AD 6F A0 43 79 7C 37 5E 97 FA EB 31 56 MT..o.Cy.7^...1V 0010: 93 8C B4 94 .... ] ] #2: ObjectId: 2.5.29.19 Criticality=true BasicConstraints:[ CA:true PathLen:2147483647 ] #3: ObjectId: 2.5.29.14 Criticality=false SubjectKeyIdentifier [ KeyIdentifier [ 0000: 4D 54 D1 AD 6F A0 43 79 7C 37 5E 97 FA EB 31 56 MT..o.Cy.7^...1V 0010: 93 8C B4 94 .... ] ] ******************************************* ******************************************* Alias name: localhost Creation date: Jul 8, 2020 Entry type: PrivateKeyEntry Certificate chain length: 2 Certificate[1]: Owner: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Issuer: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Serial number: 505953083e501a59d90c6a89760e4f9c1471bdd3 Valid from: Wed Jul 08 17:34:37 EDT 2020 until: Sun Nov 24 16:34:37 EST 2047 Certificate fingerprints: MD5: D3:6F:F5:78:B1:2A:E1:59:86:92:51:CE:16:D4:B0:B9 SHA1: AE:E0:BD:95:27:E5:B4:B1:EE:14:94:7B:AA:cd:CD:06:82:75:F1:B3 SHA256: 59:FC:19:F9:2C:AD:96:9F:12:4E:D4:10:E8:20:BC:F4:B0:5B:B3:6B:40:59:62:50:F1:D7:58:C6:81:6A:79:88 Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit DSA key Version: 1 Certificate[2]: Owner: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Issuer: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Serial number: 2c02429ecdb1902ae3a4a28bd95247b45f2835d4 Valid from: Wed Jul 08 17:25:33 EDT 2020 until: Thu Jul 08 17:25:33 EDT 2021 Certificate fingerprints: MD5: 25:19:46:58:1E:cd:1D:44:ED:A6:59:95:1C:8C:28:7D SHA1: 9E:C7:82:C8:59:24:0F:C1:5D:9C:6D:CB:80:14:AF:BF:F0:F4:51:BC SHA256: 8B:F2:6C:de:5C:B6:20:C5:33:97:C6:48:91:87:8C:57:90:24:8E:1F:7F:72:3E:EE:AD:57:55:93:3C:E4:AE:90 Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit RSA key Version: 3 Extensions: #1: ObjectId: 2.5.29.35 Criticality=false AuthorityKeyIdentifier [ KeyIdentifier [ 0000: 4D 54 D1 AD 6F A0 43 79 7C 37 5E 97 FA EB 31 56 MT..o.Cy.7^...1V 0010: 93 8C B4 94 .... ] ] #2: ObjectId: 2.5.29.19 Criticality=true BasicConstraints:[ CA:true PathLen:2147483647 ] #3: ObjectId: 2.5.29.14 Criticality=false SubjectKeyIdentifier [ KeyIdentifier [ 0000: 4D 54 D1 AD 6F A0 43 79 7C 37 5E 97 FA EB 31 56 MT..o.Cy.7^...1V 0010: 93 8C B4 94 .... ] ] ******************************************* ******************************************* Warning: The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore broker.keystore.jks -destkeystore broker.keystore.jks -deststoretype pkcs12". ``` broker.truststore.jks: ``` Keystore type: jks Keystore provider: SUN Your keystore contains 1 entry Alias name: caroot Creation date: Jul 8, 2020 Entry type: trustedCertEntry Owner: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Issuer: CN=localhost, OU=L, O=Op, L=B, ST=M, C=US Serial number: 2c02429ecdb1902ae3a4a28bd95247b45f2835d4 Valid from: Wed Jul 08 17:25:33 EDT 2020 until: Thu Jul 08 17:25:33 EDT 2021 Certificate fingerprints: MD5: 25:19:46:58:1E:cd:1D:44:ED:A6:59:95:1C:8C:28:7D SHA1: 9E:C7:82:C8:59:24:0F:C1:5D:9C:6D:CB:80:14:AF:BF:F0:F4:51:BC SHA256: 8B:F2:6C:de:5C:B6:20:C5:33:97:C6:48:91:87:8C:57:90:24:8E:1F:7F:72:3E:EE:AD:57:55:93:3C:E4:AE:90 Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit RSA key Version: 3 Extensions: #1: ObjectId: 2.5.29.35 Criticality=false AuthorityKeyIdentifier [ KeyIdentifier [ 0000: 4D 54 D1 AD 6F A0 43 79 7C 37 5E 97 FA EB 31 56 MT..o.Cy.7^...1V 0010: 93 8C B4 94 .... ] ] #2: ObjectId: 2.5.29.19 Criticality=true BasicConstraints:[ CA:true PathLen:2147483647 ] #3: ObjectId: 2.5.29.14 Criticality=false SubjectKeyIdentifier [ KeyIdentifier [ 0000: 4D 54 D1 AD 6F A0 43 79 7C 37 5E 97 FA EB 31 56 MT..o.Cy.7^...1V 0010: 93 8C B4 94 .... ] ] ******************************************* ******************************************* ``` my understanding is the CN should match up to the domain and or superUserRole. I used broker for both to avoid any issues. This wouldn't allow the initial connection to work. ---- 2020-07-10 02:21:44 UTC - Joshua Decosta: Is there something I'm missing from my configuration files? ---- 2020-07-10 02:29:53 UTC - Takahiro Hozumi: Hi, all I'm learning pulsar and have read the following article. <https://kesque.com/understanding-pulsar-message-ttl-backlog-and-retention/> I still don't understand some basic mechanism. Supposing a namespace configured 100GB retention policy, 1GB backlog quotas and 100GB messages are pushed with no subscription, as there is no subscription I think there is no message in backlog. Is it possible to retrieve all retained data from the topic by creating a subscription with `SubscriptionInitialPosition.Earliest` setting? What's happen in the subscription's backlog? Also is there a way to read acknowledged messages, which is retained by a retention policy, on a partitioned topic? It seems that this is not possible according to the documentation. <http://pulsar.apache.org/docs/en/concepts-clients/#reader-interface> > The reader interface for Pulsar cannot currently be used with partitioned topics. Is a parameter of consumer `SubscriptionInitialPosition.Earliest` means earliest message in a topic that include acknowledge message? ---- 2020-07-10 02:33:44 UTC - Joshua Decosta: in regards to extending Authentication; Is the `authenticate` method in `AuthenticationProvider` supposed to pull the role value out of the authentication choice and return a string representation? Does that then get put into the http headers and passed on to the `AuthorizationProvider`? If not where do the parameters for any of the methods noted in `AuthorizationProvider` get obtained from? I've been trying to figure out the flow of the internal codebase but I'm not sure how the authorization methods should be implemented since I'm not sure where the role access information is coming from and where are the namespace/tenant/topic role authorization information being store in Pulsar? ---- 2020-07-10 03:56:33 UTC - Jay/Fienna Liang: @Jay/Fienna Liang has joined the channel ---- 2020-07-10 04:11:06 UTC - Jay/Fienna Liang: Hi, everyone, I can’t find any hint in official documentation about how Pulsar REST API authenticate with the brokers, could someone give me some direction? Say that I use JWT authentication in my `client.conf`, and I want to use Postman as my tool to issue request to grant permission of a certain role on a namespace, where should I put the JWT in the request? In header or url query parameters? I dig into the code of Pulsar such as the `pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationDataCommand.java` file, but gain no much information. Could anyone help? Thanks in advance. ---- 2020-07-10 04:54:54 UTC - Addison Higham: @Takahiro Hozumi a few notes: In a future release (2.7.0) it will be changed such that readers can operate in a way that they aren't subject to backlog quota, see <https://github.com/apache/pulsar/pull/6787> for the discussion, but yes, until that is merged, it is a bit of a gap in the reader API and means you probably want to disable backlog quotas for that use case. For your second question, you *can* use the Reader on the underlying partition topics. For example, a partitioned topic `my-topic` is actually a "meta topic" with 5 underlying real topics `my-topic-partition-{0,1,2,3,4}`. You can create a reader per each underlying topic. And yes, `SubscriptionInitialPosition.Earliest` is the earliest message of what is retained and has no bearing on any other subscriptions. ---- 2020-07-10 05:11:21 UTC - Addison Higham: @Jay/Fienna Liang for the HTTP API, it uses the standard `Authorization` header with a `Bearer` token ---- 2020-07-10 05:11:41 UTC - Addison Higham: so physically: `Authorization: Bearer <token>` ---- 2020-07-10 05:15:19 UTC - Takahiro Hozumi: @Addison Higham Thank you for your reply! Will backlog include all retained messages If I create a regular consumer with SubscriptionInitialPosition.Earliest setting instead of a reader? I think this is not possible because backlog quota is limited to only 1GB. Can messages that were marked as ready to delete once(depends on retention policy) go back to the backlog again in the first place? ---- 2020-07-10 05:16:35 UTC - Jay/Fienna Liang: Gosh it’s so straight forward but I never thought of it! You saved my day, thanks a lot! ---- 2020-07-10 05:25:45 UTC - Addison Higham: I am not 100% sure of how that interacts, but I believe that is correct for a consumer with a backlog quota. Backlog quotas and retention policies have somewhat tricky interaction. One thing to keep in mind though is that properly configured storage offloading as well as deleting inactive subscriptions is a good way to prevent completely unbounded growth And when thinking about retention vs subscriptions. A topic and it's retention policy is are *mostly* independent of subscriptions (with the one exception that a subscription can retain messages). A subscription is basically a pointer into a topic, talking about acknowledged messages only really applies to a subscription, not the underlying topic ---- 2020-07-10 05:29:41 UTC - Addison Higham: @Joshua Decosta I haven't spent as much time with that code recently, but just as a bit of pointer: There are two parts to implementing auth, there is the `org.apache.pulsar.client.api.Authentication` interface which is what clients use to provide tokens/certs/etc. The `org.apache.pulsar.broker.authentication.AuthenticationProvider` interface is only used on the server side. So for pretty much most authentication APIs, you have an implementation of both interfaces. But the rest of what you are talking about is correct. The `AuthenticationProvider` pretty much is just there to return a string (or throw an exception) which is the role (which is somewhat misnamed, it should be called `principal`) ---- 2020-07-10 05:30:26 UTC - Addison Higham: :bow: ---- 2020-07-10 05:40:06 UTC - Takahiro Hozumi: > A subscription is basically a pointer into a topic, talking about acknowledged messages only really applies to a subscription, not the underlying topic I got it! Thank you for clarification. > Backlog quotas and retention policies have somewhat tricky interaction. Okay. One thing I worried is that to create a consumer with SubscriptionInitialPosition.Earliest might stop producer immediately because of `producer_request_hold` policy. ---- 2020-07-10 05:41:41 UTC - wuYin: sorry for bother I’m using pulsar-helm-chart, it deploy all components over k8s by default In my case, I already have a physical zk cluster and want configure bookkeeper & broker & proxy to use it so I set values.yaml/components.zookeeper as false and only found /templates/_zookeeper.tpl/pulsar.zookeeper.connect to set my physical zk address but bookkeeper initContainer is using spliced zk Service URL to detect zk ready or not other component initContainer do the same thing so deploy stage will be stucked in bookkeeper wait-zookeeper-ready container my issue: templates/_zookeeper.tpl/pulsar.zookeeper.connect is not using at component’s initContainer or it was designed this way？ ---- 2020-07-10 05:46:26 UTC - Addison Higham: that is a good question that I am not completely sure of, but seems possible. Definitely a place Pulsar could improve some. These are the sorts of questions that would actually make a good issue and we can either add some notes to the docs if that is by design or get a discussion about if there might be some behavior changes that would have less of a sharp edge ---- 2020-07-10 06:25:17 UTC - Sijie Guo: technically you don’t need to wait for PIP-31. Pulsar already support broker-side de-duplication. But the feature might not be available at the websocket part. Feel free to create an issue for it. ---- 2020-07-10 06:31:43 UTC - Sijie Guo: No I don’t think it was designed in that way. Can you send a PR to change it to use `pulsar.zookeeper.connect`? ---- 2020-07-10 06:46:39 UTC - Takahiro Hozumi: <https://github.com/apache/pulsar/issues/7500> I've created an issue just now. Thank you for your support. ---- 2020-07-10 06:46:48 UTC - wuYin: no problem, I’ll try to make zk connect address configurable by the way, my case is zookeeper component has been maintained by professional colleagues, it has high availability if I need a cluster, just apply then get zk address so I needn’t deploy zk cluster over k8s, and maintain on my own ---- 2020-07-10 08:15:33 UTC - andrius.antipovas: @andrius.antipovas has joined the channel ---- 2020-07-10 08:20:47 UTC - andrius.antipovas: Hi, I am working with Pulsar in my company. Could you help me how to select messages from topic using Java? For example by specific value in json. ---- 2020-07-10 08:41:36 UTC - Sijie Guo: Understand that. thanks ---- 2020-07-10 09:01:45 UTC - xue: Does pulsar plan to use curator instead of Apache zookeeper? ----
