2019-05-09 09:15:21 UTC - Shivji Kumar Jha: @jia zhai I must mention the low turnaround time in the PRs helps a lot in working with pulsar, a big thank you to the pulsar team :slightly_smiling_face: beers : jia zhai, Karthik Ramasamy man-running : Matteo Merli ---- 2019-05-09 09:15:58 UTC - Shivji Kumar Jha: bin/pulsar-admin namespaces set-schema-autoupdate-strategy -c none yo/logs ---- 2019-05-09 09:16:03 UTC - Shivji Kumar Jha: @Alexandre DUVAL ^ ---- 2019-05-09 09:16:37 UTC - Alexandre DUVAL: Really ? Lol ok :p ---- 2019-05-09 09:16:51 UTC - Shivji Kumar Jha: I authored it so i just remember that one :stuck_out_tongue: ---- 2019-05-09 11:10:54 UTC - James Creedy: I think I have 2.3.1 of everything. But something is not aligned correctly. Trying to build C++ client from source now, but other dependancy issues (that initially appear to be Python related). All painful. Any tips on how to remove any of this pain on Mac? ---- 2019-05-09 11:47:36 UTC - Brian Doran: Is it possible to run the pulsar standalone with tls enabled?
I have run the following command ensure all ports are available externally: `docker run --name=pulsar-tls -d -p 6650:6650 -p 8080:8080 -p 8081:8081 -p 6651:6651 -p 8443:8443 -v $PWD/data:/pulsar/data apachepulsar/pulsar:2.3.1 bin/pulsar standalone` I've edited the standalone.conf file to enable tls: brokerServicePort=6650 brokerServicePortTls=6651 # Configuration to enable authentication #authenticationEnabled=true #authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls tlsEnabled=true tlsCertificateFilePath=/certs/broker.cert.pem tlsKeyFilePath=/certs/broker.key-pk8.pem tlsTrustCertsFilePath=/certs/ca.cert.pem tlsAllowInsecureConnection=false # Port to use to server HTTP request webServicePort=8080 webServicePortTls=8081` However it still starts up with TLS disabled. ---- 2019-05-09 12:19:18 UTC - Sijie Guo: a) it is possible to run tls on standalone b) > However it still starts up with TLS disabled. which file did you modify? how did you modify the file in docker? ---- 2019-05-09 13:05:07 UTC - Brian Doran: well initially I modified broker.conf and that didn't do it. So now I have modified broker.conf/standalone.conf and client.conf with tls settings. `docker cp broker.conf pulsar-tls:/pulsar/conf/broker.conf docker cp client.conf pulsar-tls:/pulsar/conf/client.conf docker cp standalone.conf pulsar-tls:/pulsar/conf/standalone.conf' ---- 2019-05-09 13:06:02 UTC - Brian Doran: after copying them out via >docker cp pulsar-tls:/pulsar/conf/broker.conf . >docker cp pulsar-tls:/pulsar/conf/client.conf . >docker cp pulsar-tls:/pulsar/conf/standalone.conf ---- 2019-05-09 13:12:32 UTC - Brian Doran: I still get `11:44:02.465 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service on port=8080, broker url=<pulsar://030fb47520ee:6650>, cluster=standalone, conf........` ---- 2019-05-09 13:24:25 UTC - Brian Doran: @Sijie Guo Does this mean that's it's started up without TLS? I would have expected broker to be `broker url=<pulsar+ssl://030fb47520ee:6651>` ---- 2019-05-09 13:32:47 UTC - Brian Doran: Actually .. sorry for wasting your time I changed by URL connect to be pulsar+ssl://<my-pulsar-host>:6651 and I am successfully sending >2019-05-09 13:29:29.962Z INFO [pulsar-timer-35-1] o.a.p.c.i.ProducerStatsRecorderImpl - [<persistent://public/default/MYTopic>] [<persistent://public/default/TestTopic[Export-Pipeline-Queue-7]>] Pending messages: 17 --- Publish throughput: 4354.03 msg/s --- 68.85 Mbit/s --- Latency: med: 14.692 ms - 95pct: 48.839 ms - 99pct: 135.131 ms - 99.9pct: 10119.376 ms - max: 10195.522 ms --- Ack received rate: 4354.03 ack/s --- Failed messages: 0 >2019-05-09 13:29:29.963Z INFO [pulsar-timer-44-1] o.a.p.c.i.ProducerStatsRecorderImpl - [<persistent://public/default/TestTopic>] [<persistent://public/default/TestTopic[Export-Pipeline-Queue-5]>] Pending messages: 17 --- Publish throughput: 4408.42 msg/s --- 69.71 Mbit/s --- Latency: med: 15.003 ms - 95pct: 52.140 ms - 99pct: 165.113 ms - 99.9pct: 10115.282 ms - max: 10187.674 ms --- Ack received rate: 4408.42 ack/s --- Failed messages: 0 >2019-05-09 13:29:29.964Z INFO [pulsar-timer-38-1] o.a.p.c.i.ProducerStatsRecorderImpl - [<persistent://public/default/TestTopic>] [<persistent://public/default/TestTopic[Export-Pipeline-Queue-12]>] Pending messages: 16 --- Publish throughput: 4331.60 msg/s --- 68.50 Mbit/s --- Latency: med: 15.161 ms - 95pct: 48.321 ms - 99pct: 145.882 ms - 99.9pct: 10179.252 ms - max: 10190.454 ms --- Ack received rate: 4331.60 ack/s --- Failed messages: 0 ---- 2019-05-09 13:35:45 UTC - Sijie Guo: cool ---- 2019-05-09 13:48:10 UTC - Brian Doran: Hi @Sijie Guo Just wonder if there has been any progress on this feature request? Thanks ---- 2019-05-09 14:02:13 UTC - Sijie Guo: haven’t started yet unfortunately. I will try to get to the first part tomorrow. +1 : Brian Doran ---- 2019-05-09 14:05:32 UTC - Brian Doran: Thanks @Sijie Guo ---- 2019-05-09 14:32:35 UTC - Fredrick P Eisele: I see that the Zookeeper bundled with Pulsar has some differences from the general distribution. I am trying to decide which version to run. The Pulsar documentation on the subject <https://pulsar.apache.org/docs/en/deploy-bare-metal/#deploying-a-zookeeper-cluster> seems ambivalent. What are the arguments either way? ---- 2019-05-09 15:00:09 UTC - Marc: @Marc has joined the channel ---- 2019-05-09 15:17:07 UTC - Grant Wu: Are you _sure_ you have 2.3.1 of everything? ---- 2019-05-09 15:22:56 UTC - Matteo Merli: The only difference we have is that we added metrics on top of regular ZK. Out of the box, ZK has a very limited and not super-useful set of metrics. We added all the instant rates, latencies, breakdown by request types. The metrics are exported through Prometheus format. +1 : Shivji Kumar Jha ---- 2019-05-09 15:24:35 UTC - Matteo Merli: Eg. this is the resulting dashboard: ---- 2019-05-09 15:24:44 UTC - Matteo Merli: ---- 2019-05-09 17:17:12 UTC - Dan Stiner: @Dan Stiner has joined the channel ---- 2019-05-09 17:30:37 UTC - Thor Sigurjonsson: Any rough guesstimates as to when we might be seeing version 2.3.2 becoming available? ---- 2019-05-09 17:31:37 UTC - Matteo Merli: Waiting on votes for BK 4.9.2 <https://lists.apache.org/thread.html/88c0fcc5959558bb1d678f469a903943b218b83fe4b4fef6ae76f668@%3Cdev.bookkeeper.apache.org%3E> +1 : Thor Sigurjonsson, Jean-Bernard van Zuylen, Ali Ahmed ---- 2019-05-09 19:56:28 UTC - Fredrick P Eisele: I see, thanks. Any effort to get those metrics into upstream ZK? ---- 2019-05-09 19:57:26 UTC - Fredrick P Eisele: Is that a Grafana dashboard? ---- 2019-05-09 20:17:27 UTC - Matteo Merli: There are other efforts for ZK master 3.6. The stable version is still only in 3.4 though ---- 2019-05-09 20:18:00 UTC - Matteo Merli: Even 3.5 is still marked as beta ---- 2019-05-09 21:55:57 UTC - Fredrick P Eisele: Are there any issues with running Pulsar with ZK v3.5? ---- 2019-05-09 21:56:34 UTC - Fredrick P Eisele: <https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html> ---- 2019-05-09 21:59:16 UTC - Matteo Merli: No issues. Just that it’s marked beta ---- 2019-05-10 08:26:24 UTC - Kim Christian Gaarder: I have two questions related to partitioned topics in wanting to use them as a transaction log: 1) Do partitioned topics provide consistent read-ordering across consumers/readers, i.e. provide total-ordering? 2) Can partitioned topics be combined with tiered storage? ---- 2019-05-10 08:27:28 UTC - Ali Ahmed: @Kim Christian Gaarder yes for both questions ---- 2019-05-10 08:27:44 UTC - Kim Christian Gaarder: Excellent, thank you :slightly_smiling_face: ----
