2019-04-05 10:22:09 UTC - Kristian Andersen: @Kristian Andersen has joined the
channel
----
2019-04-05 12:23:07 UTC - Steve Kim: I see puzzling behavior with the REST API
on Pulsar 2.3.0:
```
$ curl -X GET <http://host/admin/v2/tenants>
["foo","public"]
$ curl -X GET <http://host/admin/v2/tenants/public>
{"adminRoles":[],"allowedClusters":["mycluster"]}
$ curl -X GET <http://host/admin/v2/tenants/foo>
{"reason":"Tenant does not exist"}
```
Why am I able to list the "foo" tenant but not get it directly? Do I have a
configuration error? Is this a known bug?
----
2019-04-05 12:34:35 UTC - Chris Bartholomew: @Steve Kim These APIs all work for
me in 2.3.0. Does "foo" have "mycluster" on its allowedClusters list? That is,
have you configured it?
----
2019-04-05 12:35:45 UTC - Steve Kim: probably not :slightly_smiling_face: i'll
RTFM and configure it correctly. thanks for answering my question.
----
2019-04-05 12:37:05 UTC - Steve Kim: this seems like a minor user experience /
documentation bug. should I file a bug report?
----
2019-04-05 12:47:42 UTC - Steve Kim: Okay. I was able to update the tenant with
a `POST /admin/v2/tenants/foo` request with a JSON body, and now the `GET
/admin/v2/tenants/foo` returns a 200 OK.
----
2019-04-05 12:49:41 UTC - Steve Kim: Is it a bug that the `GET` returned a 404
Not Found when the `allowedClusters` set was empty? I expected a 200 OK with
the response body `{"adminRoles":[],"allowedClusters":[]}`.
----
2019-04-05 13:03:57 UTC - Chris Bartholomew: I agree that it seems like the
wrong response code in this scenario. At minimum, its confusing for folks
getting started with Pulsar. Probably should be tracked by a bug.
----
2019-04-05 14:17:53 UTC - Matteo Merli: Yes, looks indeed like a bug. Please
open an issue. Also if you to take a stab at fixing that, that’s welcome too :)
grinning : Chris Bartholomew
----
2019-04-05 14:39:00 UTC - bossbaby: In pulsar cluster, i was change old service
URL to new URL
When request ```<http://localhost:8080/admin/v2/clusters/us>```
and response
```
{
"serviceUrl": "<http://dns-new-us:8080>",
"serviceUrlTls": "<https://dns-new-us:8443>",
"brokerServiceUrl": "<pulsar://dns-new-us:6650>",
"brokerServiceUrlTls": "<pulsar+ssl://dns-new-us:6651>"
}
```
But in pulsar replication
```
14:32:21.794 [pulsar-io-21-4] WARN
org.apache.pulsar.broker.service.AbstractReplicator -
[<persistent://public/default/test>][singapore -> us] Failed to create
remote producer (org.apache.pulsar.client.api.PulsarClientException:
java.util.concurrent.CompletionException: java.net.UnknownHostException: failed
to resolve 'dns-old-us' after 4 queries ), retrying in 54.853 s
```
Why does pulsar change it?
----
2019-04-05 15:12:20 UTC - Martin Ashby: @Martin Ashby has joined the channel
----
2019-04-05 15:39:23 UTC - John Crawford: @Jerry Peng the only things i see in
the BK logs are these:
```
84931.354: Total time for which application threads were stopped: 0.0001947
seconds, Stopping threads took: 0.0000327 seconds
84931.354: Total time for which application threads were stopped: 0.0001054
seconds, Stopping threads took: 0.0000207 seconds
84946.354: Total time for which application threads were stopped: 0.0001977
seconds, Stopping threads took: 0.0000429 seconds
84946.354: Total time for which application threads were stopped: 0.0001308
seconds, Stopping threads took: 0.0000270 seconds
84961.354: Total time for which application threads were stopped: 0.0002040
seconds, Stopping threads took: 0.0000493 seconds
84961.354: Total time for which application threads were stopped: 0.0001031
seconds, Stopping threads took: 0.0000201 seconds
84976.354: Total time for which application threads were stopped: 0.0002012
seconds, Stopping threads took: 0.0000496 seconds
84976.354: Total time for which application threads were stopped: 0.0001131
seconds, Stopping threads took: 0.0000276 seconds
84991.354: Total time for which application threads were stopped: 0.0001952
seconds, Stopping threads took: 0.0000350 seconds
84991.354: Total time for which application threads were stopped: 0.0001060
seconds, Stopping threads took: 0.0000110 seconds
85006.354: Total time for which application threads were stopped: 0.0003892
seconds, Stopping threads took: 0.0000364 seconds
85006.354: Total time for which application threads were stopped: 0.0001186
seconds, Stopping threads took: 0.0000106 seconds
85021.354: Total time for which application threads were stopped: 0.0002015
seconds, Stopping threads took: 0.0000314 seconds
85021.354: Total time for which application threads were stopped: 0.0001118
seconds, Stopping threads took: 0.0000126 seconds
85036.354: Total time for which application threads were stopped: 0.0002086
seconds, Stopping threads took: 0.0000339 seconds
85036.354: Total time for which application threads were stopped: 0.0001060
seconds, Stopping threads took: 0.0000237 seconds
85051.354: Total time for which application threads were stopped: 0.0002068
seconds, Stopping threads took: 0.0000331 seconds
85051.354: Total time for which application threads were stopped: 0.0000982
seconds, Stopping threads took: 0.0000181 seconds
85066.354: Total time for which application threads were stopped: 0.0002026
seconds, Stopping threads took: 0.0000327 seconds
85066.354: Total time for which application threads were stopped: 0.0001184
seconds, Stopping threads took: 0.0000202 seconds
85081.354: Total time for which application threads were stopped: 0.0002062
seconds, Stopping threads took: 0.0000334 seconds
```
----
2019-04-05 15:39:27 UTC - John Crawford: is that normal?
----
2019-04-05 15:46:35 UTC - Martin Ashby: :wave: Good afternoon, I'm hoping to
talk about how to backup & restore an Apache Pulsar deployment.
----
2019-04-05 16:11:44 UTC - Steve Kim: I opened
<https://github.com/apache/pulsar/issues/3987> and will investigate
----
2019-04-05 16:22:52 UTC - Martin Ashby: I'm intending to run a 3 node cluster
for high availability in case of a single machine failure, however this is
single-site at the moment, so I need an offsite backup strategy. I couldn't
find much documentation on the topic in
<http://pulsar.apache.org/docs|pulsar.apache.org/docs>
----
2019-04-05 16:33:52 UTC - Anqing Xu: @Anqing Xu has joined the channel
----
2019-04-05 16:47:01 UTC - Ali Ahmed: Hi Martin instead of backup you will
looking more likely to replicate you cluster live , so search for geo
replication in pulsar
----
2019-04-05 17:25:40 UTC - Martin Ashby: Thanks for responding. That sounds
great, but it doesn't fit my requirement: I don't have a secondary cluster in a
different geographical location at the moment.
----
2019-04-05 17:26:51 UTC - Anqing Xu: Hi I'm trying to make a decision between
RabbitMQ and Kafka and then I bumped into Apache Pulsar. Pulsar in theory is
superior! But how mature is it? I read the benchmark data that claims Pulsar
has better latency AND throughput than Kafka. Is that true in general? I read
Jack Vanlightly's articles on Pulsar. It seems that Pulsar has better
technologies but I'm not sure it is mature enough for mission critical
applications. I built financial electronic trading platforms that handle
hundreds of billions of dollars and I need to make sure the technology is
really stable and proven.
----
2019-04-05 17:30:37 UTC - Anqing Xu: Also two specific questions: Does Pulsar
client handle broker node failover transparently? We use both Java and
Node.js. Another questions is about management APIs an tools. First of all
does Pulsar client libraries allow me to create and destroy topics and
subscriptions programmatically? We use RabbitMQ, which has APIs that allow me
to create exclusive queues for individual users easily. I wonder whether
Pulsar has the same support as a replacement for RabbitMQ. I also wonder
whether Pulsar has a robust management web console that allows to manage and
monitor Pulsar cluster easily
----
2019-04-05 17:37:51 UTC - Matteo Merli: Pulsar has been in production at Yahoo
and other companies at large scale for multiple years, serving critical
(directly revenue impacting) use cases (eg: Yahoo mail, finance, Ads systems
and many more).
Many other companies have adopted it in production in the last couple of years.
Pulsar provide much stronger durability and availability guarantees compare to
Kafka and RabbitMQ. Messages are actually synced to disk on multiple nodes
before an ack is returned.
> read the benchmark data that claims Pulsar has better latency AND
throughput than Kafka. Is that true in general?
Benchmark are of course show casing one single case, though the differences for
throughput and latency are deriving from the architectural differences between
Pulsar and Kafka. (eg: Pulsar, with BookKeeper, can do quorum write and ignore
the slowest responding storage node, while Kafka (with acks=all) will always
have to wait for it.
----
2019-04-05 17:42:05 UTC - Matteo Merli: > Does Pulsar client handle broker
node failover transparently?
Yes, application just need to reason in term of producer or consumer API. The
Pulsar client library is taking care of the underlying service discovery and
reconnections.
> We use both Java and Node.js.
There is a NodeJS library in the works:
<https://github.com/apache/pulsar-client-node>
> First of all does Pulsar client libraries allow me to create and destroy
topics and subscriptions programmatically?
Take a look at admin API (there’s REST, Java and CLI support).
> I also wonder whether Pulsar has a robust management web console that
allows to manage and monitor Pulsar cluster easily
Currently the available dashboard is read-only. Metrics are exposed in
Prometheus format.
----
2019-04-05 18:29:55 UTC - John Crawford: sorry to bother you @Jerry Peng but do
those logs up there seem normal? we’re still encountering some issues with the
cluster and trying to get to the bottom of it
----
2019-04-05 18:35:13 UTC - Jerry Peng: @John Crawford these are GC logs for the
java JVM
----
2019-04-05 18:35:40 UTC - Jerry Peng: the numbers for GC don’t seem out of the
ordinary
----
2019-04-05 18:36:01 UTC - Jerry Peng: what are the issues you are experiencing?
----
2019-04-05 19:36:49 UTC - John Crawford: yesterday we were seeing read timeouts
from the broker to bookkeeper trying to see ledgers. i got us a little
unblocked by essentially clearing all the backlogs out (not wanting to do that
on a regular basis obviously), but today we’re seeing a different issue where
we’re not acking but i’m not 100% sure it’s pulsar related
----
2019-04-05 19:37:10 UTC - John Crawford: is there another place i should look
for bookeeper logs? i was looking in the k8s pod logs for the logs above
----
2019-04-05 21:01:44 UTC - John Crawford: we’ve isolated it to a basically
unresponsive topic. can’t publish or get stats from it
----
2019-04-06 00:19:29 UTC - Mandi Goddard: @Mandi Goddard has joined the channel
----
2019-04-06 00:20:16 UTC - Mandi Goddard: I am looking for a way to message a
large group of students. Can I do that here?
----
2019-04-06 00:21:21 UTC - Matteo Merli: Do you mean with Pulsar? Or in this
Slack?
----
2019-04-06 00:39:00 UTC - Mandi Goddard: with pulsar
----
2019-04-06 00:39:14 UTC - Mandi Goddard: does it work well with large groups?
----
2019-04-06 00:49:58 UTC - Jon Bock: @Mandi Goddard Pulsar is a messaging system
in the sense that, for example, a Java developer could write an application
that would use it to pass custom data packages to other applications. It is
not a messaging system in the sense of sending e-mail and SMS messages, which
might be what you’re looking for?
----
2019-04-06 01:13:06 UTC - Guangzhong Yao: The pulsar broker is stateless, and
the [bookkeeper](<http://bookkeeper.apache.org/>) is responsible for stateful
storage, so as long as the data is kept on the bookkeeper cluster, the pulsar
can works well.
----
