2019-01-17 09:25:25 UTC - Vincent Ngan: What is the maximum size of a message
allowed by Pulsar?
----
2019-01-17 09:25:48 UTC - Ali Ahmed: 5 mega bytes
----
2019-01-17 09:26:48 UTC - Vincent Ngan: Is this a hard limit? Can I configure
it to a larger value?
----
2019-01-17 10:47:42 UTC - Ganga Lakshmanasamy: @Karthik Ramasamy @Ravi Said
that you have some reference for real-time consumption of logs from a https
endpoint and ingesting into S3/ elastic using streamlio. Can you please share
the details.
----
2019-01-17 11:08:54 UTC - jia zhai: This is limited by BookKeeper. and it is
hard coded in pulsar currently.
----
2019-01-17 11:51:27 UTC - Harald Gustafsson: When deleting a namespace all
topics need to be deleted first, that can be done with force even when
producers or consumers are connected. The problem is that producers will
reconnect after 0.1 seconds, and then the namespace is still not empty when
trying to delete it. Is there a way of informing producers and consumers that
when forcefully disconnected they should not reconnect or alternatively is
there some broker admin interface to inform all clients on a topic to
disconnect?
----
2019-01-17 12:17:33 UTC - jia zhai: @Harald Gustafsson Currently, seems there
is no such interface.
----
2019-01-17 12:20:15 UTC - Harald Gustafsson: @jia zhai Was thinking if I
removed access right from namespace first, would that prevent clients from
reconnecting?
----
2019-01-17 12:25:15 UTC - jia zhai: :+1: you could have a try.
----
2019-01-17 12:25:47 UTC - jia zhai: In the code, when do reconnection, it will
verify the authorize
----
2019-01-17 13:00:54 UTC - Vincent Ngan: Thx
----
2019-01-17 14:26:33 UTC - Gofu: @Gofu has joined the channel
----
2019-01-17 14:29:39 UTC - Gofu: Helo guys, can someone tell me if there is some
documentation of the REST admin api, that shows the schema of the http body?
Here
<https://pulsar.apache.org/docs/latest/reference/RestApi/#/admin/persistent/:property/:cluster/:namespace/:destination/partitions>,
I can't see the body schema. I am assuming that's missing because when I do
the request the response message is: The request entity cannot be empty.
----
2019-01-17 14:52:33 UTC - Paul van der Linden: I might be missing something
----
2019-01-17 14:52:43 UTC - Paul van der Linden: But I deployed a pulsar cluster
to kubernetes
----
2019-01-17 14:52:54 UTC - Paul van der Linden: in a different namespace then
"default"
----
2019-01-17 14:53:36 UTC - Paul van der Linden: somehow the broker reports it's
url as if it is running in the default namespace:
`broker.default.svc.cluster.local` instead of
`broker.test-pulsar.svc.cluster.local`
----
2019-01-17 15:10:22 UTC - Paul van der Linden: NVM, found it in the metadata
init manifest
+1 : jia zhai
----
2019-01-17 15:24:50 UTC - Sijie Guo: @Gofu you are accessing the old website.
you can find the rest api documentation here :
<https://pulsar.apache.org/en/admin-rest-api/>
----
2019-01-17 15:44:43 UTC - Matteo Merli: Yes, removing the permissions will
immediately disconnect existing producers/consumers that have no more access
----
2019-01-17 15:55:24 UTC - Gofu: thank you so much
----
2019-01-17 16:27:48 UTC - Gofu: @Sijie Guo even in
<https://pulsar.apache.org/en/admin-rest-api/> the endpoint doesnt have body
specification in the endpoint "Create a partitioned topic" per example.
----
2019-01-17 16:32:10 UTC - Ganga Lakshmanasamy: We are having problem in sending
and receiving JSONObjects. The json values are converted into objects from
string/array. is it something expected? Here is an example. This is the data
sent {"id":"Thgkkg"} , data received {"id":"java.lang.Object@69cccc31"} Here is
the consumer creation code
consumer = client.newConsumer(Schema.AVRO(JSONObject.class))
.topic(TOPIC_NAME)
.subscriptionType(SubscriptionType.Shared)
.subscriptionName(SUBSCRIPTION_NAME)
.subscribe();
----
2019-01-17 16:34:23 UTC - Sijie Guo: ah, i see. I think swagger annotation
doesn’t render the rest api page correctly.
the post body should be an integer - the number of the partition.
----
2019-01-17 16:35:36 UTC - Sijie Guo: if your payloads are json, you should use
json schema not avro schema, no?
----
2019-01-17 16:36:52 UTC - Gofu: @Sijie Guo I have tried without success
{
"partitions": 2
}
----
2019-01-17 17:02:38 UTC - Sijie Guo: @Gofu just post with ‘2’
----
2019-01-17 17:08:54 UTC - Gofu: @Sijie Guo thank you very much, the only way I
can see it is looking into pulsar api code?
----
2019-01-17 17:14:25 UTC - Sijie Guo: @Gofu - most of the rest apis might
already have documented the post body. however I think swagger annotation
doesn’t handle well (or we misuse it), this rest api doesn’t seem to have
primitive post payload documented. for such case, pulsar api code might be an
easier way for you to figure out. or ping us on slack channel
:slightly_smiling_face:
for this particular issue, do you mind creating a github issue for us? we will
follow up to address the documentation issue.
----
2019-01-17 17:15:24 UTC - Gofu: Yes sure @Sijie Guo, thanks for the support
----
2019-01-17 18:52:17 UTC - Ravi: any luck on the GUI for pulsar in a container ?
----
2019-01-17 18:59:49 UTC - chris: @Ravi there is an image with the pulsar gui in
it (apachepulsar/pulsar-dashboard). here is an example docker compose file to
run it locally
<https://github.com/apache/pulsar/blob/master/docker-compose/standalone-dashboard/docker-compose.yml>
----
2019-01-17 19:32:08 UTC - Louis Lapierre: @Louis Lapierre has joined the channel
----
2019-01-17 20:41:15 UTC - Emma Pollum: Currently there are some topic stats
that are not available through prometheus, only the command line. Is there
anything in progress to combine these?
----
2019-01-17 20:51:56 UTC - Ivan Kelly: @Emma Pollum which stats in particular?
some stats changes have gone in recently
----
2019-01-17 20:55:45 UTC - Emma Pollum: I'm looking for average message size and
message rate expired
----
2019-01-17 21:07:12 UTC - Ivan Kelly: hmm, looks like the recent changes were
all around functions
----
2019-01-17 22:00:30 UTC - Emma Pollum: @Ivan Kelly Are there any plans to put
these into prometheus?
----
2019-01-17 23:38:32 UTC - Jerry Peng: @Grant Wu after this PR gets merged, the
issue you described in regards to getting the actual topic name of a message
when subscribing to a regex topic will be fix:
<https://github.com/apache/pulsar/pull/3384>
----
2019-01-17 23:41:07 UTC - Grant Wu: I saw that, great! :clap:
----
2019-01-18 00:16:13 UTC - Grant Wu: So I’m reading
<https://pulsar.apache.org/docs/en/functions-guarantees/> and I was wondering
if I could get a little more clarity on how exactly we know when a function has
successfully processed an input
----
2019-01-18 00:17:58 UTC - Grant Wu: @Jerry Peng I’ve skimmed through
<https://streaml.io/blog/exactly-once> and it seems like a great overview of
the concepts of effectively once processing
----
2019-01-18 00:18:17 UTC - Grant Wu: But I think “how do we know a Pulsar
Function succeeded” isn’t really described here
----
2019-01-18 00:18:24 UTC - Grant Wu: Is it that no exception was raised?
----
2019-01-18 00:18:28 UTC - Grant Wu: (For Python, at least)
----
2019-01-18 00:18:32 UTC - Grant Wu: cc @Jonathan Budd
----
2019-01-18 00:30:48 UTC - Emma Pollum: When setting cluster properties, what is
the difference between --url and --broker-service-url
----
2019-01-18 00:41:32 UTC - Emma Pollum: And what is a peer cluster?
----
2019-01-18 01:09:27 UTC - Sijie Guo: you can check this out -
<https://github.com/apache/pulsar/wiki/PIP-8:-Pulsar-beyond-1M-topics>
----
2019-01-18 01:09:55 UTC - Sijie Guo: --broker-service-url is the pulsar binary
protocol : pulsar://<dns>:6650
----
2019-01-18 01:10:08 UTC - Sijie Guo: --url is the http rest endpoint:
http://<dns>:8080
----
2019-01-18 04:51:32 UTC - bossbaby: how prevent pulsar client subcribe
subcription that they not own it. I think i use cert but im not found document
about it
----
2019-01-18 05:59:07 UTC - jia zhai: @bossbaby FYI：
<http://pulsar.apache.org/docs/en/security-overview.html#role-tokens>
----
2019-01-18 07:21:29 UTC - Ganga Lakshmanasamy: Thanks. Will check this. @Ravi
FYI
----
2019-01-18 08:09:42 UTC - xiaolong.ran: @xiaolong.ran has joined the channel
----
