2019-10-18 09:43:33 UTC - Martin Kunev: Hello,
My team is having a problem with pulsar 2.4.0. We have a topic that has
replication clusters properly set, but published messages only arrive to some
of the clusters. The other clusters never receive the messages. With pulsar
2.3.1 there i
s no such issue. What can be the cause of this problem?
----
2019-10-18 09:57:58 UTC - AlifRuliarso: @AlifRuliarso has joined the channel
----
2019-10-18 13:24:51 UTC - Junli Antolovich: @Chris Bartholomew I followed your
command and it worked for me in PowerShell! Thanks so much for the help,
appreciated.
----
2019-10-18 15:33:31 UTC - Chris Bartholomew: @Junli Antolovich Glad to hear it
is working for you now. Good luck with your POC.
+1 : Junli Antolovich
----
2019-10-18 17:56:55 UTC - Retardust: does pulsar_subscription_back_log is
proper metric to monitor consumer lag?
it's increase without incoming messages, weird))
----
2019-10-18 18:07:38 UTC - Oleg Kozlov: that 's a bit different, i do want to
store messages forever, but get them removed as soon (or shortly after) they
are ack'd, like in a typical queue. I get that behavior by creating a shared
subscription ahead of time, which is fine, like i said, it's not a blocker,
just a slight complication
----
2019-10-18 18:08:13 UTC - Oleg Kozlov: btw, do subscriptions w/o connected
consumers ever expire/dispappear ?
----
2019-10-18 18:09:25 UTC - Matteo Merli: By default, no. There’s a broker
setting to drop subscriptions for which a consumer didn’t connect in some time.
eg. 2 weeks
----
2019-10-18 18:43:14 UTC - Chris DiGiovanni: About to file a bug, though thought
I would see if anyone is familiar with this issue:
When I hit the api endpoint /admin/v2/non-persistent//{tenant}/{namespace} to
list non-persistent topics, the request just hangs. If I hit the broker to 2-3
times with this request, the broker API becomes unresponsive and needs to be
restarted. This happens in our lab deployment running 2.4.1 and in our
production environment running 2.3.0. When we run the standalone instance we
cannot replicate.
----
2019-10-18 18:44:04 UTC - Chris DiGiovanni: We found this while trying to use
Presto on Pulsar. Even though we don't have any non-persistent topics, Presto
still wants to list non-persistent topics.
----
2019-10-18 18:45:28 UTC - Chris DiGiovanni: I should also mention I never see
the GET in the logs of the broker for the request...
----
2019-10-18 18:50:15 UTC - Chris DiGiovanni: Seems related to this:
<https://github.com/apache/pulsar/issues/3877>
----
2019-10-18 18:57:27 UTC - Sijie Guo: Not aware of any similar issue.
I would suggest taking following steps to investigate.
1) use `pulsar-admin topic stats` to see if there is any backlog for the
replication subscription.
2) check if the broker log to see if there is any errors.
----
2019-10-18 18:59:23 UTC - Sijie Guo: yes. can you use `pulsar-admin topic
stats` to get the detailed stats about produce and consume rate to see if there
are messages being produced
----
2019-10-18 19:20:29 UTC - Alex: Hello, I have a question regarding schema
management in Pulsar.
Does it checks for added/removed fields in JSON/Avro schemas?
How does it works for custom SchemaType?
----
2019-10-18 19:25:26 UTC - Alex: Also I can't figure out what is the purpose of
SchemaInfo.schema. How it's used by Pulsar?
----
2019-10-18 19:52:29 UTC - Raman Gupta: Does anyone else have an issue with the
producer `sequenceId` being only a `long` value? You can't even stuff a `uuid`
into that, and it limits the utility of this to sequential producers rather
than to any unique value.
----
2019-10-18 19:55:55 UTC - Matteo Merli: well, a sequence hints that the ids are
going to be sequentials :slightly_smiling_face: . A UUID won’t have that
property
----
2019-10-18 20:00:37 UTC - Chris DiGiovanni: Created the following bug:
<https://github.com/apache/pulsar/issues/5417>
----
2019-10-18 20:05:26 UTC - Sijie Guo: it does compatibility check for Pulsar
supported schemas.
----
2019-10-18 20:06:18 UTC - Sijie Guo: if you are introducing your own schema
type, you can implement your own schema compatibility checker.
----
2019-10-18 20:06:56 UTC - Sijie Guo: `schema` field is used for storing schema
definition. currrent all the struct schemas in Pulsar (avro, json, protobuf) is
using avro specification for defining schema.
----
2019-10-18 20:13:34 UTC - Alex: @Sijie Guo so my current understanding is the
following (please correct me if I'm wrong):
1. If `Schema.type` is (avro/json/protobuf), `schema` field MUST contain
avro-based schema definition;
2. If `Schema.type` is NONE, I can specify any data in `schema` field and then
use it in my compatibility checker.
----
2019-10-18 20:21:06 UTC - Daniel Moran: @Daniel Moran has joined the channel
----
2019-10-18 20:24:11 UTC - Daniel Moran: Hi all, my team is evaluating Pulsar as
a replacement for Kafka. One thing that worries us is the lack of support for
transactional processing;
----
2019-10-18 20:24:24 UTC - Daniel Moran: but I see there’s an issue for adding
that functionality
----
2019-10-18 20:24:34 UTC - Daniel Moran: is there a planned date for when the
txn logic will be available?
----
2019-10-18 20:46:05 UTC - David Kjerrumgaard: @Daniel Moran That is tentatively
slated for the 2.5 release.
----
2019-10-18 20:47:02 UTC - Daniel Moran: is there a rough date for when that
release is expected?
----
2019-10-18 20:52:55 UTC - David Kjerrumgaard:
<https://github.com/apache/pulsar/milestones>
----
2019-10-18 20:53:13 UTC - David Kjerrumgaard: You can track the progress here.
Currently, it is 19 days overdue :smiley:
tada : Daniel Moran
----
2019-10-18 20:59:34 UTC - Daniel Moran: thank you!
----
2019-10-18 22:48:39 UTC - Luke Lu: Spot a new error in the broker log today:
`2019-10-18 21:04:55,693 bookkeeper-ml-workers-OrderedExecutor-3-0 ERROR
Recursive call to appender RoutingAppender` Any explanations?
----
2019-10-18 23:05:41 UTC - Luke Lu: Hey guys, is there anyway to move an
existing topic to a different namespace, while retaining all its retained data
(from the retention policy)?
----
2019-10-18 23:06:54 UTC - Matteo Merli: Not directly, though you could
re-publish all the data into the new topic
----
2019-10-18 23:13:21 UTC - Luke Lu: Understood. Was wondering about any secret
way to change some metadata without moving data.
----
2019-10-18 23:48:26 UTC - Oleg Kozlov: thank you
----
2019-10-19 00:33:54 UTC - Alex: When subscriptions/cursors got deleted from
Pulsar? Is it ok to create lots of them?
Basically I want to have a topic with disabled expiration and re-read it every
time my application starts (and rebuilds it state from the whole topic content).
----
2019-10-19 01:10:57 UTC - Alex: I just figured out that I can read topic
without sending acks...
----
2019-10-19 01:18:35 UTC - Sijie Guo: 1 is correct
----
2019-10-19 01:20:02 UTC - Sijie Guo: 2 is not. if Schema.type is NONE, pulsar
doesn’t interpret data. I was talking if you want to introduce a new custom
type, you can add a new type, add your own schema data in `schema` field and
implement a compatibility checker to validate the schema.
----
2019-10-19 01:25:11 UTC - Sijie Guo: seems like a bug from log4j2
----
2019-10-19 01:26:39 UTC - Alex: @Sijie Guo thanks for explanation! It looks not
that simple - I will investigate it further.
----
2019-10-19 01:34:07 UTC - Sijie Guo: yeah ping me if you need any help
----
2019-10-19 08:21:27 UTC - Retardust: It would be nice feature for smooth
production refactoring:) maybe some topic aliasing?
----
