Hi Lin,
For the producer_request_hold policy, users should set the infinite publish
timeout.
It is not like moving the create producer timeout to publish message
timeout.
The key point that we want to improve is to provide consistent behavior to
users.
For example, we have a producer that has
It's best to keep the behavior of producer_request_hold as it is, instead
of changing it. The current behavior is, the publisher holds at most
operationTimeoutMs (client side configuration) after which throwing the
exception. This behavior is consistent between ongoing clients and clients
with
I agree that metrics on these "blocked" producers are helpful to end users.
> Another concern is that if the topic is unloaded to another broker, we have
> to notify the client to reconnect, I'm not sure if
> `waitingExclusiveProducers` does that or if I missing some logic.
If I remember
Great idea, Michael
> 1. A new producer connecting to a topic that has exceeded its quota.
> This case is trivial because the broker tells the producer it is
> connected but it cannot send any messages (i.e. producer_ready=false),
> and the client holds the producer's future until the broker
Great points, Penghui.
> To optimize, I think we can only let the producer connect to the broker
> and the broker should tell the producer the backlog is exceeded.
In looking at the `CommandProducerSuccess` protobuf message, we
already have the `producer_ready` boolean field. It was added for
> The producer fails
the pending messages when the policy is producer_exception and the
producer does nothing when the policy is producer_request_hold
Eventually, it will fail [0] the user's create producer request because of
the operation timeout [1].
> The primary advantage for this solution
> IMO, we should allow the producer to connect the topic.
I actually think the current producer_request_hold feature works based
on disconnecting a producer and only letting it connect when the topic
no longer exceeds its quota.
> It looks like we do not support the `producer_request_hold`
Hi, Penghui
I support your opinion.
It looks like we do not support the `producer_request_hold` semantics. We just
easily use the same behaviour like `producer_exception`. Maybe it's a missing
feature.
Best,
Mattison
On 2022/08/11 05:28:25 PengHui Li wrote:
> Hi all,
>
> Pulsar has a
Hi all,
Pulsar has a backlog quota policy `producer_request_hold` which will hold
the message
publish request. It is very useful for some data sync scenarios. It looks
like the producer
waiting for the consumer to process the data.
But the new producer is not allowed to connect to the topic
