+1
On 2021/12/29 02:29:21 Haiting Jiang wrote: > This is the voting thread for PIP-131. It will stay open for at least 48h. > > https://github.com/apache/pulsar/issues/13544 > > The discussion thread is > https://lists.apache.org/thread/c63d9s73j9x1m3dkqr3r38gyp8s7cwzf > > ## Motivation > > Currently, chunk messages producing fails if topic level maxMessageSize is > set [1]. The root cause of this issue is because chunk message is using > broker level maxMessageSize as chunk size. And topic level maxMessageSize is > always <= broker level maxMessageSize. So once it is set, the on-going chunk > message producing fails. > > ## Goal > > Resolve topic level maxMessageSize compatibility issue with chunking messages. > > ## Implementation > > Current best solution would be just skipping topic level maxMessageSize check > in org.apache.pulsar.broker.service.AbstractTopic#isExceedMaximumMessageSize. > Topic level maxMessageSize is introduced in [2], for the purpose of "easier > to plan resource quotas for client allocation". And IMO this change will not > bring further complex into this. > > ## Reject Alternatives > > Add a client side topic level maxMessageSize and keep it synced with broker. > > Required changes: > - [client] Add a new field > org.apache.pulsar.client.impl.ProducerBase#maxMessageSize to store this > client side topic level maxMessageSize. > - [PulsarApi.proto] Add a new field maxMessageSize in the > CommandProducerSuccess for the initial value of ProducerBase#maxMessageSize > - [PulsarApi.proto] Add a new Command like > CommandUpdateClientPolicy{producerId, maxMessageSize} to update > ProducerBase#maxMessageSize when topic level maxMessageSize is updated. > Further more, some other data consistency issues need be handled very > carefully when maxMessageSize is updated. > This alternative is complex but can also solve other topic level > maxMessageSize issue [3] when batching is enabled (non-batching case is > solved with PR [4]). > > [1] https://github.com/apache/pulsar/issues/13360 > [2] https://github.com/apache/pulsar/pull/8732 > [3] https://github.com/apache/pulsar/issues/12958 > [4] https://github.com/apache/pulsar/pull/13147 > > Thanks, > Haiting Jiang >
