I support enabling it in 2.11.0 Without the `Acknowledgement in Batch Level index` Users will see a weird behavior `the acked messages dispatched to the consumer again` Many new users thought it was a bug.
For the default configuration, we should be more friendly to new users, because they have too much pulsar experience. For pulsar veterans, they can disable it to reduce memory consumption, but they know what the expected behavior is after diable it. And regarding memory consumption, we can optimize in later versions. Thanks, Penghui On Mon, May 23, 2022 at 5:54 PM Enrico Olivelli <eolive...@gmail.com> wrote: > Asaf, > I don't have numbers to say something about the runtime impact. > > It is a important feature and I am +1 to enabling it by default > > You can send a PR in the meantime. > I am not sure about the impact on the tests, maybe we will have to > take a look carefully in order to not introduce new flaky tests > > Enrico > > Il giorno dom 22 mag 2022 alle ore 18:11 Asaf Mesika > <asaf.mes...@gmail.com> ha scritto: > > > > Resurfacing this as it seems (to me) an important correction to be made. > > > > Would love your opinions on this. > > > > Thanks! > > > > Asaf > > > > On Wed, May 11, 2022 at 1:16 PM Asaf Mesika <asaf.mes...@gmail.com> > wrote: > > > > > Hello all, > > > > > > I have a question regarding a feature introduced in 2.6.0 called > Negative > > > Acknowledgement in Batch Level index, described in PIP-54 > > > < > https://github.com/apache/pulsar/wiki/PIP-54:-Support-acknowledgment-at-batch-index-level > > > > > . > > > > > > In short: Before this feature, if you would write in batches (in > producer) > > > to Pulsar (which makes sense most would for performance reasons), then > a > > > consumer sending a negative acknowledgment for 1 message in the batch > (say > > > we have 500 in a single batch), would cause the consumer to get the all > > > messages in the batch (500) redelivered as if they were not > acknowledge. > > > > > > PIP-54 fixes that by keeping the ack per message in a batch using a > bitmap > > > index. The caveats are more memory consumption since the broker keeps > those > > > bitmaps in-memory for any inflight batch. > > > > > > With PIP-54 Pulsar IMO becomes "complete" in that it acts the way you > > > would expect it to be, in normal circumstances (as opposed to > disconnects, > > > machine crashing, etc). > > > > > > This feature was introduced in 2.6.0, roughly 2 years ago, and is off > by > > > default. > > > > > > I was wondering what the developers community thoughts on turning it > on by > > > default? Has anyone experienced any performance degradation to it? > Have you > > > turned it on in your clusters? > > > > > > Thanks a lot for your time. > > > > > > Asaf Mesika > > > >
