Hi Matteo > Brokers are never going to look into batches, they will just check at the batch header, without decompression, and not dig into the individual messages properties
I agree with this. The PIP-105 doesn't support batch send I think. PIP-105 maybe is not a good case for this changing. Thanks, Xiaoyu Hou Matteo Merli <mme...@apache.org> 于2022年7月13日周三 03:58写道： > -- > Matteo Merli > <mme...@apache.org> > > On Tue, Jul 12, 2022 at 12:54 AM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > > Hello, > > I think that we could implement a small but effective enhancement to > batching. > > > > It may happen that even if you enable batching you come to create > > entries with 1 only message. > > > > Processing batch messages requires a good amount of resources, both on > > the broker and on the client side. > > > > Especially when you are using PIP-105, Broker side filtering, you have > > to unpack (and decompress) the whole entry in order to process the > > very single message. > > > > So my proposal is to change the (Java) producer, to make it produce a > > regular message instead of a batch message if the batch contains only > > 1 message > > I'm ok with the change, though I don't see a big difference in the > cost for clients and brokers. > > Brokers are never going to look into batches, they will just check at > the batch header, without decompression, and not dig into the > individual messages properties. > > Also I don't see the significant additional CPU usage in building a > batch of 1 compared to a non-batched message. > > Keep in mind that the reason we have the 2 formats is purely due to > the fact that in the very beginning we didn't have support for > batches. > > > Especially when you are using PIP-105, Broker side filtering, you have > > to unpack (and decompress) the whole entry in order to process the > > very single message. > > That is more a testament to the fact that the PIP-105 is not a good > model. Whether the message is part of a batch or not, it changes very > little. >
