Thanks for the reply. I read KIP <https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=127406453#KIP517:Addconsumermetricstoobserveuserpollbehavior-poll-idle-ratio-avg>. Can you let me know if I observe poll-idle-ration.avg equal to 1.0 then does that mean my fetch.max.bytes or max.partition.fetch.bytes is not enough and I have to increase them? If not what could be the reason that may cause poll-idle-ratio-avg to approach 1.0?
Can you let me know what On Sat, 2 Dec, 2023, 07:05 Haruki Okada, <ocadar...@gmail.com> wrote: > Hi. > > `max.poll.records` does nothing with fetch requests (refs: > > https://kafka.apache.org/35/documentation.html#consumerconfigs_max.poll.records > ) > > Then, how many records will be returned for single fetch request depends on > the partition-leader assignment. (note: we assume follower-fetch is not > used here) > If all partition leaders are in the same broker, 40MB (2MB * 20 partition) > will be returned for a single fetch request. > > 2023年11月30日(木) 17:10 Debraj Manna <subharaj.ma...@gmail.com>: > > > The doc states that fetch.max.bytes & max.partition.fetch.bytes > > > > are not absolute maximum. If the first record batch in the first > non-empty > > > partition of the fetch is larger than this limit, the batch will still > be > > > returned to ensure that the consumer can make progress. > > > > > > I am getting a bit confused. > > > > Let's say I have a configuration like below with sufficient messages in > > each partition > > > > > > - Partitions in a topic 20 > > - Single message size 2MB > > - Consumers 5 > > - max.poll.records 20 > > - fetch.max.bytes 50 MB > > - max.partition.fetch.bytes 1 MB. > > > > The broker config message.max.bytes and max.message.bytes is set to > default > > 100MB > > > > If the consumer does a poll will it receive 20 records? If yes then there > > is no significance of fetch.max.bytes & max.partition.fetch.bytes with > > max.poll.records? > > > > > > - Java Kafka Client - 3.5.1 > > - Kafka Broker - 2.8.1 > > > > > -- > ======================== > Okada Haruki > ocadar...@gmail.com > ======================== >
