Re: [DISCUSS] Broker behaviour when returning records

2022-01-26 Thread Megan Niu (BLOOMBERG/ TORONTO)
Hi Luke, Thanks for the answers and pointers to docs! From: show...@gmail.com At: 01/22/22 02:37:59 UTC-5:00To: Megan Niu (BLOOMBERG/ TORONTO ) , dev@kafka.apache.org Subject: Re: [DISCUSS] Broker behaviour when returning records Hi Megan, Answering your questions below: 1. Why can'

Re: [DISCUSS] Broker behaviour when returning records

2022-01-21 Thread Luke Chen
Hi Megan, Answering your questions below: 1. Why can't the broker remove messages from the first batch? So that fetch.max.bytes is not violated. The documentation has answered your question: If the first record batch in the first non-empty partition of the fetch is larger than this limit, the b

[DISCUSS] Broker behaviour when returning records

2022-01-21 Thread Megan Niu (BLOOMBERG/ TORONTO)
Hi all, I have some questions about how brokers batch records to send to consumers. One of the configuration properties for a consumer is fetch.max.bytes. Here's what "Kafka: The Definitive Guide" (by Shapira et al.) says about fetch.max.bytes : "This property lets you specify the maximum bytes