Hi We cannot see the image attached to this email. Maybe you can create a JIRA ticket and attach the screenshots there.
On Wed, Jan 29, 2025 at 8:38 PM Manjunath.S Horapeti < manjunaths.horap...@gmail.com> wrote: > Hi Team, > > I'm reaching out regarding the understanding of behaviour of " > pollTimeoutMs" and the discrepancy observed. > > [image: image.png] > > As per the documentation above from the camel kafka component page ( > https://camel.apache.org/components/4.8.x/kafka-component.html) 4.8.x > version, the pollTimeoutMs works in tandem with "maxPollRecords", to either > poll "maxPollRecords" or block for a maximum of "pollTimeOutMs". > > But the behaviour observed was that the camel route kept waiting > until "maxPollRecords" count was reached and then processed further. > for example: our route is as follows > > from("kafka:topic-name?brokers=brokers&groupId=groupid&pollTimeoutMs=10000&batching=true&maxPollRecords=50000").bean(this, > "methodName"); > > This route always waits until 50000 records are present in the topic and > then processes further and ignoring pollTimeoutMs of 10000 (10 seconds). > i.e. if the producer is producing msgs at a rate of 50-100 msgs per second, > then application waits for nearly 500-1000 seconds before proceeding > further that is until 50000 record count is met. > > We believe that the below code mentioned in the > "KafkaRecordBatchingProcessor" is never executed as there is always one or > two messages in poll. and hence method - "hasExpiredRecords" with condition > consumerRecords.isEmpty() is always false. > We believe this is making the application wait until maxPollRecords > (50000) is reached and then proceed further. > [image: image.png] > > [image: image.png] > > Can you please help us by letting us know if the above behaviour is as > expected and if so then can you let us know how to pull Y messages from the > topic or write whatever messages are received within X seconds? (exact > behaviour mentioned in the above Batching Consumer documentation). > > Your response would be much appreciated. > > > -- > Thank you, > Manjunath S Horapeti > -- Claus Ibsen ----------------- @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
