"max.poll.records" config property can be used to limit the number of records returned in each consumer poll() method call.
On Fri, Sep 23, 2016 at 10:49 PM, Ramanan, Buvana (Nokia - US) < buvana.rama...@nokia-bell-labs.com> wrote: > Hello, > > Do Kafka protocol & KafkaConsumer (java) client address the following need? > > The caller specifies that it needs N number of records with a max wait > time of Tn milliseconds. If N records are available within Tn, the records > are returned to the caller. If Tn expires, then the caller gets whatever > number of records (< N) that were available. > > If this functionality not readily supported by KafkaConsumer, can someone > please give me the logic I can use at the caller to simulate the scenario. > I need to execute an application as soon as N records (or less, in case Tn > is reached) become available (at the broker itself - that is, N records > since the time of last fetch). > > Kafka Consumer properties allows us to specify fetch.max.wait.ms and > fetch.min.bytes, with which you can indirectly achieve the above > functionality in case you know the periodicity & the record size of the > topic of interest. But it is too restrictive and I am looking for > alternatives. > > Thanks, > Buvana >
