Hi,
Sorry, i have one more query regarding consumer.poll()
As KafkaConsumer fetches record when it is available irrespective of
poll(timeout), does KafkaConsumer splits the poll(timeout) in to multiple
intervals and checks Kafka Server for any messages.
Eg., poll timeout is 10min , does it split into 1min intervals of 10 times
do{
if( pollForMessageIfAny()){
return;
}
Thread.sleep(1000);
timeremaining = curretime - timeout;
}while(timeremainning > 0)
Thanks and Regards
A.SathishKumar
On Wed, Aug 3, 2016 at 5:46 PM, sat <sathish.al...@gmail.com> wrote:
> Hi,
>
>
> Thanks for your reply Kamal and Oleg.
>
>
> Thanks and Regards
>
> A.SathishKumar
>
> >Also keep in mind that unfortunately KafkaConsumer.poll(..) will deadlock
> >regardless of the
> >timeout if connection to the broker can not be established and won't react
> >to thread interrupts.
> >This essentially means that the only way to exit is to kill jvm. This is all
> >because Kafka
> >fetches topic metadata synchronously before timeout takes effect.
> >While it is my understanding that the reason for it is there is a background
> >thread attempting
> >to reconnect in the event of temporary broker outage, it doesn't help if you
> >accidentally
> >specified wrong broker url.
>
> Oleg
>
> > On Aug 2, 2016, at 10:27, Kamal C <kamaltar...@gmail.com> wrote:
> >
> > See the answers inline.
> >
> >> On Tue, Aug 2, 2016 at 12:23 AM, sat <sathish.al...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> I am new to Kafka. We are planning to use Kafka messaging for our
> >> application. I was playing with Kafka 0.9.0.1 version and i have following
> >> queries. Sorry for asking basic questions.
> >>
> >>
> >> 1) I have instantiated Kafka Consumer and invoked
> >> consumer.poll(Long.MAX_VALUE). Although i have specified timeout as
> >> Long.MAX_VALUE, i observe my consumer to fetch records whenever the
> >> publisher publishes a message to a topic. This makes me wonder whether
> >> Kafka Consumer is push or pull mechanism. Please help us understand the
> >> logic of consumer.poll(timeout).
> >
> > Fetches the data from the topic, waiting up to the specified wait time *if
> > necessary *for a record to become available.
> > Kafka Consumer by design is pull mechanism.
> >
> > Take a look into Kafka Consumer java docs[1]. It's explained in detail.
> >
> >
> >> 2) What are the pros and cons of poll for long timeout vs short timeout.
> >>
> >> Short Timeout
> >
> > Pros:
> > - On shutdown, if no data available in the topic -- Shutdown will be quick
> >
> > Cons:
> > - Number of network trips will be high
> >
> >
> >>
> >> Thanks and Regards
> >> A.SathishKumar
> >
> > [1]:
> > https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
> >
> > -- Kamal
>
>
> On Mon, Aug 1, 2016 at 11:53 AM, sat <sathish.al...@gmail.com> wrote:
>
>> Hi,
>>
>> I am new to Kafka. We are planning to use Kafka messaging for our
>> application. I was playing with Kafka 0.9.0.1 version and i have following
>> queries. Sorry for asking basic questions.
>>
>>
>> 1) I have instantiated Kafka Consumer and invoked
>> consumer.poll(Long.MAX_VALUE). Although i have specified timeout as
>> Long.MAX_VALUE, i observe my consumer to fetch records whenever the
>> publisher publishes a message to a topic. This makes me wonder whether
>> Kafka Consumer is push or pull mechanism. Please help us understand the
>> logic of consumer.poll(timeout).
>>
>> 2) What are the pros and cons of poll for long timeout vs short timeout.
>>
>>
>> Thanks and Regards
>> A.SathishKumar
>>
>>
>
>
> --
> A.SathishKumar
> 044-24735023
>
--
A.SathishKumar
044-24735023
