Re: Kafka consumer cluster setup

2016-10-19 Thread sat
s Confluent, Inc > On Oct 16, 2016, at 11:17 PM, sat wrote: > > Hi, > > We are planning to run Kafka servers along with Zookeeper in 3 remote > machines for our cluster setup. > > We have our application with Kafka Consumer and Publisher running in > another/separ

Kafka consumer cluster setup

2016-10-16 Thread sat
Hi, We are planning to run Kafka servers along with Zookeeper in 3 remote machines for our cluster setup. We have our application with Kafka Consumer and Publisher running in another/separate 2 machines. We will be configuring 1 or 2 out of 3 Kafka servers as bootstrap servers in Kafka Consumer

Consumer skip/ignore uncommitted message

2016-08-10 Thread sat
Hi, We have come up with Kafka consumer and we only have 1 consumer in the group. We are using Kafka 0.9. We are able to consume messages as soon as we start polling, however for some reason our consumer gets crashed after processing the message but before manually committing it. When we restart

Re: Kafka consumers in cluster

2016-08-06 Thread sat
Thanks Kamal, >Yes, it gets called on every re-balance. >-- Kamal On Thu, Aug 4, 2016 at 10:54 AM, sat wrote: > Hi Kamal, > > Thanks for your prompt response. Does our custom partition assignor gets > called during every rebalancing. > > Thanks and Regards > A.Sath

Re: Kafka consumers in cluster

2016-08-04 Thread sat
ns to the first consumer instance in >the group. >See 'partition.assignment.strategy' config in the consumer configs [1] >[1]: http://kafka.apache.org/documentation.html#newconsumerconfigs >On Thu, Aug 4, 2016 at 8:54 AM, sat wrote: > Hi, > > We have Kafka server/b

Re: Kafka Consumer poll

2016-08-03 Thread sat
split into 1min intervals of 10 times do{ if( pollForMessageIfAny()){ return; } Thread.sleep(1000); timeremaining = curretime - timeout; }while(timeremainning > 0) Thanks and Regards A.SathishKumar On Wed, Aug 3, 2016 at 5:46 PM, sat wrote: > Hi, > > >

Kafka consumers in cluster

2016-08-03 Thread sat
Hi, We have Kafka server/broker running in a seperate machine (say machine A), for now we are planning to have in one node. We have multiple topics and all topics have only 1 partition for now. We have our application which includes Kafka consumers installed in machine B and machine C. Our applic

Re: Kafka Consumer poll

2016-08-03 Thread sat
vent of temporary broker outage, it doesn't help if you >accidentally >specified wrong broker url. Oleg > On Aug 2, 2016, at 10:27, Kamal C wrote: > > See the answers inline. > >> On Tue, Aug 2, 2016 at 12:23 AM, sat wrote: >> >> Hi, >> >> I am ne

Kafka Consumer poll

2016-08-01 Thread sat
Hi, I am new to Kafka. We are planning to use Kafka messaging for our application. I was playing with Kafka 0.9.0.1 version and i have following queries. Sorry for asking basic questions. 1) I have instantiated Kafka Consumer and invoked consumer.poll(Long.MAX_VALUE). Although i have specified t