Hi Team, Can you please provide any information on my above problem?
Thanks & Regards, Pankaj Ojha On Wed, Aug 6, 2014 at 8:04 PM, pankaj ojha <[email protected]> wrote: > Hi Team, > > I have a requirement of reading real time data using kafka and write to > cassandra. > For this I am using SimpleConsumer to read data from Kafka topics and > writing into Cassandra. > I am maintaining offsets of topics in my log files. > The issue is that after few days like 3-4 days my cosumer code does not > read data from kafka topics and produce below log output : > > 20:01:17,068 INFO NposKafkaConsumer:48 - Taking partition from > application properties > 20:01:17,482 DEBUG BlockingChannel:52 - Created socket with SO_TIMEOUT = > 100000 (requested 100000), SO_RCVBUF = 65536 (requested 65536), SO_SNDBUF = > 64512 (requested -1). > 20:01:17,545 DEBUG SimpleConsumer:52 - Disconnecting from <IP address>:9092 > 20:01:17,578 DEBUG NposKafkaConsumer:113 - NposKafkaConsumer.run() method > Inside while loop :: Value of max_reads::1 > 20:01:17,662 DEBUG BlockingChannel:52 - Created socket with SO_TIMEOUT = > 100000 (requested 100000), SO_RCVBUF = 65536 (requested 65536), SO_SNDBUF = > 64512 (requested -1). > 20:01:17,804 DEBUG NposKafkaConsumer:193 - numRead::0 Sleeping > 20:01:18,804 DEBUG NposKafkaConsumer:113 - NposKafkaConsumer.run() method > Inside while loop :: Value of max_reads::1 > 20:01:18,826 DEBUG NposKafkaConsumer:193 - numRead::0 Sleeping > 20:01:19,827 DEBUG NposKafkaConsumer:113 - NposKafkaConsumer.run() method > Inside while loop :: Value of max_reads::1 > 20:01:19,852 DEBUG NposKafkaConsumer:193 - numRead::0 Sleeping > > NposKafkaConsumer is my main SimpleCosumer class. > But, when I restart the kafka process by incrementing the offset by one > then again my code starts running fine for next few days. > > Can you please help me how I can solve this and where I am going wrong ? > > Thanks & Regards, > Pankaj Ojha > >
