[ https://issues.apache.org/jira/browse/KAFKA-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikumar resolved KAFKA-3553. ------------------------------ Resolution: Auto Closed {color:#000000}Closing inactive issue. Please raise issue with kafka-python project, if the issue still exists with newer versions.{color} > Issue with getting data of next offset after restarting the consumer and > producer is always running > --------------------------------------------------------------------------------------------------- > > Key: KAFKA-3553 > URL: https://issues.apache.org/jira/browse/KAFKA-3553 > Project: Kafka > Issue Type: Bug > Components: consumer, offset manager > Affects Versions: 0.9.0.0 > Environment: operating system : ubuntu 14.04 > using kafka-python > Reporter: pooja sadashiv deokar > Assignee: Neha Narkhede > Priority: Major > Labels: performance > > I am pushing data to kafka topic after every 1 second in python. And written > consumer to fetch data from topic with consumer timeout as 500 ms and > enable_auto_commit as false. > Following are the scripts: > 1) prod.py > from kafka import KafkaClient,SimpleConsumer > from kafka.producer import SimpleProducer > import random, time > kafka = KafkaClient("localhost:9092") > producer = SimpleProducer(kafka) > i=0 > while(True): > l1 = ['Rahul','Narendra','NaMo','ManMohan','Sonia'] > msg = str(i) + ',' + str(random.choice(l1)) > print('putting data : '+msg) > producer.send_messages("test", msg) > i = i + 1 > time.sleep(1) > 2) con.py > from kafka import KafkaConsumer > consumer = > KafkaConsumer('test',bootstrap_servers=['localhost:9092'],consumer_timeout_ms > = 500, enable_auto_commit = False) > for message in consumer: > print ("%s:%d:%d: key=%s value=%s" % (message.topic, message.partition, > message.offset, message.key, > message.value)) > consumer.commit() > My producer is continuously running. And I am running consumer again when it > stops. For first time consumer works well and gets data for me on time. But > after first time its taking minimum 1 minute to maximum 5 minutes (or above > 5 min also) to get next data. It should give me data after 1 second or > maximum after 5 seconds. But its giving problem :( . > Also 1 thing which I observe is If I wait for starting consumer about 1 min > then data comes as expected. > Please correct me if I am going wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)