Hi Nitin, 1) A Kafka Consumer uses a poll loop to pull messages from the topics it is subscribed to. You can see examples of how this can be implemented in Java here: https://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0-9-consumer-client/
2,3) Consumers normally consume messages as part of consumer groups. The last consumed offset is preserved for each consumer group. Therefore, if a consumer group stops consuming for a period of time, the next time a consumer starts consuming as part of that group it will resume fetching messages from those preserved offsets (assuming the messages have not expired yet). In your example, the consumer will get message #6 as its next fetched message. You can read more about consumers and consumer groups here: https://docs.confluent.io/current/clients/consumer.html --Vahid From: Nitin Gupta <nitin...@gmail.com> To: users@kafka.apache.org Date: 06/18/2018 04:53 AM Subject: Multiple consumers subscribing to a topic Hi I am looking for the below setup in one of the Applications I am working on: Application generates events and publishes the events to a Kafka topic. The topic is subscribed by multiple consumers. Assuming there are 10 events triggered in the application and published to the topic. How do we handle each of the below scenarios: 1) Once the message is published to the topic, should consumers pull the message from topic or as a subscriber to the topic, the message be pushed to them. 2) In case one of the consumers is down, will Kafka push all the 10 messages once it is up? If yes, how? 3) In case on of the consumers goes down, after 5 messages have been pushed/consumed by it. What happens to the remaining messages? Once consumer is up should it notify Kafka to send messages fromn 6-10 or Kafka is intelligent enough to only trigger the messages from 6-10. Thanks. Regards Nitin
