[ https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854475#comment-15854475 ]
Nicholas Carenza commented on NIFI-3437: ---------------------------------------- [~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that matter. I was trying to find something in the ConsumeKafka processor that kept track of state. I read up on the Kafka documentation a bit and I think I understand the processor works with Kafka to keep track of its offset. Let me see if I have this right: The processor commits its latest consumed offset to Kafka at least every MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's group id. So when I stop and start the processor, as long as it has the same group id, Kafka will start sending messages starting from the last committed offset. https://kafka.apache.org/documentation/#distributionimpl > KafkaConsumer pause/resume > -------------------------- > > Key: NIFI-3437 > URL: https://issues.apache.org/jira/browse/NIFI-3437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Reporter: Nicholas Carenza > Priority: Minor > > Could the KafkaConsumer processor keep track of it's position so that if it > is stopped and started again it can attempt to resume consuming from that > position? > If I have to restart nifi or rename the consumer processor I want to make > sure I don't miss any messages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)