[jira] [Created] (CAMEL-10333) (Consumer goes into loop while processing messages) DEBUG Cannot auto-commit offsets now since the coordinator is unknown

[Shashank Madhav (JIRA)]

Shashank Madhav created CAMEL-10333:
---------------------------------------

             Summary: (Consumer goes into loop while processing messages) DEBUG 
Cannot auto-commit offsets now since the coordinator is unknown
                 Key: CAMEL-10333
                 URL: https://issues.apache.org/jira/browse/CAMEL-10333
             Project: Camel
          Issue Type: Bug
          Components: camel-kafka
    Affects Versions: 2.17.1
         Environment: Spring Framework is used for creating consumers, which 
processes the messages.
            Reporter: Shashank Madhav


I have a topic which have 300 messages in the queue. When the camel is used for 
processing the messages with event-based consumer, I am getting following error 
after all the messages are processed.

/////error message start
[ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            DEBUG Cannot 
auto-commit offsets now since the coordinator is unknown, will retry after 
backoff
[ad #0 - KafkaTopic[Audit_logs]] AbstractCoordinator            DEBUG Issuing 
group metadata request to broker 0
[ad #0 - KafkaTopic[Audit_logs]] AbstractCoordinator            DEBUG Group 
metadata response ClientResponse(receivedTimeMs=1473953020273, 
disconnected=false, request=ClientRequest(expectResponse=true, 
callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@1edc39d,
 
request=RequestSend(header={api_key=10,api_version=0,correlation_id=9,client_id=consumer-1},
 body={group_id=test3}), createdTimeMs=1473953020218, 
sendTimeMs=1473953020218), 
responseBody={error_code=0,coordinator={node_id=0,host=<host-name>,port=<port-no>}})
[ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            ERROR Error 
ILLEGAL_GENERATION occurred while committing offsets for group test3
[ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            WARN  Auto 
offset commit failed: Commit cannot be completed due to group rebalance
[ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            ERROR Error 
ILLEGAL_GENERATION occurred while committing offsets for group test3
[ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            WARN  Auto 
offset commit failed: 
////error message end

 Following the error, it then starts from the beginning and processes the 
entire queue again causing issues. The route used for camel is as follows 

//camel-route start
        <c:endpoint id="kafkaTest" 
uri="kafka:${broker.server}?topic=${topics1}&amp;groupId=test3&amp;autoOffsetReset=earliest&amp;consumersCount=1&amp;sessionTimeoutMs=30000&amp;consumerRequestTimeoutMs=40000&amp;keyDeserializer=${broker.keyDeserializer}&amp;valueDeserializer=${broker.valueDeserializer}"/>
                
                <c:route id="route1">
            <c:from uri="ref:kafkaTest"/>
            <c:delay>
            <c:constant>500</c:constant>
            </c:delay>
             <c:to uri="log:input"/> 
        </c:route>
//camel-route ends

Delay is purposely added in the route to create message processing scenario. If 
the delay is reduced or the sessionTimeoutMs is increased then it works fine, 
but this is not a feasible solution. Without camel it works fine and messages 
are processed properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)