Hi all,
We have a streaming app running for over a year now without much incident
until recently. Suddenly the process keeps failing after a minute or so
with a failed to rebalance exception caused by Unexpected error in
InitProducerIdResponse; The server experienced an unexpected error when
processing the request. Please see stack trace at end of the email.
I've hunted on Google to some insights into the problem but to no avail.
We're running Kafka 2.0.0. The stream app/client is running on java 1.8.,
spring-kafka 2.1.10.Release and related kafa libs at 2.0.0 .
Your help is much appreciated. Many thanks in advance.
Stack trace below:
13:13:30.448 [ERROR]
[filter-dps-app-b86df7a2-da04-4093-9b09-45d1ea5dc7fb-StreamThread-31]
[StreamThread$RebalanceListener:onPartitionsAssigned:273] - stream-thread
[filter-dps-app-b86df7a2-da04-4093-9b09-45d1ea5dc7fb-StreamThread-31] Error
caught during partition assignment, will abort the current process and
re-throw at the end of rebalance: {}
org.apache.kafka.common.KafkaException: Unexpected error in
InitProducerIdResponse; The server experienced an unexpected error when
processing the request
at
org.apache.kafka.clients.producer.internals.TransactionManager$InitProducerIdHandler.handleResponse(TransactionManager.java:982)
at
org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:907)
at
org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)
at
org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:532)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:524)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:748)
Exception in thread
"filter-dps-app-b86df7a2-da04-4093-9b09-45d1ea5dc7fb-StreamThread-31"
org.apache.kafka.streams.errors.StreamsException: stream-thread
[filter-dps-app-b86df7a2-da04-4093-9b09-45d1ea5dc7fb-StreamThread-31]
Failed to rebalance.
at
org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:870)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:810)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:767)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:736)
Caused by: org.apache.kafka.common.KafkaException: Unexpected error in
InitProducerIdResponse; The server experienced an unexpected error when
processing the request
at
org.apache.kafka.clients.producer.internals.TransactionManager$InitProducerIdHandler.handleResponse(TransactionManager.java:982)
at
org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:907)
at
org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)
at
org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:532)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:524)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:748)
