Bill Bejeck created KAFKA-9398:
----------------------------------
Summary: Kafka Streams main thread may not exit even after close
timeout has passed
Key: KAFKA-9398
URL: https://issues.apache.org/jira/browse/KAFKA-9398
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Bill Bejeck
Assignee: Bill Bejeck
Fix For: 2.5.0
Kafka Streams offers the {{KafkaStreams.close()}} method when shutting down a
Kafka Streams application. There are two overloads to this method, one that
takes no parameters and another taking a {{Duration}} specifying how long the
{{close()}} method should block waiting for streams shutdown operations to
complete. The no-arg version of {{close()}} sets the timeout to
{{Long.MAX_VALUE}}.
The issue is that if a {{StreamThread}} is some how hung or if one of the
{{Consumer}} or {{Producer}} clients are in a hung state, the Kafka Streams
application won't exit even after the specified timeout has expired.
For example consider this scenario:
# A sink topic gets deleted by accident
# The {{Producer max.block.ms}} config is set to high value
In this case the {{Producer}} will issue a {{WARN}} logging statement and will
continue to make metadata requests looking for the expected topic. This will
continue up until the {{max.block.ms}} expires. If this value is high enough,
calling {{close()}} with a timeout won't fix the issue as when the timeout
expires, the Kafka Streams application main thread won't exit.
To prevent this type of issue, we should call {{Thread.interrupt()}} on all
{{StreamThread}} instances once the {{close()}} timeout has expired.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
