[jira] [Created] (KAFKA-16600) Periodically receive "Failed to transition to PENDING_SHUTDOWN, current state is PENDING_SHUTDOWN" during streams close
Alex Leung created KAFKA-16600:
--
Summary: Periodically receive "Failed to transition to
PENDING_SHUTDOWN, current state is PENDING_SHUTDOWN" during streams close
Key: KAFKA-16600
URL: https://issues.apache.org/jira/browse/KAFKA-16600
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 3.3.2
Reporter: Alex Leung
>From time to time, we observe the following ERROR message during streams close:
{code:java}
2024-04-13 07:40:16,222 INFO o.a.k.s.KafkaStreams [32] stream-client
[testapp-83aa0e09-a6ef-45e4-9393-69dfba4928bf] State transition from RUNNING to
PENDING_SHUTDOWN
2024-04-13 07:40:16,222 ERROR o.a.k.s.KafkaStreams [55] stream-client
[testapp-83aa0e09-a6ef-45e4-9393-69dfba4928bf] Failed to transition to
PENDING_SHUTDOWN, current state is PENDING_SHUTDOWN
{code}
These ERRORs started showing up when we moved from 2.7.2 to 3.3.2 and we have
not changed any code related to streams shutdown.
When the problem does not occur (most of the time), it looks like the following:
{code:java}
2024-04-13 07:40:11,333 INFO o.a.k.s.KafkaStreams [55] stream-client
[testapp-e2212b31-e9c2-4c75-92f6-1e557e27bf66] State transition from RUNNING to
PENDING_SHUTDOWN
2024-04-13 07:40:11,341 INFO o.a.k.s.KafkaStreams [32] stream-client
[testapp-e2212b31-e9c2-4c75-92f6-1e557e27bf66] Streams client is in
PENDING_SHUTDOWN, all resources are being closed and the client will be
stopped. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-8671) NullPointerException occurs if topic associated with GlobalKTable changes
Alex Leung created KAFKA-8671:
-
Summary: NullPointerException occurs if topic associated with
GlobalKTable changes
Key: KAFKA-8671
URL: https://issues.apache.org/jira/browse/KAFKA-8671
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 2.0.0
Reporter: Alex Leung
The following NullPointerException occurs when the global/.checkpoint file
contains a line with a topic previously associated with (but no longer
configured for) a GlobalKTable:
{code:java}
java.lang.NullPointerException
at
org.apache.kafka.streams.processor.internals.GlobalStateUpdateTask.update(GlobalStateUpdateTask.java:85)
at
org.apache.kafka.streams.processor.internals.GlobalStreamThread$StateConsumer.pollAndUpdate(GlobalStreamThread.java:241)
at
org.apache.kafka.streams.processor.internals.GlobalStreamThread.run(GlobalStreamThread.java:290){code}
After line 84
([https://github.com/apache/kafka/blob/2.0/streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateUpdateTask.java#L84)]
`sourceNodeAndDeserializer` is null for the old, but still valid, topic. This
can be reproduced with the following sequence:
# create a GlobalKTable associated with topic, 'global-topic1'
# change the topic associated with the GlobalKTable to 'global-topic2'
## at this point, the global/.checkpoint file will contain lines for both
topics
# produce messages to previous topic ('global-topic1')
# the consumer will attempt to consume from global-topic1, but no deserializer
associated with global-topic1 will be found and the NPE will occur
It looks like the following recent commit has included checkpoint validations
that may prevent this issue:
https://github.com/apache/kafka/commit/53b4ce5c00d61be87962f603682873665155cec4#diff-cc98a6c20f2a8483e1849aea6921c34dR425
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
