[jira] [Commented] (KAFKA-7970) Missing topic causes service shutdown without exception
[ https://issues.apache.org/jira/browse/KAFKA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1638#comment-1638 ] Jonny Heavey commented on KAFKA-7970: - Thanks for your responses and confirmation both. I'll await a fix in an upcoming release, and look to mitigate in the meantime. Thanks again. > Missing topic causes service shutdown without exception > --- > > Key: KAFKA-7970 > URL: https://issues.apache.org/jira/browse/KAFKA-7970 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 2.1.0 >Reporter: Jonny Heavey >Priority: Minor > > When launching a KafkaStreams application that depends on a topic that > doesn't exist, the streams application correctly logs an error such as: > " is unknown yet during rebalance, please make sure they have > been pre-created before starting the Streams application." > The stream is then shutdown, however, no exception is thrown indicating that > an error has occurred. > In our circumstances, we run our streams app inside a container. The streams > service is shutdown, but the process is not exited, meaning that the > container does not crash (reducing visibility of the issue). > As no exception is thrown in the missing topic scenario described above, our > application code has no way to determine that something is wrong that would > then allow it to terminate the process. > > Could the onPartitionsAssigned method in StreamThread.java throw an exception > when it decides to shutdown the stream (somewhere around line 264)? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KAFKA-7970) Missing topic causes service shutdown without exception
[ https://issues.apache.org/jira/browse/KAFKA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774941#comment-16774941 ] Jonny Heavey edited comment on KAFKA-7970 at 2/22/19 9:30 AM: -- Thanks for your swift reply Matthias, I have set both an exception handler and a state listener as you suggest, but neither is triggered helpfully during these 'missing topic' circumstances: * No exception hits the uncaughtExceptionHandler * The stateListener receives transitions for RUNNING and REBALANCING only (it appears that the user 'Or' is seeing the same behaviour) After these state transitions, the StreamThread logs that shutdown is complete, and nothing else happens (no further state transitions to listen for etc). I'm fairly new to Kafka, so apologies if this is incorrect, but I wonder if the StreamThread could throw an exception when its shutting down for this reason, so that the uncaughtExceptionHandler receives notification, and/or that the stateListener isn't unset (as descirbed by 'Or'), so it could continue receiving further state transitions during shutdown? was (Author: jonnyheavey): Thanks for your swift reply Matthias, I have set both an exception handler and a state listener as you suggest, but neither is triggered helpfully during these 'missing topic' circumstances: * No exception hits the uncaughtExceptionHandler * The stateListener receives transitions for RUNNING and REBALANCING only (it appears that the user 'Or' is seeing the same behaviour) After these state transitions, the StreamThread logs that shutdown is complete, and nothing else happens (no further state transitions to listen for etc). I'm fairly new to Kafka, so apologies if this is incorrect, but I wonder if the StreamThread should throw and exception when its shutting down for this reason, so that the uncaughtExceptionHandler receives notification, or that the stateListener isn't unset, so it (presumably) continues to receives these further state transitions? > Missing topic causes service shutdown without exception > --- > > Key: KAFKA-7970 > URL: https://issues.apache.org/jira/browse/KAFKA-7970 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 2.1.0 >Reporter: Jonny Heavey >Priority: Minor > > When launching a KafkaStreams application that depends on a topic that > doesn't exist, the streams application correctly logs an error such as: > " is unknown yet during rebalance, please make sure they have > been pre-created before starting the Streams application." > The stream is then shutdown, however, no exception is thrown indicating that > an error has occurred. > In our circumstances, we run our streams app inside a container. The streams > service is shutdown, but the process is not exited, meaning that the > container does not crash (reducing visibility of the issue). > As no exception is thrown in the missing topic scenario described above, our > application code has no way to determine that something is wrong that would > then allow it to terminate the process. > > Could the onPartitionsAssigned method in StreamThread.java throw an exception > when it decides to shutdown the stream (somewhere around line 264)? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7970) Missing topic causes service shutdown without exception
[ https://issues.apache.org/jira/browse/KAFKA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774941#comment-16774941 ] Jonny Heavey commented on KAFKA-7970: - Thanks for your swift reply Matthias, I have set both an exception handler and a state listener as you suggest, but neither is triggered helpfully during these 'missing topic' circumstances: * No exception hits the uncaughtExceptionHandler * The stateListener receives transitions for RUNNING and REBALANCING only (it appears that the user 'Or' is seeing the same behaviour) After these state transitions, the StreamThread logs that shutdown is complete, and nothing else happens (no further state transitions to listen for etc). I'm fairly new to Kafka, so apologies if this is incorrect, but I wonder if the StreamThread should throw and exception when its shutting down for this reason, so that the uncaughtExceptionHandler receives notification, or that the stateListener isn't unset, so it (presumably) continues to receives these further state transitions? > Missing topic causes service shutdown without exception > --- > > Key: KAFKA-7970 > URL: https://issues.apache.org/jira/browse/KAFKA-7970 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 2.1.0 >Reporter: Jonny Heavey >Priority: Minor > > When launching a KafkaStreams application that depends on a topic that > doesn't exist, the streams application correctly logs an error such as: > " is unknown yet during rebalance, please make sure they have > been pre-created before starting the Streams application." > The stream is then shutdown, however, no exception is thrown indicating that > an error has occurred. > In our circumstances, we run our streams app inside a container. The streams > service is shutdown, but the process is not exited, meaning that the > container does not crash (reducing visibility of the issue). > As no exception is thrown in the missing topic scenario described above, our > application code has no way to determine that something is wrong that would > then allow it to terminate the process. > > Could the onPartitionsAssigned method in StreamThread.java throw an exception > when it decides to shutdown the stream (somewhere around line 264)? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KAFKA-7970) Missing topic causes service shutdown without exception
[ https://issues.apache.org/jira/browse/KAFKA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonny Heavey updated KAFKA-7970: Description: When launching a KafkaStreams application that depends on a topic that doesn't exist, the streams application correctly logs an error such as: " is unknown yet during rebalance, please make sure they have been pre-created before starting the Streams application." The stream is then shutdown, however, no exception is thrown indicating that an error has occurred. In our circumstances, we run our streams app inside a container. The streams service is shutdown, but the process is not exited, meaning that the container does not crash (reducing visibility of the issue). As no exception is thrown in the missing topic scenario described above, our application code has no way to determine that something is wrong that would then allow it to terminate the process. Could the onPartitionsAssigned method in StreamThread.java throw an exception when it decides to shutdown the stream (somewhere around line 264)? was: When launching a KafkaStreams application that depends on a topic that doesn't exist, the streams application correctly logs an error such as: " is unknown yet during rebalance, please make sure they have been pre-created before starting the Streams application." The stream is then shutdown, however, no exception is thrown indicating that an error has occurred. In our circumstances, we run our streams app inside a container. The streams service is shutdown, but the process is not exited, meaning that the container does not crash (reducing visibility of the issue). As no exception is thrown in the missing topic scenario described above, our application code has no way to determine that something is wrong that would then allow it to terminate the process. Could the StreamsThread:onPartitionsAssigned method throw an exception when it decides to shutdown the stream (somewhere around line 264)? > Missing topic causes service shutdown without exception > --- > > Key: KAFKA-7970 > URL: https://issues.apache.org/jira/browse/KAFKA-7970 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 2.1.0 >Reporter: Jonny Heavey >Priority: Minor > > When launching a KafkaStreams application that depends on a topic that > doesn't exist, the streams application correctly logs an error such as: > " is unknown yet during rebalance, please make sure they have > been pre-created before starting the Streams application." > The stream is then shutdown, however, no exception is thrown indicating that > an error has occurred. > In our circumstances, we run our streams app inside a container. The streams > service is shutdown, but the process is not exited, meaning that the > container does not crash (reducing visibility of the issue). > As no exception is thrown in the missing topic scenario described above, our > application code has no way to determine that something is wrong that would > then allow it to terminate the process. > > Could the onPartitionsAssigned method in StreamThread.java throw an exception > when it decides to shutdown the stream (somewhere around line 264)? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KAFKA-7970) Missing topic causes service shutdown without exception
Jonny Heavey created KAFKA-7970: --- Summary: Missing topic causes service shutdown without exception Key: KAFKA-7970 URL: https://issues.apache.org/jira/browse/KAFKA-7970 Project: Kafka Issue Type: Bug Components: streams Affects Versions: 2.1.0 Reporter: Jonny Heavey When launching a KafkaStreams application that depends on a topic that doesn't exist, the streams application correctly logs an error such as: " is unknown yet during rebalance, please make sure they have been pre-created before starting the Streams application." The stream is then shutdown, however, no exception is thrown indicating that an error has occurred. In our circumstances, we run our streams app inside a container. The streams service is shutdown, but the process is not exited, meaning that the container does not crash (reducing visibility of the issue). As no exception is thrown in the missing topic scenario described above, our application code has no way to determine that something is wrong that would then allow it to terminate the process. Could the StreamsThread:onPartitionsAssigned method throw an exception when it decides to shutdown the stream (somewhere around line 264)? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
