Re: Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions
> > if we are just creating a global state store (GlobalKTable > for instance) from a topic, then that is what you are calling as > global-only topology. Exactly. If there is no KStream or KTable in your Streams topology, just a GlobalKTable, then that would be a "global-only" topology On Mon, Sep 14, 2020 at 7:46 AM John Roesler wrote: > Hi Pushkar, > > I'd recommend always keeping Streams and the Clients at the > same version, since we build, test, and release them > together. FWIW, I think there were some bugfixes for the > clients in 2.5.1 anyway. > > Thanks, > -John > > On Mon, 2020-09-14 at 20:08 +0530, Pushkar Deole wrote: > > Sophie, one more question: will just upgrading kafka-streams jar to 2.5.1 > > will work or we need to other jars also to be upgraded to 2.5.1 e.g. > > kafka-clients etc. ? > > > > On Mon, Sep 14, 2020 at 7:16 PM Pushkar Deole > wrote: > > > > > Thanks Sophie... if we are just creating a global state store > > > (GlobalKTable for instance) from a topic, then that is what you are > calling > > > as global-only topology. In our application that is what we are doing > and > > > there is no source topic for the stream to process data from, i mean > there > > > is however it is done through a consumer-producer kind of design and > not > > > through stream topology. > > > > > > On Fri, Sep 11, 2020 at 10:58 PM Sophie Blee-Goldman < > sop...@confluent.io> > > > wrote: > > > > > > > You should upgrade to 2.5.1, it contains a fix for this. > > > > > > > > Technically the "fix" is just to automatically set the > num.stream.threads > > > > to 0 > > > > when a global-only topology is detected, so setting this manually > would > > > > accomplish the same thing. But the fix also includes a tweak of the > > > > KafkaStreams state machine to make sure it reaches the RUNNING state > > > > even with no stream threads. So if you use a state listener, you'll > want > > > > to > > > > use 2.5.1 > > > > > > > > It's always a good idea to upgrade when a new bugfix version is > released > > > > anyway > > > > > > > > On Fri, Sep 11, 2020 at 5:15 AM Pushkar Deole > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > I upgraded from Kafka streams 2.4 to 2.5.0 and one of the > applications > > > > > suddenly stopped working with the error message: > > > > > > > > > > Exception in thread > > > > > > "DsiApplication-0fcde033-dab2-431c-9d82-76e85fcb4c91-StreamThread-1" > > > > > java.lang.IllegalStateException: Consumer is not subscribed to any > > > > topics > > > > > or assigned any partitions > > > > > at > > > > > > > > > > > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1228) > > > > > at > > > > > > > > > > > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) > > > > > at > > > > > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:853) > > > > > at > > > > > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:753) > > > > > at > > > > > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:697) > > > > > at > > > > > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670) > > > > > This application uses streams just to create a global state store > from a > > > > > topic in order to create a global state store as a cache for > static data > > > > > across application instances and the stream doesn't consume from > any > > > > input > > > > > topic. Came across following thread on stackoverflow > > > > > > > > > > > > > > > https://stackoverflow.com/questions/61342530/kafka-streams-2-5-0-requires-input-topic > > > > > Matthias, I see you have answered some queries there, so would > like to > > > > > confirm if setting num.stream.threads to 0 will solve this issue? > > > > > > >
Re: Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions
Hi Pushkar, I'd recommend always keeping Streams and the Clients at the same version, since we build, test, and release them together. FWIW, I think there were some bugfixes for the clients in 2.5.1 anyway. Thanks, -John On Mon, 2020-09-14 at 20:08 +0530, Pushkar Deole wrote: > Sophie, one more question: will just upgrading kafka-streams jar to 2.5.1 > will work or we need to other jars also to be upgraded to 2.5.1 e.g. > kafka-clients etc. ? > > On Mon, Sep 14, 2020 at 7:16 PM Pushkar Deole wrote: > > > Thanks Sophie... if we are just creating a global state store > > (GlobalKTable for instance) from a topic, then that is what you are calling > > as global-only topology. In our application that is what we are doing and > > there is no source topic for the stream to process data from, i mean there > > is however it is done through a consumer-producer kind of design and not > > through stream topology. > > > > On Fri, Sep 11, 2020 at 10:58 PM Sophie Blee-Goldman > > wrote: > > > > > You should upgrade to 2.5.1, it contains a fix for this. > > > > > > Technically the "fix" is just to automatically set the num.stream.threads > > > to 0 > > > when a global-only topology is detected, so setting this manually would > > > accomplish the same thing. But the fix also includes a tweak of the > > > KafkaStreams state machine to make sure it reaches the RUNNING state > > > even with no stream threads. So if you use a state listener, you'll want > > > to > > > use 2.5.1 > > > > > > It's always a good idea to upgrade when a new bugfix version is released > > > anyway > > > > > > On Fri, Sep 11, 2020 at 5:15 AM Pushkar Deole > > > wrote: > > > > > > > Hi All, > > > > > > > > I upgraded from Kafka streams 2.4 to 2.5.0 and one of the applications > > > > suddenly stopped working with the error message: > > > > > > > > Exception in thread > > > > "DsiApplication-0fcde033-dab2-431c-9d82-76e85fcb4c91-StreamThread-1" > > > > java.lang.IllegalStateException: Consumer is not subscribed to any > > > topics > > > > or assigned any partitions > > > > at > > > > > > > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1228) > > > > at > > > > > > > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) > > > > at > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:853) > > > > at > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:753) > > > > at > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:697) > > > > at > > > > > > > > > > > org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670) > > > > This application uses streams just to create a global state store from a > > > > topic in order to create a global state store as a cache for static data > > > > across application instances and the stream doesn't consume from any > > > input > > > > topic. Came across following thread on stackoverflow > > > > > > > > > > > https://stackoverflow.com/questions/61342530/kafka-streams-2-5-0-requires-input-topic > > > > Matthias, I see you have answered some queries there, so would like to > > > > confirm if setting num.stream.threads to 0 will solve this issue? > > > >
Re: Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions
Sophie, one more question: will just upgrading kafka-streams jar to 2.5.1 will work or we need to other jars also to be upgraded to 2.5.1 e.g. kafka-clients etc. ? On Mon, Sep 14, 2020 at 7:16 PM Pushkar Deole wrote: > Thanks Sophie... if we are just creating a global state store > (GlobalKTable for instance) from a topic, then that is what you are calling > as global-only topology. In our application that is what we are doing and > there is no source topic for the stream to process data from, i mean there > is however it is done through a consumer-producer kind of design and not > through stream topology. > > On Fri, Sep 11, 2020 at 10:58 PM Sophie Blee-Goldman > wrote: > >> You should upgrade to 2.5.1, it contains a fix for this. >> >> Technically the "fix" is just to automatically set the num.stream.threads >> to 0 >> when a global-only topology is detected, so setting this manually would >> accomplish the same thing. But the fix also includes a tweak of the >> KafkaStreams state machine to make sure it reaches the RUNNING state >> even with no stream threads. So if you use a state listener, you'll want >> to >> use 2.5.1 >> >> It's always a good idea to upgrade when a new bugfix version is released >> anyway >> >> On Fri, Sep 11, 2020 at 5:15 AM Pushkar Deole >> wrote: >> >> > Hi All, >> > >> > I upgraded from Kafka streams 2.4 to 2.5.0 and one of the applications >> > suddenly stopped working with the error message: >> > >> > Exception in thread >> > "DsiApplication-0fcde033-dab2-431c-9d82-76e85fcb4c91-StreamThread-1" >> > java.lang.IllegalStateException: Consumer is not subscribed to any >> topics >> > or assigned any partitions >> > at >> > >> > >> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1228) >> > at >> > >> > >> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) >> > at >> > >> > >> org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:853) >> > at >> > >> > >> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:753) >> > at >> > >> > >> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:697) >> > at >> > >> > >> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670) >> > >> > This application uses streams just to create a global state store from a >> > topic in order to create a global state store as a cache for static data >> > across application instances and the stream doesn't consume from any >> input >> > topic. Came across following thread on stackoverflow >> > >> > >> https://stackoverflow.com/questions/61342530/kafka-streams-2-5-0-requires-input-topic >> > >> > Matthias, I see you have answered some queries there, so would like to >> > confirm if setting num.stream.threads to 0 will solve this issue? >> > >> >
Re: Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions
Thanks Sophie... if we are just creating a global state store (GlobalKTable for instance) from a topic, then that is what you are calling as global-only topology. In our application that is what we are doing and there is no source topic for the stream to process data from, i mean there is however it is done through a consumer-producer kind of design and not through stream topology. On Fri, Sep 11, 2020 at 10:58 PM Sophie Blee-Goldman wrote: > You should upgrade to 2.5.1, it contains a fix for this. > > Technically the "fix" is just to automatically set the num.stream.threads > to 0 > when a global-only topology is detected, so setting this manually would > accomplish the same thing. But the fix also includes a tweak of the > KafkaStreams state machine to make sure it reaches the RUNNING state > even with no stream threads. So if you use a state listener, you'll want to > use 2.5.1 > > It's always a good idea to upgrade when a new bugfix version is released > anyway > > On Fri, Sep 11, 2020 at 5:15 AM Pushkar Deole > wrote: > > > Hi All, > > > > I upgraded from Kafka streams 2.4 to 2.5.0 and one of the applications > > suddenly stopped working with the error message: > > > > Exception in thread > > "DsiApplication-0fcde033-dab2-431c-9d82-76e85fcb4c91-StreamThread-1" > > java.lang.IllegalStateException: Consumer is not subscribed to any topics > > or assigned any partitions > > at > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1228) > > at > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) > > at > > > > > org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:853) > > at > > > > > org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:753) > > at > > > > > org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:697) > > at > > > > > org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670) > > > > This application uses streams just to create a global state store from a > > topic in order to create a global state store as a cache for static data > > across application instances and the stream doesn't consume from any > input > > topic. Came across following thread on stackoverflow > > > > > https://stackoverflow.com/questions/61342530/kafka-streams-2-5-0-requires-input-topic > > > > Matthias, I see you have answered some queries there, so would like to > > confirm if setting num.stream.threads to 0 will solve this issue? > > >
Re: Kafka stream error - Consumer is not subscribed to any topics or assigned any partitions
You should upgrade to 2.5.1, it contains a fix for this. Technically the "fix" is just to automatically set the num.stream.threads to 0 when a global-only topology is detected, so setting this manually would accomplish the same thing. But the fix also includes a tweak of the KafkaStreams state machine to make sure it reaches the RUNNING state even with no stream threads. So if you use a state listener, you'll want to use 2.5.1 It's always a good idea to upgrade when a new bugfix version is released anyway On Fri, Sep 11, 2020 at 5:15 AM Pushkar Deole wrote: > Hi All, > > I upgraded from Kafka streams 2.4 to 2.5.0 and one of the applications > suddenly stopped working with the error message: > > Exception in thread > "DsiApplication-0fcde033-dab2-431c-9d82-76e85fcb4c91-StreamThread-1" > java.lang.IllegalStateException: Consumer is not subscribed to any topics > or assigned any partitions > at > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1228) > at > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) > at > > org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:853) > at > > org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:753) > at > > org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:697) > at > > org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670) > > This application uses streams just to create a global state store from a > topic in order to create a global state store as a cache for static data > across application instances and the stream doesn't consume from any input > topic. Came across following thread on stackoverflow > > https://stackoverflow.com/questions/61342530/kafka-streams-2-5-0-requires-input-topic > > Matthias, I see you have answered some queries there, so would like to > confirm if setting num.stream.threads to 0 will solve this issue? >
