Thanks, Jira ticket created:
https://issues.apache.org/jira/browse/KAFKA-10758

On Sat, Nov 21, 2020 at 8:12 PM Matthias J. Sax <[email protected]> wrote:

> Thanks for reaching out. Sounds like a bug to me. Would you mind
> creating a Jira?
>
> -Matthias
>
> On 11/21/20 10:29 AM, Davide Icardi wrote:
> > Hi all,
> >
> > I have a simple Kafka Stream app that consumes from multiple input topics
> > using the *stream* function that accepts a Pattern (link
> > <
> https://kafka.apache.org/26/javadoc/org/apache/kafka/streams/StreamsBuilder.html#stream-java.util.regex.Pattern-
> >
> > ).
> >
> > Whenever I add a new topic that matches the pattern the kafka stream
> state
> > goes to REBALANCING -> ERROR -> PENDING_SHUTDOWN .
> > If I restart the app it correctly starts reading again without problems.
> > It is by design? Should I handle this and simply restart the app?
> >
> > Kafka Stream version is 2.6.0.
> > The error is the following:
> >
> > *ERROR o.a.k.s.p.i.ProcessorTopology - Set of source nodes do not match:
> *
> > *sourceNodesByName = [KSTREAM-SOURCE-0000000003,
> KSTREAM-SOURCE-0000000002]*
> > *sourceTopicsByName = [KSTREAM-SOURCE-0000000000,
> > KSTREAM-SOURCE-0000000014, KSTREAM-SOURCE-0000000003,
> > KSTREAM-SOURCE-0000000002]*
> > *org.apache.kafka.common.KafkaException: User rebalance callback throws
> an
> > error*
> > *  at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:436)*
> > *  at
> >
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:440)*
> > *  at
> >
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:359)*
> > *  at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:513)*
> > *  at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1268)*
> > *  at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1230)*
> > *  at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1210)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:766)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:624)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:551)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:510)*
> > * Caused by: java.lang.IllegalStateException: Tried to update source
> topics
> > but source nodes did not match*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.ProcessorTopology.updateSourceTopics(ProcessorTopology.java:151)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.AbstractTask.update(AbstractTask.java:109)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.StreamTask.update(StreamTask.java:514)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.TaskManager.updateInputPartitionsAndResume(TaskManager.java:397)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.TaskManager.handleAssignment(TaskManager.java:261)*
> > *  at
> >
> org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor.onAssignment(StreamsPartitionAssignor.java:1428)*
> > *  at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.invokeOnAssignment(ConsumerCoordinator.java:279)*
> > *  at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:421)*
> > *  ... 10 common frames omitted*
> > * KafkaStream state is ERROR*
> > * 17:28:53.200 [datalake-StreamThread-1] ERROR
> > o.apache.kafka.streams.KafkaStreams - stream-client [datalake] All stream
> > threads have died. The instance will be in error state and should be
> > closed.*
> > * ============> User rebalance callback throws an error*
> > * KafkaStream state is PENDING_SHUTDOWN*
> >
> > thanks
> >
>

Reply via email to