[ https://issues.apache.org/jira/browse/KAFKA-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253783#comment-16253783 ]
Damian Guy commented on KAFKA-6214: ----------------------------------- Looks like the issue is in {{ProcessorStateManager}}. We only restore and hence assign partitions if the store is persistent {code} if (store.persistent()) { log.trace("Preparing standby replica of persistent state store {} with changelog topic {}", store.name(), topic); restoreCallbacks.put(topic, stateRestoreCallback); } {code} > Using standby replicas with an in memory state store causes Streams to crash > ---------------------------------------------------------------------------- > > Key: KAFKA-6214 > URL: https://issues.apache.org/jira/browse/KAFKA-6214 > Project: Kafka > Issue Type: Bug > Components: streams > Affects Versions: 0.11.0.1 > Reporter: Matt Farmer > Labels: statestore > > We decided to start experimenting with Standby Replicas of our State Stores > by setting the following configuration setting: > {code} > num.standby.replicas=1 > {code} > Most applications did okay with this except for one that used an in memory > state store instead of a persistent state store. With the new configuration, > the first instance of this application booted fine. When the second instance > came up, both instances crashed with the following exception: > {code} > java.lang.IllegalStateException: Consumer is not subscribed to any topics or > assigned any partitions > at > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1037) > at > org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:752) > at > org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:524) > at > org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480) > at > org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457) > {code} > Monit attempted to restart both instances but they would just continue to > crash over and over again. The state store in our problematic application is > declared like so: > {code} > Stores > .create("TheStateStore") > .withStringKeys() > .withStringValues() > .inMemory() > .build() > {code} > Luckily we had a config switch in place that could turn on an alternate, > persistent state store. As soon as we flipped to the persistent state store, > things started working as we expected. -- This message was sent by Atlassian JIRA (v6.4.14#64029)