Hi We had several incidents where a streams application crashed while maintaining a global state store. Updating global state failed. You can restart KafkaStreams to recover from this error.: org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of range with no configured reset policy for partitions: ...
As we never set this to none I checked the code and found that StreamsConfig getGlobalConsumerConfigs and getRestoreConsumerConfigs both set this explicitely: baseConsumerProps.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "none"); The logs confirms this: 2018-10-02 11:07:06,057 INFO [org.apache.kafka.common.utils.AppInfoParser] (ServerService Thread Pool -- 70) - [short-component-name:; transaction-id:; user-id:; creation-time:] Kafka version : 2.0.0-cp1 2018-10-02 11:07:06,057 INFO [org.apache.kafka.common.utils.AppInfoParser] (ServerService Thread Pool -- 70) - [short-component-name:; transaction-id:; user-id:; creation-time:] Kafka commitId : a8c648ff08b9235d 2018-10-02 11:07:06,104 INFO [org.apache.kafka.clients.consumer.ConsumerConfig] (ServerService Thread Pool -- 72) - [short-component-name:; transaction-id:; user-id:; creation-time:] ConsumerConfig values: auto.commit.interval.ms = 5000 auto.offset.reset = none bootstrap.servers = [...] check.crcs = true client.id = ...-3f809a8a-3915-4ae4-9a37-f7a392e3dff3-global-consumer ... 2018-10-02 11:07:06,418 INFO [org.apache.kafka.streams.processor.internals.StreamThread] (ServerService Thread Pool -- 72) - [short-component-name:; transaction-id:; user-id:; creation-time:] stream-thread [...-3f809a8a-3915-4ae4-9a37-f7a392e3dff3-StreamThread-1] Creating restore consumer client 2018-10-02 11:07:06,419 INFO [org.apache.kafka.clients.consumer.ConsumerConfig] (ServerService Thread Pool -- 72) - [short-component-name:; transaction-id:; user-id:; creation-time:] ConsumerConfig values: auto.commit.interval.ms = 5000 auto.offset.reset = none bootstrap.servers = [...] check.crcs = true client.id = ...-3f809a8a-3915-4ae4-9a37-f7a392e3dff3-StreamThread-1-restore-consumer Is this intentional and if yes, why can this not use the default policy and recover? best regards Patrik
