[
https://issues.apache.org/jira/browse/FLINK-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-3037:
----------------------------------
Description:
Currently, if the {{FlinkKafkaConsumer}} is restoring a checkpoint and the
offset is not available anymore in Kafka, its restoring according to
{{auto.offset.reset}}.
This leads to inconsistent behavior (not exactly-once anymore) because the
operators will not receive data in sync with the checkpoint.
With this pull request, I would like to make the behavior controllable, using a
flag. The simplest approach would be to let the consumer fail in that case.
was:
Currently, if the {{FlinkKafkaConsumer}} is restoring a checkpoint and the
offset is not available anymore in Kafka, its restoring according to
{{auto.offset.reset}}.
This leads to inconsistent behavior (not exactly-once anymore) because the
operators will not receive data in sync with the checkpoint.
With this pull request, I would like to make the behavior controllable, using a
flag.
> Make the behavior of the Kafka consumer configurable if the offsets to
> restore from are not available
> -----------------------------------------------------------------------------------------------------
>
> Key: FLINK-3037
> URL: https://issues.apache.org/jira/browse/FLINK-3037
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Reporter: Robert Metzger
>
> Currently, if the {{FlinkKafkaConsumer}} is restoring a checkpoint and the
> offset is not available anymore in Kafka, its restoring according to
> {{auto.offset.reset}}.
> This leads to inconsistent behavior (not exactly-once anymore) because the
> operators will not receive data in sync with the checkpoint.
> With this pull request, I would like to make the behavior controllable, using
> a flag. The simplest approach would be to let the consumer fail in that case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
