[
https://issues.apache.org/jira/browse/FLINK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399575#comment-15399575
]
Stephan Ewen commented on FLINK-3398:
-------------------------------------
Let's reboot the discussion and work on this issue. Some observations:
1. Many cases do not not need committing of offsets externally. Users often
specify some random "group.id" if no meaningful group can be specified.
2. In some cases, committing is even harmful, as the picked up start offsets
override the reset strategies like (earliest offset). To avoid that, people use
a random "group.id".
3. If one wants to use external offsets, one typically also wants to update
them, and vice versa. Both is tied to a "group.id"
With these observations, it seems we can tie this to the existence of a group:
- If one sets a "group.id", we use it for start offset and commit offsets.
- If no "group.id" exists, no start offsets are obtained and no offsets are
committed.
Does that make sense?
> Flink Kafka consumer should support auto-commit opt-outs
> --------------------------------------------------------
>
> Key: FLINK-3398
> URL: https://issues.apache.org/jira/browse/FLINK-3398
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Reporter: Shikhar Bhushan
> Fix For: 1.2.0
>
>
> Currently the Kafka source will commit consumer offsets to Zookeeper, either
> upon a checkpoint if checkpointing is enabled, otherwise periodically based
> on {{auto.commit.interval.ms}}
> It should be possible to opt-out of committing consumer offsets to Zookeeper.
> Kafka has this config as {{auto.commit.enable}} (0.8) and
> {{enable.auto.commit}} (0.9).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
