[
https://issues.apache.org/jira/browse/KAFKA-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104937#comment-17104937
]
Guozhang Wang commented on KAFKA-9909:
--------------------------------------
[~gopikrishna.chaga...@target.com] We do have future plans to enable
checkpointing (potentially to remote storage as well), but to your use cases, I
think it still boils down to whether the error is retriable: if the message
itself is ill-formatted (like shown in your example code) then simply
restarting the process and hoping it would self-heal would not make sense.
Instead some manual intervention is still needed, sth like: after checkpointed
at offset 1, restart at offset 3 (i.e. skipping offset 2 since we know it is a
poison pill), by doing that the record at offset 2 would NOT be processed and
users may optionally require the producer to re-send the record etc.
> Kafka Streams : offset control to Streams API
> ---------------------------------------------
>
> Key: KAFKA-9909
> URL: https://issues.apache.org/jira/browse/KAFKA-9909
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 2.5.0
> Environment: All
> Reporter: Gopikrishna
> Priority: Minor
> Labels: Offset, commit
>
> Hello team, really inspired the way streams api is running today. I would
> like to have a feature to be flexible regarding the offset. when we write the
> processor api, processor context object can be used to commit the offset.
> this is not effective. but streams are controlling the offset. the moment the
> process method executed or scheduled window completed, the offset is
> committed automatically by streams internally.
> Like traditional kafka consumer, its better the context object should have
> complete control over the offset whether to commit or not. This will give
> more control to the api to handle failovers and especially when message
> cannot be processed, context should not commit the offset. Appreciate this
> can be implemented.
>
> h4. enable.auto.commit is by default false, but streams are committing
> automatically the offset.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
