[
https://issues.apache.org/jira/browse/STORM-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286933#comment-15286933
]
Robert Joseph Evans commented on STORM-1757:
--------------------------------------------
[~arunmahadevan],
BEAM does exactly once, by doing at least once, but making sure that the
checkpoints happen in an idempotent and coordinated way. The issue is not at
least once vs exactly once, but is around failure cases and consistency. From
what I have seen with the state API + windowing offers some coordination and
restore, reprocess, re-checkpoint should be idempotent, but it can only role
back to the last successful checkpoint. So if I ask all bolts to checkpoint
and most succeed but one does not, I can not longer restore everyone to a
consistent place, so I can start replaying again. This is what I am concerned
about. I would like to be able to fully support the BEAM model, and then if we
can offer better performance by reducing guarantees I am OK with that.
> Apache Beam Runner for Storm
> ----------------------------
>
> Key: STORM-1757
> URL: https://issues.apache.org/jira/browse/STORM-1757
> Project: Apache Storm
> Issue Type: Brainstorming
> Reporter: P. Taylor Goetz
> Priority: Minor
>
> This is a call for interested parties to collaborate on an Apache Beam [1]
> runner for Storm, and express their thoughts and opinions.
> Given the addition of the Windowing API to Apache Storm, we should be able to
> map naturally to the Beam API. If not, it may be indicative of shortcomings
> of the Storm API that should be addressed.
> [1] http://beam.incubator.apache.org
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
