[
https://issues.apache.org/jira/browse/STORM-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287098#comment-15287098
]
Arun Mahadevan commented on STORM-1757:
---------------------------------------
> So if I ask all bolts to checkpoint and most succeed but one does not, I can
> not longer restore everyone to a consistent place, so I can start replaying
> again. This is what I am concerned about.
The checkpointing handle this via a prepare and a commit phase.
- First a "prepare" message is send. If the prepare fails (most of the bolts
succeeded but one did not), the checkpoint is restored to the last successful
point (rollback) and tuples are replayed.
- If prepare succeeds, a "commit" message is send.
- If commit fails, the commit is re-attempted by sending the "commit" message
again. The bolts that had already committed ignores this and the bolt that had
previously not committed would now commit. They can do so because the prepared
data is persisted in the state during prepare phase.
- Once the commit succeeds the txn is marked as complete.
> Apache Beam Runner for Storm
> ----------------------------
>
> Key: STORM-1757
> URL: https://issues.apache.org/jira/browse/STORM-1757
> Project: Apache Storm
> Issue Type: Brainstorming
> Reporter: P. Taylor Goetz
> Priority: Minor
>
> This is a call for interested parties to collaborate on an Apache Beam [1]
> runner for Storm, and express their thoughts and opinions.
> Given the addition of the Windowing API to Apache Storm, we should be able to
> map naturally to the Beam API. If not, it may be indicative of shortcomings
> of the Storm API that should be addressed.
> [1] http://beam.incubator.apache.org
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
