[
https://issues.apache.org/jira/browse/SAMZA-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Chen updated SAMZA-2590:
--------------------------------
Attachment: Samza State Backup and Async Checkpointing.pdf
> Async State Backup
> ------------------
>
> Key: SAMZA-2590
> URL: https://issues.apache.org/jira/browse/SAMZA-2590
> Project: Samza
> Issue Type: New Feature
> Reporter: Daniel Chen
> Assignee: Daniel Chen
> Priority: Major
> Attachments: Samza State Backup and Async Checkpointing.pdf
>
>
> # We must create a way of committing to the state store that allows for the
> remote stores to be uploaded independently from processing.
> # The commit phase must be transactional for Kafka changelog commits
> (preserve existing behaviour) and remote store commits.
> # For remote store commits, we require exclusive, non-overlapping commits so
> that the commits that appear in the remote store are in order.
> # We need to timeout on the asynchronous commit such that we will stop
> further processing until the state in the remote store is caught up to the
> current state of the job.
> # This feature will be required to be enabled by default of all Samza jobs
> regardless of the remote state store backup implementation (Kafka, Azure blob
> store, HDFS) and the processing parallelism (async or sync processing) to
> improve performance for all present and future remote state commits.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
