[ https://issues.apache.org/jira/browse/SAMZA-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Chen updated SAMZA-2590: -------------------------------- Attachment: Samza State Backup and Async Checkpointing.pdf > Async State Backup > ------------------ > > Key: SAMZA-2590 > URL: https://issues.apache.org/jira/browse/SAMZA-2590 > Project: Samza > Issue Type: New Feature > Reporter: Daniel Chen > Assignee: Daniel Chen > Priority: Major > Attachments: Samza State Backup and Async Checkpointing.pdf > > > # We must create a way of committing to the state store that allows for the > remote stores to be uploaded independently from processing. > # The commit phase must be transactional for Kafka changelog commits > (preserve existing behaviour) and remote store commits. > # For remote store commits, we require exclusive, non-overlapping commits so > that the commits that appear in the remote store are in order. > # We need to timeout on the asynchronous commit such that we will stop > further processing until the state in the remote store is caught up to the > current state of the job. > # This feature will be required to be enabled by default of all Samza jobs > regardless of the remote state store backup implementation (Kafka, Azure blob > store, HDFS) and the processing parallelism (async or sync processing) to > improve performance for all present and future remote state commits. -- This message was sent by Atlassian Jira (v8.3.4#803005)