[ https://issues.apache.org/jira/browse/BEAM-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amit Sela updated BEAM-648: --------------------------- Assignee: (was: Amit Sela) > Persist and restore Aggergator values in case of recovery from failure > ---------------------------------------------------------------------- > > Key: BEAM-648 > URL: https://issues.apache.org/jira/browse/BEAM-648 > Project: Beam > Issue Type: Bug > Components: runner-spark > Reporter: Amit Sela > > While Aggregators are fault-tolerant (pending > https://github.com/apache/incubator-beam/pull/909), they will recover into > their initial value. > The SparkRunner should persist the values to durable storage and recover from > the persisted state. > One way to go would be to persist via Spark's listeners, for example > "onBatchCompleted". > For more details see SPARK-5206. -- This message was sent by Atlassian JIRA (v6.3.4#6332)