Hi, Beam would only provide an abstract API for handling state in DoFns. How this state is maintained is up to the runner implementation and differs by a great deal between the Google Dataflow runner, the Spark runner and the Flink runner.
The Flink runner maintains faul-tolerant state by doing periodic checkpoints of the state in all operations. Cheers, Aljoscha On Thu, 16 Jun 2016 at 23:32 amir bahmanyari <[email protected]> wrote: > Hi Colleagues, > How does Beam store/keeps state? > State store mechanism that persists from line to line for the life of the > pipeline execution. > Any example(s)? > Thanks & have a great weekend all. > Amir- >
