Hi all
I'm having a datastream transformation, that updates a mutable
hashmap that exists outside of the stream.
So it's something like
object FlinkJob {
val uriLookup = mutable.HashMap.empty[String, Int]
def main(args: Array[String]) {
val stream: DataStream = ...
stream.keybBy(1).timeWindow(..).fold(..)
.window(..)
.map(..).fold(..)
.addSink(..)
}
}
where the uriLookup hashmap gets updated inside the stream
transformation,
and is serialized in the step before the addSink
It works fine, however
Does the snapshotting mechanism in case of a node failure actually
serialize this map?
And out of curiousity, can I actually see what data exists inside the
snapshot data?
Thanks.
Bart