I've been using `DStream.mapWithState` and was looking forward to trying out Structured Streaming. The thing I can't under is, does Structured Streaming in it's current state support stateful aggregations?
Looking at the StateStore design document (https://docs.google.com/document/d/1-ncawFx8JS5Zyfq1HAEGBx56RDet9wfVp_hDM8ZL254/edit#heading=h.2h7zw4ru3nw7), and then doing a bit of digging around in the Spark codebase, I've seen `mapPartitionsWithStateStore` as the only viable way of doing something with a store, but the API requires an `UnsafeRow` for key and value which makes we question if this is a real public API one should be using? Does anyone know what the state of things are currently in regards to an equivalent to `mapWithState` in Structured Streaming? Thanks, Yuval. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Stateful-aggregations-with-Structured-Streaming-tp28108.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
