Re: Interesting Stateful Streaming question

This does sound like a good use case for that feature. Note that Spark 2.2. adds a similar [flat]MapGroupsWithState operation to structured streaming. Stay tuned for a blog post on that! On Thu, Jun 29, 2017 at 6:11 PM, kant kodali wrote: > Is mapWithState an answer for

Re: Interesting Stateful Streaming question

Is mapWithState an answer for this ? https://databricks.com/blog/2016/02/01/faster-stateful-stream-processing-in-apache-spark-streaming.html On Thu, Jun 29, 2017 at 11:55 AM, kant kodali wrote: > Hi All, > > Here is a problem and I am wondering if Spark Streaming is the

Interesting Stateful Streaming question

Hi All, Here is a problem and I am wondering if Spark Streaming is the right tool for this ? I have stream of messages m1, m2, m3and each of those messages can be in state s1, s2, s3,sn (you can imagine the number of states are about 100) and I want to compute some metrics that visit all