Re: Structured Stream equivalent of reduceByKey

Hmmm, I see. You could output the delta using flatMapGroupsWithState

Re: Structured Stream equivalent of reduceByKey

Thanks, Michael I have explored Aggregator with update mode. The problem is it will give the overall aggregated value for the changed. while I only want the delta change in the group as the

Re: Structured Stream equivalent of reduceByKey

- dev I think you should be able to write an Aggregator . You probably want to run in update mode if you are looking for it to output any group that has changed in the batch. On Wed, Oct 25,

Structured Stream equivalent of reduceByKey

Hi, we are migrating some jobs from Dstream to Structured Stream. Currently to handle aggregations we call map and reducebyKey on each RDD like rdd.map(event => (event._1, event)).reduceByKey((a, b) => merge(a, b)) The final output of each RDD is merged to the sink with support for aggregation