updateStateByKey will run for all keys, whether they have new data in a batch or not so you should be able to still use it.
On 7/3/15, 7:34 AM, "micvog" <mich...@micvog.com> wrote: >UpdateStateByKey is useful but what if I want to perform an operation to all >existing keys (not only the ones in this RDD). > >Word count for example - is there a way to decrease *all* words seen so far >by 1? > >I was thinking of keeping a static class per node with the count information >and issuing a broadcast command to take a certain action, but could not find >a broadcast-to-all-nodes functionality or a better way. > >Thanks, >Michael > > > >----- >Michael Vogiatzis >@mvogiatzis >-- >View this message in context: >http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-broadcast-to-all-keys-tp23609.html >Sent from the Apache Spark User List mailing list archive at Nabble.com. > >--------------------------------------------------------------------- >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >For additional commands, e-mail: user-h...@spark.apache.org >
