Can I invoke UpdateStateByKey twice on the same RDD. My requirement is as follows.
1. Get the event stream from Kafka 2. UpdateStateByKey to aggregate and filter events based on timestamp 3. Do some processing and save results to Cassandra DB 4. UpdateStateByKey to remove keys based on logout eventType. I tried doing it assigning results from step 2 to a VAR and reassigning it to the updated value in step 4. But seems it does not work that way. I am new to spark and not sure how this kind of behaviour is possible. Appreciate any help. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Invoking-updateStateByKey-twice-on-the-same-RDD-tp21623.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
