smime.p7m
Description: S/MIME encrypted message
This sounds like something you can solve by a stateful operator. check out
mapWithState. If both the message can be keyed with a common key, then you
can define a keyed-state. the state will have a field for the first
message.When you see the first message for a key, fill the first field with
times
any valuable feedback is appreciated!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-How-to-do-join-two-messages-in-spark-streaming-Probabaly-messasges-are-in-differnet--tp28161p28163.html
Sent from the Apache Spark User List mailing list ar
smime.p7m
Description: S/MIME encrypted message
Hello,
we are trying to use Spark streaming to do some billing related application.
so our case is
that we need to correlate two different messages, and calculate the time
invterval between two
messages, the two message should be in same partition but probabaly not in
the same RDD,
it seems spar