RE: [Spark Streaming] How to do join two messages in spark streaming(Probabaly messasges are in differnet RDD) ?
smime.p7m Description: S/MIME encrypted message
Re: [Spark Streaming] How to do join two messages in spark streaming(Probabaly messasges are in differnet RDD) ?
any valuable feedback is appreciated! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-How-to-do-join-two-messages-in-spark-streaming-Probabaly-messasges-are-in-differnet--tp28161p28163.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
RE: [Spark Streaming] How to do join two messages in spark streaming(Probabaly messasges are in differnet RDD) ?
smime.p7m Description: S/MIME encrypted message
[Spark Streaming] How to do join two messages in spark streaming(Probabaly messasges are in differnet RDD) ?
Hello, we are trying to use Spark streaming to do some billing related application. so our case is that we need to correlate two different messages, and calculate the time invterval between two messages, the two message should be in same partition but probabaly not in the same RDD, it seems spark DStream.join can only join between current RDDs. We want to avoid store the erailer message to some datastore but prefer to use natvie solution if these is any in Spark. pls advice, thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-How-to-do-join-two-messages-in-spark-streaming-Probabaly-messasges-are-in-differnet--tp28161.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
