DStream RDD
Something almost like that can be achieved with updateStateByKey, but is
there a way to do it as described here
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/adding-new-elements-to-batch-RDD-from-DStream-RDD-tp22504.html
Sent from the Apache
that can be achieved with updateStateByKey, but is
there a way to do it as described here
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/adding-new-elements-to-batch-RDD-from-DStream-RDD-tp22504.html
Sent from the Apache Spark User List mailing list archive
. a second
time moreover after specific period of time
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 8:14 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
Yes, I mean there's
@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
What do you mean by batch RDD? they're just RDDs, though store their data
in different ways and come from different sources. You can union an RDD
from an HDFS file with one from a DStream.
It sounds like you want streaming
7:43 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
What do you mean by batch RDD? they're just RDDs, though store their data in
different ways and come from different sources. You can union an RDD from an
HDFS file with one from
to the newly instantiated/loaded batch
RDD - is that what you mean by reloading batch RDD from file
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 7:43 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch
batch RDDs from file for e.g. a
second time moreover after specific period of time
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 8:14 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from
that since we are not getting anywhere
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 8:30 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
What API differences are you talking about
