wizz.com>
> *Sent:* Wednesday, September 30, 2015 4:07 PM
> *To:* user@spark.apache.org
> *Subject:* Re: partition recomputation in big lineage RDDs
>
>
> Hi,
>
> In fact, my RDD will get a new version (a new RDD assigned to the same
> var) quite frequently, by merging bulks of 100
30, 2015 4:07 PM
To: user@spark.apache.org
Subject: Re: partition recomputation in big lineage RDDs
Hi,
In fact, my RDD will get a new version (a new RDD assigned to the same var)
quite frequently, by merging bulks of 1000 events of events of last 10s.
But recomputation would be more efficient to d
:05 PM
To: user@spark.apache.org
Subject: partition recomputation in big lineage RDDs
Hi,
If I implement a manner to have an up-to-date version of my RDD by ingesting
some new events, called RDD_inc (from increment), and I provide a "merge"
function m(RDD, RDD_inc), which ret
Hi,
If I implement a manner to have an up-to-date version of my RDD by ingesting
some new events, called RDD_inc (from increment), and I provide a "merge"
function m(RDD, RDD_inc), which returns the RDD_new, it looks like I can evolve
the state of my RDD by constructing new RDDs all the time,
