Re: subtractByKey increases RDD size in memory - any ideas?

2016-02-19 Thread DaPsul
but rdd3 is SubtractedRDD. On Thu, Feb 18, 2016 at 1:37 PM, DaPsul <dap...@gmx.de <mailto:dap...@gmx.de>> wrote: (copy from http://stackoverflow.com/questions/35467128/spark-subtractbykey-increases-rdd-cached-memory-size) I've found a very strange behavior for RDD's

subtractByKey increases RDD size in memory - any ideas?

2016-02-18 Thread DaPsul
(copy from http://stackoverflow.com/questions/35467128/spark-subtractbykey-increases-rdd-cached-memory-size) I've found a very strange behavior for RDD's (spark 1.6.0 with scala 2.11): When i use subtractByKey on an RDD the resulting RDD should be of equal or smaller size. What i get is an RDD