RE: Size of RDD larger than Size of data on disk

2014-02-25 Thread Suraj Satishkumar Sheth
[mailto:mayur.rust...@gmail.com] Sent: Tuesday, February 25, 2014 11:19 PM To: user@spark.apache.org Cc: u...@spark.incubator.apache.org Subject: Re: Size of RDD larger than Size of data on disk Spark may take more RAM than reqiured by RDD, can you look at storage section of Spark see how much space RDD

Re: Size of RDD larger than Size of data on disk

2014-02-25 Thread Matei Zaharia
, Suraj Sheth From: Mayur Rustagi [mailto:mayur.rust...@gmail.com] Sent: Tuesday, February 25, 2014 11:19 PM To: user@spark.apache.org Cc: u...@spark.incubator.apache.org Subject: Re: Size of RDD larger than Size of data on disk Spark may take more RAM than reqiured by RDD, can you look