Not a stupid question! I would like to be able to do this. For now, you might try writing the data to tachyon <http://tachyon-project.org/> instead of HDFS. This is untested though, please report any issues you run into.
Michael On Fri, Jun 6, 2014 at 8:13 PM, Xu (Simon) Chen <xche...@gmail.com> wrote: > This might be a stupid question... but it seems that saveAsParquetFile() > writes everything back to HDFS. I am wondering if it is possible to cache > parquet-format intermediate results in memory, and therefore making spark > sql queries faster. > > Thanks. > -Simon >
