Spark SQL question: how to control the storage level of cached SchemaRDD?

Thanks for the response. From Spark Web-UI's Storage tab, I do see cached RDD there. But the storage level is Memory Deserialized 1x Replicated. How can I change the storage level? Because I have a big table there. Thanks! From: Cheng Lian

Re: Spark SQL question: how to control the storage level of cached SchemaRDD?

This is not possible until https://github.com/apache/spark/pull/2501 is merged. On Sun, Sep 28, 2014 at 6:39 PM, Haopu Wang hw...@qilinsoft.com wrote: Thanks for the response. From Spark Web-UI's Storage tab, I do see cached RDD there. But the storage level is Memory Deserialized 1x

Re: Spark SQL question: how to control the storage level of cached SchemaRDD?

You might consider instead storing the data using saveAsParquetFile and then querying that after running sqlContext.parquetFile(...).registerTempTable(...). On Sun, Sep 28, 2014 at 6:43 PM, Michael Armbrust mich...@databricks.com wrote: This is not possible until