This is not possible until https://github.com/apache/spark/pull/2501 is merged.
On Sun, Sep 28, 2014 at 6:39 PM, Haopu Wang <hw...@qilinsoft.com> wrote: > Thanks for the response. From Spark Web-UI's Storage tab, I do see > cached RDD there. > > > > But the storage level is "Memory Deserialized 1x Replicated". How can I > change the storage level? Because I have a big table there. > > > > Thanks! > > > ------------------------------ > > *From:* Cheng Lian [mailto:lian.cs....@gmail.com] > *Sent:* 2014年9月26日 21:24 > *To:* Haopu Wang; user@spark.apache.org > *Subject:* Re: Spark SQL question: is cached SchemaRDD storage controlled > by "spark.storage.memoryFraction"? > > > > Yes it is. The in-memory storage used with SchemaRDD also uses RDD.cache() > under the hood. > > On 9/26/14 4:04 PM, Haopu Wang wrote: > > Hi, I'm querying a big table using Spark SQL. I see very long GC time in > > some stages. I wonder if I can improve it by tuning the storage > > parameter. > > > > The question is: the schemaRDD has been cached with "cacheTable()" > > function. So is the cached schemaRDD part of memory storage controlled > > by the "spark.storage.memoryFraction" parameter? > > > > Thanks! > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > > For additional commands, e-mail: user-h...@spark.apache.org > > > > >
