RE: Spark SQL API Doc IsCached as SQL command

2014-12-16 Thread Judy Nash
Thanks Cheng. Tried it out and saw the InMemoryColumnarTableScan word in the physical plan. From: Cheng Lian [mailto:lian.cs@gmail.com] Sent: Friday, December 12, 2014 11:37 PM To: Judy Nash; user@spark.apache.org Subject: Re: Spark SQL API Doc IsCached as SQL command There isn’t a SQL

Spark SQL API Doc IsCached as SQL command

2014-12-12 Thread Judy Nash
Hello, Few questions on Spark SQL: 1) Does Spark SQL support equivalent SQL Query for Scala command: IsCached(table name) ? 2) Is there a documentation spec I can reference for question like this? Closest doc I can find is this one:

Re: Spark SQL API Doc IsCached as SQL command

2014-12-12 Thread Mark Hamstra
http://spark.apache.org/docs/latest/sql-programming-guide.html#caching-data-in-memory On Fri, Dec 12, 2014 at 3:14 PM, Judy Nash judyn...@exchange.microsoft.com wrote: Hello, Few questions on Spark SQL: 1) Does Spark SQL support equivalent SQL Query for Scala command:

Re: Spark SQL API Doc IsCached as SQL command

2014-12-12 Thread Cheng Lian
There isn’t a SQL statement that directly maps |SQLContext.isCached|, but you can use |EXPLAIN EXTENDED| to check whether the underlying physical plan is a |InMemoryColumnarTableScan|. On 12/13/14 7:14 AM, Judy Nash wrote: Hello, Few questions on Spark SQL: 1)Does Spark SQL support