nfeld@uncharted.software>
> wrote:
>
>> Ah, in 2.1.0.
>>
>> I'm in 2.0.1 at the moment... is there any way that works that far back?
>>
>> On Fri, Sep 1, 2017 at 11:46 AM, Nick Pentreath <nick.pentre...@gmail.com
>> > wrote:
>>
>>> Datase
t;
> On Fri, Sep 1, 2017 at 11:46 AM, Nick Pentreath <nick.pentre...@gmail.com>
> wrote:
>
>> Dataset does have storageLevel. So you can use isCached = (storageLevel
>> != StorageLevel.NONE) as a test.
>>
>> Arguably isCached could be added to dataset too, sh
Ah, in 2.1.0.
I'm in 2.0.1 at the moment... is there any way that works that far back?
On Fri, Sep 1, 2017 at 11:46 AM, Nick Pentreath <nick.pentre...@gmail.com>
wrote:
> Dataset does have storageLevel. So you can use isCached = (storageLevel !=
> StorageLevel.NONE) as a test.
Dataset does have storageLevel. So you can use isCached = (storageLevel !=
StorageLevel.NONE) as a test.
Arguably isCached could be added to dataset too, shouldn't be a
controversial change.
On Fri, 1 Sep 2017 at 17:31, Nathan Kronenfeld
<nkronenfeld@uncharted.software> wrote:
> I'm
I'm currently porting some of our code from RDDs to Datasets.
With RDDs it's pretty easy to figure out if they are cached or not.
I notice that the catalog has a function for determining this on Datasets
too, but it's private[sql]. Is there any reason for it not to be public?
Is there any way
Thanks Cheng. Tried it out and saw the InMemoryColumnarTableScan word in the
physical plan.
From: Cheng Lian [mailto:lian.cs@gmail.com]
Sent: Friday, December 12, 2014 11:37 PM
To: Judy Nash; user@spark.apache.org
Subject: Re: Spark SQL API Doc IsCached as SQL command
There isn’t a SQL
Hello,
Few questions on Spark SQL:
1) Does Spark SQL support equivalent SQL Query for Scala command:
IsCached(table name) ?
2) Is there a documentation spec I can reference for question like this?
Closest doc I can find is this one:
https://spark.apache.org/docs/1.1.0/sql
http://spark.apache.org/docs/latest/sql-programming-guide.html#caching-data-in-memory
On Fri, Dec 12, 2014 at 3:14 PM, Judy Nash judyn...@exchange.microsoft.com
wrote:
Hello,
Few questions on Spark SQL:
1) Does Spark SQL support equivalent SQL Query for Scala command:
IsCached
equivalent SQL Query for Scala command:
IsCached(table name) ?
2)Is there a documentation spec I can reference for question like this?
Closest doc I can find is this one:
https://spark.apache.org/docs/1.1.0/sql-programming-guide.html#deploying-in-existing-hive-warehouses
Thanks,
Judy
