No unfortunately not - as i recall storageLevel accesses some private methods to get the result.
On Fri, 1 Sep 2017 at 17:55, Nathan Kronenfeld <nkronenfeld@uncharted.software> wrote: > Ah, in 2.1.0. > > I'm in 2.0.1 at the moment... is there any way that works that far back? > > On Fri, Sep 1, 2017 at 11:46 AM, Nick Pentreath <nick.pentre...@gmail.com> > wrote: > >> Dataset does have storageLevel. So you can use isCached = (storageLevel >> != StorageLevel.NONE) as a test. >> >> Arguably isCached could be added to dataset too, shouldn't be a >> controversial change. >> >> On Fri, 1 Sep 2017 at 17:31, Nathan Kronenfeld >> <nkronenfeld@uncharted.software> wrote: >> >>> I'm currently porting some of our code from RDDs to Datasets. >>> >>> With RDDs it's pretty easy to figure out if they are cached or not. >>> >>> I notice that the catalog has a function for determining this on >>> Datasets too, but it's private[sql]. Is there any reason for it not to be >>> public? Is there any way at the moment to determine if a dataset is cached >>> or not? >>> >>> Thanks in advance >>> -Nathan Kronenfeld >>> >> >
