The table data is cached in block managers on executors. Could you paste
the log on your driver about OOM ?
On Thu, Mar 31, 2016 at 1:24 PM, Soam Acharya <s...@altiscale.com> wrote:
> Hi folks,
>
> I understand that invoking sqlContext.cacheTable("tableName") will load
> the table into a compressed in-memory columnar format. When Spark is
> launched via spark shell in YARN client mode, is the table loaded into the
> local Spark driver process in addition to the executors in the Hadoop
> cluster or is it just loaded into the executors? We're exploring an OOM
> issue on the local Spark driver for some SQL code and was wondering if the
> local cache load could be the culprit.
>
> Appreciate any thoughts. BTW, we're running Spark 1.6.0 on this particular
> cluster.
>
> Regards,
>
> Soam
>
--
Best Regards
Jeff Zhang
