Re: Class cast exception while using Data Frames

Can you give the output of “printSchema” ? > On 26 Mar 2018, at 22:39, Nikhil Goyal wrote: > > Hi guys, > > I have a Map[(String, String), Double] as one of my columns. Using > input.getAs[Map[(String, String), Double]](0) > throws exception: Caused by:

Re: Local dirs

Thanks > On 26 Mar 2018, at 22:09, Marcelo Vanzin <van...@cloudera.com> wrote: > > On Mon, Mar 26, 2018 at 1:08 PM, Gauthier Feuillen > <gauth...@dataroots.io> wrote: >> Is there a way to change this value without changing yarn-site.xml ? > > No. Local

Local dirs

Hi I am trying to change the spark.local.dir property. I am running spark on yarn and have already tried the following properties: export LOCAL_DIRS= spark.yarn.appMasterEnv.LOCAL_DIRS= spark.yarn.appMasterEnv.SPARK_LOCAL_DIRS= spark.yarn.nodemanager.local-dirs=/ spark.local.dir= But still it