Yes. I do see files, actually I missed copying the other settings: spark.master spark:// skarri-lt05.redmond.corp.microsoft.com:7077 spark.eventLog.enabled true spark.rdd.compress true spark.storage.memoryFraction 1 spark.core.connection.ack.wait.timeout 6000 spark.akka.frameSize 50 spark.executor.extraClassPath D:\\Apache\\spark-1.2.1-bin-hadoop2\\spark-1.2.1-bin-hadoop2.4\\bin\\classes spark.eventLog.dir D:/Apache/spark-1.2.1-bin-hadoop2/spark-1.2.1-bin-hadoop2.4/bin/tmp/spark-events spark.history.fs.logDirectory D:/Apache/spark-1.2.1-bin-hadoop2/spark-1.2.1-bin-hadoop2.4/bin/tmp/spark-events
On Wed, Mar 4, 2015 at 10:15 AM, Marcelo Vanzin <van...@cloudera.com> wrote: > On Wed, Mar 4, 2015 at 10:08 AM, Srini Karri <skarri....@gmail.com> wrote: > > spark.executor.extraClassPath > > > D:\\Apache\\spark-1.2.1-bin-hadoop2\\spark-1.2.1-bin-hadoop2.4\\bin\\classes > > spark.eventLog.dir > > > D:/Apache/spark-1.2.1-bin-hadoop2/spark-1.2.1-bin-hadoop2.4/bin/tmp/spark-events > > spark.history.fs.logDirectory > > > D:/Apache/spark-1.2.1-bin-hadoop2/spark-1.2.1-bin-hadoop2.4/bin/tmp/spark-events > > Do you see any files in that directory? > > spark.eventLog.dir won't do anything unless you also have > spark.eventLog.enabled=true somewhere. And these are application > configs, so make sure they're set when running your application (not > when starting the history server). > > -- > Marcelo >
