I see, I guess I should have set the historyServer.
Strangely enough peeking in the yarn seems like nothing is "happening", it list a single application running with 0% progress but each node has 0 running containers which confuses me to wether anything is actually happening Should I restart the job with the spark.yarn.historyServer.address ? [hadoop@sslabnode02 ~]$ yarn node -list 16/01/04 14:43:40 INFO client.RMProxy: Connecting to ResourceManager at sslabnode01/10.15.235.239:8032 16/01/04 14:43:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Total Nodes:3 Node-Id Node-State Node-Http-Address Number-of-Running-Containers sslabnode02:54142 RUNNING sslabnode02:8042 0 sslabnode01:60780 RUNNING sslabnode01:8042 0 sslabnode03:60569 RUNNING sslabnode03:8042 0 [hadoop@sslabnode02 ~]$ yarn application -list 16/01/04 14:43:55 INFO client.RMProxy: Connecting to ResourceManager at sslabnode01/10.15.235.239:8032 16/01/04 14:43:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):1 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1451947397662_0001 ClusterIncidents SPARK hadoop default ACCEPTED UNDEFINED 0% N/A > On Jan 4, 2016, at 2:55 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > Please look at history server related content under: > https://spark.apache.org/docs/latest/running-on-yarn.html > <https://spark.apache.org/docs/latest/running-on-yarn.html> > > Note spark.yarn.historyServer.address > FYI > > On Mon, Jan 4, 2016 at 2:49 PM, Daniel Valdivia <h...@danielvaldivia.com > <mailto:h...@danielvaldivia.com>> wrote: > Hello everyone, happy new year, > > I submitted an app to yarn, however I'm unable to monitor it's progress on > the driver node, not in <drive_address>:8080 or <drive_address>:4040 as > documented, when submitting to the standalone mode I could monitor however > seems liek its not the case right now. > > I submitted my app this way: > > spark-submit --class my.class --master yarn --deploy-mode cluster myjar.jar > > and so far the job is on it's way it seems, the console is vivid with > Application report messages, however I can't access the status of the app, > should I have submitted the app in a different fashion to access the status > of it? > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > <mailto:user-unsubscr...@spark.apache.org> > For additional commands, e-mail: user-h...@spark.apache.org > <mailto:user-h...@spark.apache.org> > >