You'll need to look at the driver output to have a better idea of what's going on. You can use "yarn logs --applicationId blah" after your app is finished (e.g. by killing it) to look at it.
My guess is that your cluster doesn't have enough resources available to service the container request you're making. That will show up in the driver as periodic messages that no containers have been allocated yet. On Wed, Sep 24, 2014 at 10:25 AM, Raghuveer Chanda <raghuveer.cha...@gmail.com> wrote: > Hi, > > I'm new to spark and facing problem with running a job in cluster using > YARN. > > Initially i ran jobs using spark master as --master spark://dml2:7077 and it > is running fine on 3 workers. > > But now im shifting to YARN, so installed YARN in cloud era on 3 node > cluster and changed the master to yarn-cluster but it is not working I > attached the screenshots of UI which are not progressing and just hanging > on. > > Output on terminal : > > This error is repeating > > ./spark-submit --class "class-name" --master yarn-cluster --num-executors 3 > --executor-cores 3 jar-with-dependencies.jar > > > Do i need to configure YARN or why it is not getting all the workers .. > please help ... > > > 14/09/24 22:44:21 INFO yarn.Client: Application report from ASM: > application identifier: application_1411578463780_0001 > appId: 1 > clientToAMToken: null > appDiagnostics: > appMasterHost: dml3 > appQueue: root.chanda > appMasterRpcPort: 0 > appStartTime: 1411578513545 > yarnAppState: RUNNING > distributedFinalState: UNDEFINED > appTrackingUrl: http://dml2:8088/proxy/application_1411578463780_0001/ > appUser: chanda > 14/09/24 22:44:22 INFO yarn.Client: Application report from ASM: > application identifier: application_1411578463780_0001 > appId: 1 > clientToAMToken: null > appDiagnostics: > appMasterHost: dml3 > appQueue: root.chanda > appMasterRpcPort: 0 > appStartTime: 1411578513545 > yarnAppState: RUNNING > distributedFinalState: UNDEFINED > appTrackingUrl: http://dml2:8088/proxy/application_1411578463780_0001/ > > > > > -- > Regards, > Raghuveer Chanda > 4th year Undergraduate Student > Computer Science and Engineering > IIT Kharagpur > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
