Look at the trace again. It is a very weird error. The SparkSubmit is running on client side, but YarnClusterSchedulerBackend is supposed in running in YARN AM.
I suspect you are running the cluster with yarn-client mode, but in JavaSparkContext you set "yarn-cluster”. As a result, spark context initiate YarnClusterSchedulerBackend instead of YarnClientSchedulerBackend, which I think is the root cause. Thanks. Zhan Zhang On Feb 25, 2015, at 1:53 PM, Zhan Zhang <zzh...@hortonworks.com<mailto:zzh...@hortonworks.com>> wrote: Hi Mate, When you initialize the JavaSparkContext, you don’t need to specify the mode “yarn-cluster”. I suspect that is the root cause. Thanks. Zhan Zhang On Feb 25, 2015, at 10:12 AM, gulyasm <mgulya...@gmail.com<mailto:mgulya...@gmail.com>> wrote: JavaSparkContext.
