Hi I have installed a standalone Spark set up in standalone mode in a Linux server and I am trying to access that spark setup from Java in windows. When I try connecting to Spark I see the following exception
14/12/16 12:52:52 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory 14/12/16 12:52:56 INFO AppClient$ClientActor: Connecting to master spark://01hw294954.INDIA:7077... 14/12/16 12:53:07 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory 14/12/16 12:53:16 INFO AppClient$ClientActor: Connecting to master spark://01hw294954.INDIA:7077... 14/12/16 12:53:22 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory 14/12/16 12:53:36 ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up. 14/12/16 12:53:36 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 14/12/16 12:53:36 INFO TaskSchedulerImpl: Cancelling stage 0 14/12/16 12:53:36 INFO DAGScheduler: Failed to run collect at MySqlConnector.java:579 Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: All masters are unresponsive! Giving up. at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1033) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1017) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1015) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1015) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:633) at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1207) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) at akka.actor.ActorCell.invoke(ActorCell.scala:456) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) at akka.dispatch.Mailbox.run(Mailbox.scala:219) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run I have attached the Spark Master UI Spark Master at spark://01hw294954.INDIA:7077 URL: spark://01hw294954.INDIA:7077 Workers: 1 Cores: 2 Total, 0 Used Memory: 835.0 MB Total, 0.0 B Used Applications: 0 Running, 0 Completed Drivers: 0 Running, 0 Completed Status: ALIVE Workers Id Address State Cores Memory worker-20141216123503-01hw294954.INDIA-38962 01hw294954.INDIA:38962 ALIVE 2 (0 Used) 835.0 MB (0.0 B Used) Running Applications ID Name Cores Memory per Node Submitted Time User State Duration Completed Applications ID Name Cores Memory per Node Submitted Time User State Duration My Spark Slave is Spark Worker at 01hw294954.INDIA:38962 ID: worker-20141216123503-01hw294954.INDIA-38962 Master URL: spark://01hw294954.INDIA:7077 Cores: 2 (0 Used) Memory: 835.0 MB (0.0 B Used) Back to Master Running Executors (0) ExecutorID Cores State Memory Job Details Logs My Java Master Code looks like this SparkConf sparkConf = new SparkConf().setAppName("JdbcRddTest"); sparkConf.setMaster("spark://01hw294954.INDIA:7077"); When I tried using the same code with the local spark set up as the master it ran. Any help for solving this issue is very much appreciated. Thanks and Regards Jai -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Accessing-Apache-Spark-from-Java-tp20700.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org