1. When you call new SparkContext(), spark driver is started which internally create Akka ActorSystem which registers on this port.
2. Since you are running in local mode, starting of executor is short circuited and an Executor object is created in the same process (see LocalEndpoint). This Executor object logs this message with executor ID as "driver". On Wed, Sep 16, 2015 at 9:44 AM, Muler <mulugeta.abe...@gmail.com> wrote: > I'm running Spark in local mode and getting these two log messages who > appear to be similar. I want to understand what each is doing: > > > 1. [main] util.Utils (Logging.scala:logInfo(59)) - Successfully > started service 'sparkDriver' on port 60782. > 2. [main] executor.Executor (Logging.scala:logInfo(59)) - Starting > executor ID driver on host localhost > > 1. is created using: > > val actorSystemName = if (isDriver) driverActorSystemName else > executorActorSystemName > > val rpcEnv = RpcEnv.create(actorSystemName, hostname, port, conf, > securityManager) > val actorSystem = rpcEnv.asInstanceOf[AkkaRpcEnv].actorSystem > > 2. is created when: > > _taskScheduler.start() > > > What is the difference and what does each do? > > >
