actors and async communication between driver and workers/executors

2015-12-08 Thread Manolis Sifalakis1
I ve been looking around for some examples of information of how can the driver and the executors exchange information asynchronously, but have not found much apart from the ActorWordCount.scala streaming example that uses Akka. Is there any "in-band" (within Spark) method that such communicati

Re: understanding and disambiguating CPU-core related properties

2015-12-08 Thread Manolis Sifalakis1
Thanks lots for the pointer! Helpful even though a bit layman's style. (On the nagging end, this information as usual with spark, is not were it is expected to be: neither the book nor the spark doc) m. From: Leonidas Patouchas To: Manolis Sifalakis1 Cc: user@spark.apach

understanding and disambiguating CPU-core related properties

2015-12-03 Thread Manolis Sifalakis1
I have found the documentation rather poor in helping me understand the interplay among the following properties in spark, even more how to set them. So this post is sent in hope for some discussion and "enlightenment" on the topic Let me start by asking if I have understood well the following: