Re: enable debug-level log output of akka?

2015-01-14 Thread Nan Zhu 
I quickly went through the code,  

In ExecutorBackend, we build the actor system with  

// Create SparkEnv using properties we fetched from the driver.
val driverConf = new SparkConf().setAll(props)
val env = SparkEnv.createExecutorEnv(
driverConf, executorId, hostname, port, cores, isLocal = false)

props is the spark properties fetched from driver,  

In Driver side, we start driver actor with  

val properties = new ArrayBuffer[(String, String)]
for ((key, value) - scheduler.sc.conf.getAll) {
if (key.startsWith(spark.)) {
properties += ((key, value))
}
}
// TODO (prashant) send conf instead of properties
driverActor = actorSystem.actorOf(
Props(new DriverActor(properties)), name = 
CoarseGrainedSchedulerBackend.ACTOR_NAME)
This “properties” is the stuff being sent to executor. It seems that we only 
admit the properties starting with “spark.”

It seems that we cannot pass akka.* to executor?
Hi, Josh, would you mind giving some hints, as you created and closed the JIRA?
Best,


--  
Nan Zhu



On Wednesday, January 14, 2015 at 6:19 PM, Nan Zhu wrote:

 Hi, Ted,  
  
 Thanks
  
 I know how to set in Akka’s context, my question is just how to pass this 
 aka.loglevel=DEBUG to Spark’s actor system
  
 Best,
 --  
 Nan Zhu
 http://codingcat.me
  
  
 On Wednesday, January 14, 2015 at 6:09 PM, Ted Yu wrote:
  
  I assume you have looked at:
   
  http://doc.akka.io/docs/akka/2.0/scala/logging.html
  http://doc.akka.io/docs/akka/current/additional/faq.html (Debugging, last 
  question)
   
  Cheers
   
  On Wed, Jan 14, 2015 at 2:55 PM, Nan Zhu zhunanmcg...@gmail.com 
  (mailto:zhunanmcg...@gmail.com) wrote:
   Hi, all

   though https://issues.apache.org/jira/browse/SPARK-609 was closed, I’m 
   still unclear about how to enable debug level log output in Spark’s actor 
   system  

   Anyone can give the suggestion?

   BTW, I think we need to document it on somewhere, as the user who writes 
   actor-based receiver of spark streaming, (like me), usually needs the 
   detailed log for debugging….

   Best,  

   --  
   Nan Zhu
   http://codingcat.me

Re: enable debug-level log output of akka?

2015-01-14 Thread Nan Zhu 
sorry for the mistake,  

I found that those akka related messages are from Spark Akka-related component 
(ActorLogReceive) , instead of Akka itself, though it has been enough for the 
debugging purpose (in my case)

the question in this thread is still in open status….

Best,  

--  
Nan Zhu
http://codingcat.me


On Wednesday, January 14, 2015 at 6:56 PM, Nan Zhu wrote:

 for others who have the same question:
  
 you can simply set logging level in log4j.properties to DEBUG to achieve this
  
 Best,  
  
 --  
 Nan Zhu
 http://codingcat.me
  
  
 On Wednesday, January 14, 2015 at 6:28 PM, Nan Zhu wrote:
  
  I quickly went through the code,  
   
  In ExecutorBackend, we build the actor system with  
   
  // Create SparkEnv using properties we fetched from the driver.
  val driverConf = new SparkConf().setAll(props)
  val env = SparkEnv.createExecutorEnv(
  driverConf, executorId, hostname, port, cores, isLocal = false)
   
  props is the spark properties fetched from driver,  
   
  In Driver side, we start driver actor with  
   
  val properties = new ArrayBuffer[(String, String)]
  for ((key, value) - scheduler.sc (http://scheduler.sc).conf.getAll) {
  if (key.startsWith(spark.)) {
  properties += ((key, value))
  }
  }
  // TODO (prashant) send conf instead of properties
  driverActor = actorSystem.actorOf(
  Props(new DriverActor(properties)), name = 
  CoarseGrainedSchedulerBackend.ACTOR_NAME)
  This “properties” is the stuff being sent to executor. It seems that we 
  only admit the properties starting with “spark.”
   
  It seems that we cannot pass akka.* to executor?
  Hi, Josh, would you mind giving some hints, as you created and closed the 
  JIRA?
  Best,
   
   
  --  
  Nan Zhu
   
   
  On Wednesday, January 14, 2015 at 6:19 PM, Nan Zhu wrote:
   
   Hi, Ted,  

   Thanks

   I know how to set in Akka’s context, my question is just how to pass this 
   aka.loglevel=DEBUG to Spark’s actor system

   Best,
   --  
   Nan Zhu
   http://codingcat.me


   On Wednesday, January 14, 2015 at 6:09 PM, Ted Yu wrote:

I assume you have looked at:
 
http://doc.akka.io/docs/akka/2.0/scala/logging.html
http://doc.akka.io/docs/akka/current/additional/faq.html (Debugging, 
last question)
 
Cheers
 
On Wed, Jan 14, 2015 at 2:55 PM, Nan Zhu zhunanmcg...@gmail.com 
(mailto:zhunanmcg...@gmail.com) wrote:
 Hi, all
  
 though https://issues.apache.org/jira/browse/SPARK-609 was closed, 
 I’m still unclear about how to enable debug level log output in 
 Spark’s actor system  
  
 Anyone can give the suggestion?
  
 BTW, I think we need to document it on somewhere, as the user who 
 writes actor-based receiver of spark streaming, (like me), usually 
 needs the detailed log for debugging….
  
 Best,  
  
 --  
 Nan Zhu
 http://codingcat.me

Re: enable debug-level log output of akka?

2015-01-14 Thread Ted Yu 
I assume you have looked at:

http://doc.akka.io/docs/akka/2.0/scala/logging.html
http://doc.akka.io/docs/akka/current/additional/faq.html (Debugging, last
question)

Cheers

On Wed, Jan 14, 2015 at 2:55 PM, Nan Zhu zhunanmcg...@gmail.com wrote:

  Hi, all

 though https://issues.apache.org/jira/browse/SPARK-609 was closed, I’m
 still unclear about how to enable debug level log output in Spark’s actor
 system

 Anyone can give the suggestion?

 BTW, I think we need to document it on somewhere, as the user who writes
 actor-based receiver of spark streaming, (like me), usually needs the
 detailed log for debugging….

 Best,

 --
 Nan Zhu
 http://codingcat.me

Re: enable debug-level log output of akka?

2015-01-14 Thread Nan Zhu 
for others who have the same question:

you can simply set logging level in log4j.properties to DEBUG to achieve this

Best,  

--  
Nan Zhu
http://codingcat.me


On Wednesday, January 14, 2015 at 6:28 PM, Nan Zhu wrote:

 I quickly went through the code,  
  
 In ExecutorBackend, we build the actor system with  
  
 // Create SparkEnv using properties we fetched from the driver.
 val driverConf = new SparkConf().setAll(props)
 val env = SparkEnv.createExecutorEnv(
 driverConf, executorId, hostname, port, cores, isLocal = false)
  
 props is the spark properties fetched from driver,  
  
 In Driver side, we start driver actor with  
  
 val properties = new ArrayBuffer[(String, String)]
 for ((key, value) - scheduler.sc (http://scheduler.sc).conf.getAll) {
 if (key.startsWith(spark.)) {
 properties += ((key, value))
 }
 }
 // TODO (prashant) send conf instead of properties
 driverActor = actorSystem.actorOf(
 Props(new DriverActor(properties)), name = 
 CoarseGrainedSchedulerBackend.ACTOR_NAME)
 This “properties” is the stuff being sent to executor. It seems that we only 
 admit the properties starting with “spark.”
  
 It seems that we cannot pass akka.* to executor?
 Hi, Josh, would you mind giving some hints, as you created and closed the 
 JIRA?
 Best,
  
  
 --  
 Nan Zhu
  
  
 On Wednesday, January 14, 2015 at 6:19 PM, Nan Zhu wrote:
  
  Hi, Ted,  
   
  Thanks
   
  I know how to set in Akka’s context, my question is just how to pass this 
  aka.loglevel=DEBUG to Spark’s actor system
   
  Best,
  --  
  Nan Zhu
  http://codingcat.me
   
   
  On Wednesday, January 14, 2015 at 6:09 PM, Ted Yu wrote:
   
   I assume you have looked at:

   http://doc.akka.io/docs/akka/2.0/scala/logging.html
   http://doc.akka.io/docs/akka/current/additional/faq.html (Debugging, last 
   question)

   Cheers

   On Wed, Jan 14, 2015 at 2:55 PM, Nan Zhu zhunanmcg...@gmail.com 
   (mailto:zhunanmcg...@gmail.com) wrote:
Hi, all
 
though https://issues.apache.org/jira/browse/SPARK-609 was closed, I’m 
still unclear about how to enable debug level log output in Spark’s 
actor system  
 
Anyone can give the suggestion?
 
BTW, I think we need to document it on somewhere, as the user who 
writes actor-based receiver of spark streaming, (like me), usually 
needs the detailed log for debugging….
 
Best,  
 
--  
Nan Zhu
http://codingcat.me

Re: enable debug-level log output of akka?

2015-01-14 Thread Nan Zhu 
Hi, Ted,  

Thanks

I know how to set in Akka’s context, my question is just how to pass this 
aka.loglevel=DEBUG to Spark’s actor system

Best,
--  
Nan Zhu
http://codingcat.me


On Wednesday, January 14, 2015 at 6:09 PM, Ted Yu wrote:

 I assume you have looked at:
  
 http://doc.akka.io/docs/akka/2.0/scala/logging.html
 http://doc.akka.io/docs/akka/current/additional/faq.html (Debugging, last 
 question)
  
 Cheers
  
 On Wed, Jan 14, 2015 at 2:55 PM, Nan Zhu zhunanmcg...@gmail.com 
 (mailto:zhunanmcg...@gmail.com) wrote:
  Hi, all
   
  though https://issues.apache.org/jira/browse/SPARK-609 was closed, I’m 
  still unclear about how to enable debug level log output in Spark’s actor 
  system  
   
  Anyone can give the suggestion?
   
  BTW, I think we need to document it on somewhere, as the user who writes 
  actor-based receiver of spark streaming, (like me), usually needs the 
  detailed log for debugging….
   
  Best,  
   
  --  
  Nan Zhu
  http://codingcat.me

enable debug-level log output of akka?

2015-01-14 Thread Nan Zhu 
Hi, all

though https://issues.apache.org/jira/browse/SPARK-609 was closed, I’m still 
unclear about how to enable debug level log output in Spark’s actor system  

Anyone can give the suggestion?

BTW, I think we need to document it on somewhere, as the user who writes 
actor-based receiver of spark streaming, (like me), usually needs the detailed 
log for debugging….

Best,  

--  
Nan Zhu
http://codingcat.me