Hi Arush I have configured log4j by updating the file log4j.properties in SPARK_HOME/conf folder.
If it was a log4j defect we would get error in debug mode in all apps. Thanks Ankur Hi Ankur, How are you enabling the debug level of logs. It should be a log4j configuration. Even if there would be some issue it would be in log4j and not in spark. Thanks Arush On Fri, Jan 30, 2015 at 4:24 AM, Ankur Srivastava < ankur.srivast...@gmail.com> wrote: > Hi, > > When ever I enable DEBUG level logs for my spark cluster, on running a job > all the executors die with the below exception. On disabling the DEBUG logs > my jobs move to the next step. > > > I am on spark-1.1.0 > > Is this a known issue with spark? > > Thanks > Ankur > > 2015-01-29 22:27:42,467 [main] INFO org.apache.spark.SecurityManager - > SecurityManager: authentication disabled; ui acls disabled; users with view > permissions: Set(ubuntu); users with modify permissions: Set(ubuntu) > > 2015-01-29 22:27:42,478 [main] DEBUG org.apache.spark.util.AkkaUtils - In > createActorSystem, requireCookie is: off > > 2015-01-29 22:27:42,871 > [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO > akka.event.slf4j.Slf4jLogger - Slf4jLogger started > > 2015-01-29 22:27:42,912 > [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO Remoting - > Starting remoting > > 2015-01-29 22:27:43,057 > [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO Remoting - > Remoting started; listening on addresses :[akka.tcp:// > driverPropsFetcher@10.77.9.155:36035] > > 2015-01-29 22:27:43,060 > [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO Remoting - > Remoting now listens on addresses: [akka.tcp:// > driverPropsFetcher@10.77.9.155:36035] > > 2015-01-29 22:27:43,067 [main] INFO org.apache.spark.util.Utils - > Successfully started service 'driverPropsFetcher' on port 36035. > > 2015-01-29 22:28:13,077 [main] ERROR > org.apache.hadoop.security.UserGroupInformation - > PriviledgedActionException as:ubuntu > cause:java.util.concurrent.TimeoutException: Futures timed out after [30 > seconds] > > Exception in thread "main" java.lang.reflect.UndeclaredThrowableException: > Unknown exception in doAs > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1134) > > at > org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:52) > > at > org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:113) > > at > org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:156) > > at > org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala) > > Caused by: java.security.PrivilegedActionException: > java.util.concurrent.TimeoutException: Futures timed out after [30 seconds] > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > > ... 4 more > > Caused by: java.util.concurrent.TimeoutException: Futures timed out after > [30 seconds] > > at > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) > > at > scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) > > at > scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) > > at > scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) > > at scala.concurrent.Await$.result(package.scala:107) > > at > org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:125) > > at > org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:53) > > at > org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:52) > > ... 7 more > -- [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com> *Arush Kharbanda* || Technical Teamlead ar...@sigmoidanalytics.com || www.sigmoidanalytics.com
