Hi Ankur, Its running fine for me for spark 1.1 and changes to log4j properties file.
Thanks Arush On Fri, Jan 30, 2015 at 9:49 PM, Ankur Srivastava < ankur.srivast...@gmail.com> wrote: > Hi Arush > > I have configured log4j by updating the file log4j.properties in > SPARK_HOME/conf folder. > > If it was a log4j defect we would get error in debug mode in all apps. > > Thanks > Ankur > Hi Ankur, > > How are you enabling the debug level of logs. It should be a log4j > configuration. Even if there would be some issue it would be in log4j and > not in spark. > > Thanks > Arush > > On Fri, Jan 30, 2015 at 4:24 AM, Ankur Srivastava < > ankur.srivast...@gmail.com> wrote: > >> Hi, >> >> When ever I enable DEBUG level logs for my spark cluster, on running a >> job all the executors die with the below exception. On disabling the DEBUG >> logs my jobs move to the next step. >> >> >> I am on spark-1.1.0 >> >> Is this a known issue with spark? >> >> Thanks >> Ankur >> >> 2015-01-29 22:27:42,467 [main] INFO org.apache.spark.SecurityManager - >> SecurityManager: authentication disabled; ui acls disabled; users with view >> permissions: Set(ubuntu); users with modify permissions: Set(ubuntu) >> >> 2015-01-29 22:27:42,478 [main] DEBUG org.apache.spark.util.AkkaUtils - In >> createActorSystem, requireCookie is: off >> >> 2015-01-29 22:27:42,871 >> [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO >> akka.event.slf4j.Slf4jLogger - Slf4jLogger started >> >> 2015-01-29 22:27:42,912 >> [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO Remoting - >> Starting remoting >> >> 2015-01-29 22:27:43,057 >> [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO Remoting - >> Remoting started; listening on addresses :[akka.tcp:// >> driverPropsFetcher@10.77.9.155:36035] >> >> 2015-01-29 22:27:43,060 >> [driverPropsFetcher-akka.actor.default-dispatcher-4] INFO Remoting - >> Remoting now listens on addresses: [akka.tcp:// >> driverPropsFetcher@10.77.9.155:36035] >> >> 2015-01-29 22:27:43,067 [main] INFO org.apache.spark.util.Utils - >> Successfully started service 'driverPropsFetcher' on port 36035. >> >> 2015-01-29 22:28:13,077 [main] ERROR >> org.apache.hadoop.security.UserGroupInformation - >> PriviledgedActionException as:ubuntu >> cause:java.util.concurrent.TimeoutException: Futures timed out after [30 >> seconds] >> >> Exception in thread "main" >> java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs >> >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1134) >> >> at >> org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:52) >> >> at >> org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:113) >> >> at >> org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:156) >> >> at >> org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala) >> >> Caused by: java.security.PrivilegedActionException: >> java.util.concurrent.TimeoutException: Futures timed out after [30 seconds] >> >> at java.security.AccessController.doPrivileged(Native Method) >> >> at javax.security.auth.Subject.doAs(Subject.java:415) >> >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) >> >> ... 4 more >> >> Caused by: java.util.concurrent.TimeoutException: Futures timed out after >> [30 seconds] >> >> at >> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) >> >> at >> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) >> >> at >> scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) >> >> at >> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) >> >> at scala.concurrent.Await$.result(package.scala:107) >> >> at >> org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:125) >> >> at >> org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:53) >> >> at >> org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:52) >> >> ... 7 more >> > > > > -- > > [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com> > > *Arush Kharbanda* || Technical Teamlead > > ar...@sigmoidanalytics.com || www.sigmoidanalytics.com > -- [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com> *Arush Kharbanda* || Technical Teamlead ar...@sigmoidanalytics.com || www.sigmoidanalytics.com