I think you have to using "604800" instead of "7 * 24 * 3600", obviously SparkConf will not do multiplication for you..
The exception is quite obvious: "Caused by: java.lang.NumberFormatException: For input string: "3 * 24 * 3600"" 2015-06-16 14:52 GMT+08:00 <luohui20...@sina.com>: > Hi guys: > > I added a parameter "spark.worker.cleanup.appDataTtl 3 * 24 * > 3600" in my conf/spark-default.conf, then I start my spark cluster. However > I got an exception: > > > 15/06/16 14:25:14 INFO util.Utils: Successfully started service > 'sparkWorker' on port 43344. > 15/06/16 14:25:14 ERROR actor.OneForOneStrategy: exception during creation > akka.actor.ActorInitializationException: exception during creation > at akka.actor.ActorInitializationException$.apply(Actor.scala:164) > at akka.actor.ActorCell.create(ActorCell.scala:596) > at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456) > at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478) > at > akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263) > at akka.dispatch.Mailbox.run(Mailbox.scala:219) > at > akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) > at > scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at akka.util.Reflect$.instantiate(Reflect.scala:66) > at akka.actor.ArgsReflectConstructor.produce(Props.scala:352) > at akka.actor.Props.newActor(Props.scala:252) > at akka.actor.ActorCell.newActor(ActorCell.scala:552) > at akka.actor.ActorCell.create(ActorCell.scala:578) > ... 9 more > Caused by: java.lang.NumberFormatException: For input string: "3 * 24 * > 3600" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Long.parseLong(Long.java:589) > at java.lang.Long.parseLong(Long.java:631) > at > scala.collection.immutable.StringLike$class.toLong(StringLike.scala:230) > at scala.collection.immutable.StringOps.toLong(StringOps.scala:31) > at > org.apache.spark.SparkConf$$anonfun$getLong$2.apply(SparkConf.scala:194) > at > org.apache.spark.SparkConf$$anonfun$getLong$2.apply(SparkConf.scala:194) > at scala.Option.map(Option.scala:145) > at org.apache.spark.SparkConf.getLong(SparkConf.scala:194) > at org.apache.spark.deploy.worker.Worker.<init>(Worker.scala:89) > ... 18 more > > > How to set this parameter correctly? > > BTW, I searched this property in > http://spark.apache.org/docs/latest/configuration.html and got no match. > This property was found in > http://spark.apache.org/docs/latest/spark-standalone.html with a default > value "7 * 24 * 3600 (7 days)", which I also tried but also failed. > > Thanks > > > -------------------------------- > > Thanks&Best regards! > San.Luo >
