I'm experiencing the same error while upgrading from 0.9.1 to 1.0.0.
Im using google compute engine and cloud storage. but saveAsTextFile is
returning errors while saving in the cloud or saving local. When i start a
job in the cluster i do get an error but after this error it keeps on
running fine untill the saveAsTextFile. ( I don't know if the two are
connected)
-----------Error at job startup-------
ERROR metrics.MetricsSystem: Sink class
org.apache.spark.metrics.sink.MetricsServlet cannot be instantialized
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:136)
at
org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:130)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
at
scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
at
org.apache.spark.metrics.MetricsSystem.registerSinks(MetricsSystem.scala:130)
at
org.apache.spark.metrics.MetricsSystem.<init>(MetricsSystem.scala:84)
at
org.apache.spark.metrics.MetricsSystem$.createMetricsSystem(MetricsSystem.scala:167)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:230)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:202)
at Hello$.main(Hello.scala:101)
at Hello.main(Hello.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sbt.Run.invokeMain(Run.scala:72)
at sbt.Run.run0(Run.scala:65)
at sbt.Run.sbt$Run$$execute$1(Run.scala:54)
at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:58)
at sbt.Run$$anonfun$run$1.apply(Run.scala:58)
at sbt.Run$$anonfun$run$1.apply(Run.scala:58)
at sbt.Logger$$anon$4.apply(Logger.scala:90)
at sbt.TrapExit$App.run(TrapExit.scala:244)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NoSuchMethodError:
com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
at
com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:445)
at
com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:366)
at
org.apache.spark.metrics.sink.MetricsServlet.<init>(MetricsServlet.scala:45)
... 31 more
then it runs fine till i get to saveAsTextFile
14/06/06 09:05:12 INFO scheduler.TaskSetManager: Loss was due to
java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1 [duplicate 17]
14/06/06 09:05:12 INFO scheduler.DAGScheduler: Failed to run saveAsTextFile
at Hello.scala:123
14/06/06 09:05:12 INFO scheduler.TaskSchedulerImpl: Cancelling stage 0
[error] (run-main-0) org.apache.spark.SparkException: Job aborted due to
stage failure: Task 0.0:3 failed 4 times, most recent failure: Exception
failure in TID 142 on host sparky-s1.c.quick-heaven-560.internal:
java.lang.ClassNotFoundException:
org.apache.spark.rdd.RDD$$anonfun$saveAsTextFile$1
[error] java.net.URLClassLoader$1.run(URLClassLoader.java:366)
[error] java.net.URLClassLoader$1.run(URLClassLoader.java:355)
[error] java.security.AccessController.doPrivileged(Native Method)
[error] java.net.URLClassLoader.findClass(URLClassLoader.java:354)
[error] java.lang.ClassLoader.loadClass(ClassLoader.java:425)
[error] java.lang.ClassLoader.loadClass(ClassLoader.java:358)
[error] java.lang.Class.forName0(Native Method)
[error] java.lang.Class.forName(Class.java:270)
[error]
org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:60)
[error]
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
[error]
java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
[error]
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
[error]
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
[error]
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
[error]
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
[error]
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
[error]
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
[error]
java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
[error]
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
[error]
org.apache.spark.scheduler.ResultTask$.deserializeInfo(ResultTask.scala:61)
[error]
org.apache.spark.scheduler.ResultTask.readExternal(ResultTask.scala:141)
[error]
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)
[error]
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
[error]
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
[error]
java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
[error]
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
[error]
org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:85)
[error]
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:169)
[error]
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[error]
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[error] java.lang.Thread.run(Thread.java:744)
Thanks for any help or guidance.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Strange-problem-with-saveAsTextFile-after-upgrade-Spark-0-9-0-1-0-0-tp6832p7122.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
