We had various operational configuration problems with snappy as well so had to disable it for now completely until somebody has time to figure it out (which has been like forever)
On Thu, Feb 19, 2015 at 4:26 PM, Pat Ferrel <p...@occamsmachete.com> wrote: > It seems like after a clean install I fail some test in spark like the one > below. Then when I build again starting with -rf spark everything passes. > I’m not sure its always after a clean but happens often. > > Anyone running into this? > > - C = A cbind B, cogroup *** FAILED *** > org.apache.spark.SparkException: Job aborted due to stage failure: Task > 0 in stage 205.0 failed 1 times, most recent failure: Lost task 0.0 in > stage 205.0 (TID 442, localhost): java.io.IOException: PARSING_ERROR(2) > org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:78) > org.xerial.snappy.SnappyNative.uncompressedLength(Native Method) > org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:545) > > org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:125) > > org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:88) > > org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:58) > > org.apache.spark.io.SnappyCompressionCodec.compressedInputStream(CompressionCodec.scala:128) > > org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:232) > > org.apache.spark.broadcast.TorrentBroadcast.readObject(TorrentBroadcast.scala:169) > sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > java.lang.reflect.Method.invoke(Method.java:606) > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) > java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) > > org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62) > > org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:87) > > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:159) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > java.lang.Thread.run(Thread.java:745) > Driver stacktrace: > at org.apache.spark.scheduler.DAGScheduler.org > $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173) > at > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) > at > org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at scala.Option.foreach(Option.scala:236) > at > org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688) > ... > - C = A cbind B, zip > >
