[ 
https://issues.apache.org/jira/browse/SPARK-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Rosen resolved SPARK-2520.
-------------------------------
    Resolution: Cannot Reproduce

Resolving as "Cannot reproduce" since this issue is very old. Please re-open or 
file a new ticket if this is still a problem.

> the executor is thrown java.io.StreamCorruptedException
> -------------------------------------------------------
>
>                 Key: SPARK-2520
>                 URL: https://issues.apache.org/jira/browse/SPARK-2520
>             Project: Spark
>          Issue Type: Bug
>          Components: Shuffle
>    Affects Versions: 1.0.0
>            Reporter: Guoqiang Li
>            Priority: Critical
>
> This issue occurs with a very small probability. I can not reproduce it.
> The executor log: 
> {code}
> 14/07/15 21:54:50 INFO spark.MapOutputTrackerMasterActor: Asked to send map 
> output locations for shuffle 0 to spark@sanshan:34429
> 14/07/15 21:54:50 INFO spark.MapOutputTrackerMasterActor: Asked to send map 
> output locations for shuffle 0 to spark@sanshan:31934
> 14/07/15 21:54:50 INFO spark.MapOutputTrackerMasterActor: Asked to send map 
> output locations for shuffle 0 to spark@sanshan:30557
> 14/07/15 21:54:50 INFO spark.MapOutputTrackerMasterActor: Asked to send map 
> output locations for shuffle 0 to spark@sanshan:42606
> 14/07/15 21:54:50 INFO spark.MapOutputTrackerMasterActor: Asked to send map 
> output locations for shuffle 0 to spark@sanshan:37314
> 14/07/15 21:54:50 INFO scheduler.TaskSetManager: Starting task 0.0:166 as TID 
> 4948 on executor 20: tuan221 (PROCESS_LOCAL)
> 14/07/15 21:54:50 INFO scheduler.TaskSetManager: Serialized task 0.0:166 as 
> 3129 bytes in 1 ms
> 14/07/15 21:54:50 WARN scheduler.TaskSetManager: Lost TID 4868 (task 0.0:86)
> 14/07/15 21:54:50 WARN scheduler.TaskSetManager: Loss was due to 
> java.io.StreamCorruptedException
> java.io.StreamCorruptedException: invalid type code: AC
>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1377)
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
>         at 
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
>         at 
> org.apache.spark.serializer.DeserializationStream$$anon$1.getNext(Serializer.scala:125)
>         at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:71)
>         at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
>         at 
> org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:30)
>         at 
> org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)
>         at 
> org.apache.spark.Aggregator.combineCombinersByKey(Aggregator.scala:87)
>         at 
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:101)
>         at 
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$combineByKey$3.apply(PairRDDFunctions.scala:100)
>         at org.apache.spark.rdd.RDD$$anonfun$14.apply(RDD.scala:582)
>         at org.apache.spark.rdd.RDD$$anonfun$14.apply(RDD.scala:582)
>         at 
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>         at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31)
>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>         at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
>         at org.apache.spark.scheduler.Task.run(Task.scala:51)
>         at 
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:187)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)
> 14/07/15 21:54:50 INFO scheduler.TaskSetManager: Starting task 0.0:86 as TID 
> 4949 on executor 20: tuan221 (PROCESS_LOCAL)
> 14/07/15 21:54:50 INFO scheduler.TaskSetManager: Serialized task 0.0:86 as 
> 3129 bytes in 0 ms
> 14/07/15 21:54:50 WARN scheduler.TaskSetManager: Lost TID 4785 (task 0.0:3)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to