Hi want to try new tungsten-sort shuffle manager, but on 1 stage
executors start to die with NPE:
15/06/19 17:53:35 WARN TaskSetManager: Lost task 38.0 in stage 41.0 (TID
3176, ip-10-50-225-214.ec2.internal): java.lang.NullPointerException
at
org.apache.spark.shuffle.unsafe.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:151)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:70)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Any suggestions?
Thanks,
Peter Rudenko