I'm seeing the following message in the log of an executor. Anyone
seen this error? After this, the executor seems to lose the cache, and
but besides that the whole thing slows down drastically - I.e. it gets
stuck in a reduce phase for 40+ minutes, whereas before it was
finishing reduces in 2~3 seconds.
14/06/25 19:22:31 WARN SendingConnection: Error writing in connection
to ConnectionManagerId(alpinenode7.alpinenow.local,46251)
java.lang.NullPointerException
at
org.apache.spark.network.MessageChunkHeader.buffer$lzycompute(MessageChunkHeader.scala:35)
at
org.apache.spark.network.MessageChunkHeader.buffer(MessageChunkHeader.scala:32)
at
org.apache.spark.network.MessageChunk.buffers$lzycompute(MessageChunk.scala:31)
at org.apache.spark.network.MessageChunk.buffers(MessageChunk.scala:29)
at
org.apache.spark.network.SendingConnection.write(Connection.scala:349)
at
org.apache.spark.network.ConnectionManager$$anon$5.run(ConnectionManager.scala:142)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
