Hi Nick,
I think it is due to a bug in UnsafeKVExternalSorter. I created a Jira and a
PR for this bug:
https://issues.apache.org/jira/browse/SPARK-18800
-
Liang-Chi Hsieh | @viirya
Spark Technology Center
--
View this message in context:
I was testing out a new project at scale on Spark 2.0.2 running on YARN,
and my job failed with an interesting error message:
TaskSetManager: Lost task 37.3 in stage 31.0 (TID 10684,
server.host.name): java.lang.IllegalStateException: There is no space
for new record
05:27:09.573
