It should be fixed by https://github.com/apache/spark/pull/20561 soon.
On Fri, Feb 9, 2018 at 6:16 PM, Wenchen Fan <cloud0...@gmail.com> wrote: > This has been reported before: http://apache-spark- > developers-list.1001551.n3.nabble.com/java-lang- > IllegalStateException-There-is-no-space-for-new-record-tc20108.html > > I think we may have a real bug here, but we need a reproduce. Can you > provide one? thanks! > > On Fri, Feb 9, 2018 at 5:59 PM, SNEHASISH DUTTA <info.snehas...@gmail.com> > wrote: > >> Hi , >> >> I am facing the following when running on EMR >> >> Caused by: java.lang.IllegalStateException: There is no space for new >> record >> at org.apache.spark.util.collection.unsafe.sort.UnsafeInMemoryS >> orter.insertRecord(UnsafeInMemorySorter.java:226) >> at org.apache.spark.sql.execution.UnsafeKVExternalSorter.<init> >> (UnsafeKVExternalSorter.java:132) >> at org.apache.spark.sql.execution.UnsafeFixedWidthAggregationMa >> p.destructAndCreateExternalSorter(UnsafeFixedWidthAggregatio >> nMap.java:250) >> >> I am using pyspark 2.2 , what spark configuration should be >> changed/modified to get this resolved >> >> >> Regards, >> Snehasish >> >> >> Regards, >> Snehasish >> >> On Fri, Feb 9, 2018 at 1:26 PM, SNEHASISH DUTTA <info.snehas...@gmail.com >> > wrote: >> >>> Hi , >>> >>> I am facing the following when running on EMR >>> >>> Caused by: java.lang.IllegalStateException: There is no space for new >>> record >>> at org.apache.spark.util.collection.unsafe.sort.UnsafeInMemoryS >>> orter.insertRecord(UnsafeInMemorySorter.java:226) >>> at org.apache.spark.sql.execution.UnsafeKVExternalSorter.<init> >>> (UnsafeKVExternalSorter.java:132) >>> at org.apache.spark.sql.execution.UnsafeFixedWidthAggregationMa >>> p.destructAndCreateExternalSorter(UnsafeFixedWidthAggregatio >>> nMap.java:250) >>> >>> I am using spark 2.2 , what spark configuration should be >>> changed/modified to get this resolved >>> >>> >>> Regards, >>> Snehasish >>> >> >> >
