Re: [Spark 1.5]: Exception in thread "broadcast-hash-join-2" java.lang.OutOfMemoryError: Java heap space -- Work in 1.4, but 1.5 doesn't

On 16 December 2015 at 06:19, Deenar Toraskar < deenar.toras...@thinkreactive.co.uk> wrote: > Hi > > I had the same problem. There is a query with a lot of small tables (5x) > all below the broadcast threshold and Spark is broadcasting all these > tables together without checking if there is

Re: [Spark 1.5]: Exception in thread "broadcast-hash-join-2" java.lang.OutOfMemoryError: Java heap space -- Work in 1.4, but 1.5 doesn't

Hi I have created an issue for this https://issues.apache.org/jira/browse/SPARK-12358 Regards Deenar On 16 December 2015 at 06:21, Deenar Toraskar wrote: > > > On 16 December 2015 at 06:19, Deenar Toraskar < > deenar.toras...@thinkreactive.co.uk> wrote: > >> Hi >>

RE: [Spark 1.5]: Exception in thread "broadcast-hash-join-2" java.lang.OutOfMemoryError: Java heap space -- Work in 1.4, but 1.5 doesn't

And an update is: this ONLY happen in Spark 1.5, I try to run it under Spark 1.4 and 1.4.1, there are no issue (the program is developed under Spark 1.4 last time, and I just re-test it, it works). So this is proven that there is no issue on the logic and data, it is caused by the new version of