Hi Jay, The recommended way to build spark from source is through the maven system. You would want to follow the steps in https://spark.apache.org/docs/latest/building-with-maven.html to set the MAVEN_OPTS to prevent OOM build errors.
Thanks On Tue, Aug 26, 2014 at 5:49 PM, jay vyas <jayunit100.apa...@gmail.com> wrote: > Hi spark. > > I've been trying to build spark, but I've been getting lots of oome > exceptions. > > https://gist.github.com/jayunit100/d424b6b825ce8517d68c > > For the most part, they are of the form: > > java.lang.OutOfMemoryError: unable to create new native thread > > I've attempted to hard code the "get_mem_opts" function, which is in the > sbt-launch-lib.bash file, to > have various very high parameter sizes (i.e. -Xms5g") with high > MaxPermSize, etc... and to no avail. > > Any thoughts on this would be appreciated. > > I know of others having the same problem as well. > > Thanks! > > -- > jay vyas >
