yes i observed that too. however, it is a parameter of a specific job (and spark version, perhaps). toy-ish problems do not cause that happen. so the idea was that MAHOUT-OPTS should be one place to adjust it (along with other gen sizes) to fit a particular purpose.
On Thu, Aug 13, 2015 at 8:42 AM, Sergey Tryuber (JIRA) <[email protected]> wrote: > Sergey Tryuber created MAHOUT-1766: > -------------------------------------- > > Summary: Increase default PermGen size for spark-shell > Key: MAHOUT-1766 > URL: https://issues.apache.org/jira/browse/MAHOUT-1766 > Project: Mahout > Issue Type: Improvement > Components: Mahout spark shell > Reporter: Sergey Tryuber > > > Mahout spark-shell is run with default perm gen size (64MB). Taking into > account that it depends on lots of external jars and the whole count of > used Java classes is very large, we constantly observe spontaneous > corresponding OOM exceptions. > > A hot fix from our side is to modify envelope bash script (added > -XX:PermSize=512m): > {code} > "$JAVA" $JAVA_HEAP_MAX -XX:PermSize=512m $MAHOUT_OPTS -classpath > "$CLASSPATH" "org.apache.mahout.sparkbindings.shell.Main" $@ > {code} > Of course, more elegant solution is needed. After the applied fix, the > errors had gone. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) >
