Hi Yifan This works for me:
export SPARK_JAVA_OPTS="-Xms10g -Xmx40g -XX:MaxPermSize=10g" export ADD_JARS=/home/abel/spark/MLI/target/MLI-assembly-1.0.jar export SPARK_MEM=40g ./spark-shell Regards On Mon, Jul 21, 2014 at 7:48 AM, Yifan LI <iamyifa...@gmail.com> wrote: > Hi, > > I am trying to load the Graphx example dataset(LiveJournal, 1.08GB) > through *Scala Shell* on my standalone multicore machine(8 cpus, 16GB > mem), but an OutOfMemory error was returned when below code was running, > > val graph = GraphLoader.edgeListFile(sc, path, minEdgePartitions = > 16).partitionBy(PartitionStrategy.RandomVertexCut) > > I guess I should set some parameters to JVM? like "-Xmx5120m" > But how to do this in Scala Shell? > I directly used the "bin/spark-shell" to start spark and seems everything > works correctly in WebUI. > > Or, I should do parameters setting at somewhere(spark-1.0.1)? > > > > Best, > Yifan LI >
