Re: Running "beyond memory limits" in ConnectedComponents

2015-01-15 Thread Sean Owen
owed container size and the JVM heap size is too small. See >>>> >> spark.yarn.executor.memoryOverhead in >>>> >> http://spark.apache.org/docs/latest/running-on-yarn.html >>>> >> >>>> >> On Wed, Jan 14, 2015 at 9:18 PM, nitinka

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-15 Thread Nitin kak
>>>> >> >>>> >> On Wed, Jan 14, 2015 at 9:18 PM, nitinkak001 >>>> >> wrote: >>>> >> > I am trying to run connected components algorithm in Spark. The >>>> graph >>>> >> > has >>>&g

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-15 Thread Sean Owen
tion application_1418082773407_8587 failed 2 times due to AM >> >> > Container for appattempt_1418082773407_8587_02 exited with >> exitCode: >> >> > -104 due to: Container >> >> > [pid=19790,containerID=container_1418082773407_8587_02_01]

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-15 Thread Nitin kak
gt; Container for appattempt_1418082773407_8587_02 exited with >> exitCode: >> >> > -104 due to: Container >> >> > [pid=19790,containerID=container_1418082773407_8587_02_01] is >> &

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-15 Thread Nitin kak
> > >> > > -Djava.io.tmpdir=/mnt/DATA1/yarn/nm/usercache/kakn/appcache/application_1418082773407_8587/container_1418082773407_8587_02_01/tmp > >> > '-Dspark.executor.memory=8g' '-Dspark.eventLog.enabled=true' > >>

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-15 Thread Sean Owen
82773407_8587/container_1418082773407_8587_02_000001/stdout >> > 2> >> > >> > /var/log/hadoop-yarn/container/application_1418082773407_8587/container_1418082773407_8587_02_01/stderr >> > |- 19794 19790 19790 19790 (java) 205066 9152 9477726208 1707599 >

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-14 Thread Nitin kak
park.app.name=ConnectedComponentsTest > > > -Dspark.eventLog.dir=hdfs://:8020/user/spark/applicationHistory > > -Dspark.master=yarn-cluster > org.apache.spark.deploy.yarn.ApplicationMaster > > --class ConnectedComponentsTest --jar > > > file:/home/kakn01/Spark/SparkSo

Re: Running "beyond memory limits" in ConnectedComponents

2015-01-14 Thread Sean Owen
connectedcomponentstest_2.10-1.0.jar > --executor-memory 8192 --executor-cores 1 --num-executors 7 > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > .Failing this attempt.. Failing the application. > > > > -- > View this m

Running "beyond memory limits" in ConnectedComponents

2015-01-14 Thread nitinkak001
ecutors 7 Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 .Failing this attempt.. Failing the application. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-beyond-memor