Hello everyone,
I have a question about running the Graph analytics benchmark on the Xeon
server.
I typed in these commands sequentially
docker create --name data cloudsuite/twitter-dataset-graph
docker network create spark-net
docker run -dP --net spark-net --hostname spark-master --name spark-master
cloudsuite/spark master
docker run -dP --net spark-net --volumes-from data --name spark-worker-01
cloudsuite/spark worker spark://spark-master:7077
docker run --rm --net spark-net --volumes-from data cloudsuite/graph-analytics
--driver-memory 1g --executor-memory 4g --master spark://spark-master:7077 2>&1
| tee graph_analytics_worker_mem_1-to-1.log
and it shows some errors which is indicated as OutOfMemoryError: Java heap
space.
17/08/02 00:19:18 INFO TaskSetManager: Starting task 29.1 in stage 0.0 (TID 40,
172.18.0.3, executor 0, partition 29, PROCESS_LOCAL, 5956 bytes)
17/08/02 00:19:34 ERROR TaskSchedulerImpl: Lost executor 0 on 172.18.0.3:
Remote RPC client disassociated. Likely due to containers exceeding thresholds,
or network issues. Check driver logs for WARN messages.
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.spark.graphx.impl.EdgePartitionBuilder$mcI$sp.toEdgePartition$mcI$sp(EdgePartitionBuilder.scala:42)
at
org.apache.spark.graphx.GraphLoader$$anonfun$1.apply(GraphLoader.scala:92)
at
org.apache.spark.graphx.GraphLoader$$anonfun$1.apply(GraphLoader.scala:75)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)
at
org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
at
org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
at
org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/08/02 00:31:34 WARN JobProgressListener: Task start for unknown stage 2
17/08/02 00:31:34 INFO SparkContext: Invoking stop() from shutdown hook
17/08/02 00:31:34 INFO SparkUI: Stopped Spark web UI at http://172.18.0.4:4040
17/08/02 00:31:34 INFO StandaloneSchedulerBackend: Shutting down all executors
17/08/02 00:31:34 INFO CoarseGrainedSchedulerBackend$DriverEndpoint: Asking
each executor to shut down
17/08/02 00:31:34 INFO MapOutputTrackerMasterEndpoint:
MapOutputTrackerMasterEndpoint stopped!
17/08/02 00:31:34 INFO MemoryStore: MemoryStore cleared
17/08/02 00:31:34 INFO BlockManager: BlockManager stopped
17/08/02 00:31:34 INFO BlockManagerMaster: BlockManagerMaster stopped
17/08/02 00:31:34 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:
OutputCommitCoordinator stopped!
17/08/02 00:31:34 INFO SparkContext: Successfully stopped SparkContext
17/08/02 00:31:34 INFO ShutdownHookManager: Shutdown hook called
17/08/02 00:31:34 INFO ShutdownHookManager: Deleting directory
/tmp/spark-a1248aba-7976-40a6-9ef1-b5dd284e9472
My servers hardware specs are
CPU: Intel(R) Xeon(R) CPU E5-2630
MemTotal: 65810820 kB
Kernel: 4.9.20
OS: Ubuntu 14.04.5 LTS
So, if anybody knows the solution for that please let me know
Thanks in advance.
Best Regards
Ziho Shin (Joe) | Research Student
Cloud Computing Research Group | Infra/Core Software Technology Research
Division
Electronics and Telecommunications Research Institute(ETRI)
[email protected]<mailto:[email protected]> | +82 42 860 3919 | +82 10 9241 2276
218 Gajeongno, Yuseong-gu, Daejeon, 34129, S.Korea
CONFIDENTIALITY NOTICE: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.
