Hi,
I'm running out of memory when I run a GraphX program for dataset moe than
10 GB, It was handle pretty well in case of noraml spark operation when did
StorageLevel.MEMORY_AND_DISK.
In case of GraphX I found its only allowed storing in memory, and it is
because in Graph constructor, this
Just figured it out using Graph constructor you can pass the storage level
for both Edge and Vertex :
Graph.fromEdges(edges, defaultValue =
(,),StorageLevel.MEMORY_AND_DISK,StorageLevel.MEMORY_AND_DISK )
Thanks to this post : https://issues.apache.org/jira/browse/SPARK-1991
-
--Harihar
