graphx - trianglecount of 2B edges

seem to be reasonable. Can anyone point out any optimization or other tweaks i need to perform to get this to work ? Thanks! Vinod -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/graphx-trianglecount-of-2B-edges-tp25371.html Sent from the Apache Spark User List mailing

graphx - trianglecount of 2B edges

I was attempting to use the graphx triangle count method on a 2B edge graph (Friendster dataset on SNAP) . I have access to a 60 node cluster with 90GB memory and 30v cores per node . I am running into memory issues I am using 1000 partitions using the RandomVertexCut. Here’s my submit