Marko A. Rodriguez created TINKERPOP-1118:
---------------------------------------------
Summary: SparkGraphComputer should use StarGraph, not
VertexWritable.
Key: TINKERPOP-1118
URL: https://issues.apache.org/jira/browse/TINKERPOP-1118
Project: TinkerPop
Issue Type: Improvement
Components: hadoop
Affects Versions: 3.1.1-incubating
Reporter: Marko A. Rodriguez
Fix For: 3.2.0-incubating
{{SparkGraphComputer}} input RDDs are typed as:
{code}
JavaPairRDD<Object,VertexWritable>
{code}
The {{VertexWritable}} usage is a vestige from Hadoop and Giraph. In Spark, we
don't need to have this wrapper and thus, we can reduce the overhead (one less
object header) by making the input RDDs typed as:
{code}
JavaPairRDD<Object,StarGraph>
{code}
This would be a breaking change for graph providers that implement their own
{{InputRDD}} and {{OutputRDD}}, however, the fix is trivial. Instead of {{new
VertexWritable(vertex)}}, they would simply do {{StarGraph.of(vertex)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
