[
https://issues.apache.org/jira/browse/GIRAPH-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005443#comment-14005443
]
Roman Shaposhnik commented on GIRAPH-873:
-----------------------------------------
[~pavanka] and [[email protected]] do you guys see this happening
in the week or so? If not, I'd rather push this out of 1.1.0 release.
> Specialized edge stores
> -----------------------
>
> Key: GIRAPH-873
> URL: https://issues.apache.org/jira/browse/GIRAPH-873
> Project: Giraph
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Craig Muchinsky
> Fix For: 1.1.0
>
> Attachments: GIRAPH-873-2.patch, GIRAPH-873.patch
>
>
> While doing some performance tuning I discovered that loading the edge store
> can be a very expensive operation. Similar to GIRAPH-704, the use of
> primitive maps can provide significant performance benefit. Part of the
> benefit comes with the lower memory overhead associated with the primitive
> maps however the larger benefit comes with the fact that you don't have to
> release and reconstruct the vertexId object every time a new vertex is
> encountered.
> When processing a large graph with 4B vertices and 5B edges (3B of the edges
> loaded via EdgeInputFormat) the worker edge requests were taking ~15 seconds
> each, but after implementing the above suggestions that number dropped down
> sub-second.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
