[
https://issues.apache.org/jira/browse/GIRAPH-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991224#comment-13991224
]
Pavan Kumar commented on GIRAPH-895:
------------------------------------
ArrayListEdges: trimToSize() does this: elementData =
Arrays.copyOf(elementData, size);
so during loading phase you end up using more resident memory & gc pressure
increases
did u test this change for any memory bound application
ByteArrayEdges: uses ExtendedDataOutput which can have upto 2 times more memory
than what it stores
so you might want to add trim here as well with considerations written for
ArrayListEdges
EdgeStore / Vertex
vertex.trimEdges => all vertex classes should now support trimEdges, this is a
breaking change
Trimmable.java - is generic so better to move it out of edges
for all changes in graph/*
similar concern -> vertex.trimEdges
the changes made in io/formats are good. you can trim edges where even &
whenever they can be trimmed
without imposing that vertex be trimmable
> Trim the edges in Giraph
> ------------------------
>
> Key: GIRAPH-895
> URL: https://issues.apache.org/jira/browse/GIRAPH-895
> Project: Giraph
> Issue Type: Improvement
> Components: graph
> Affects Versions: 1.1.0
> Reporter: Sergey Edunov
> Fix For: 1.1.0
>
> Attachments: GIRAPH-895.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In many Giraph applications, graphs are immutable, but edges are never
> trimmed to the proper size, after input phase. This means that on average we
> often use 1.5x memory for storing them. Considering we are often memory
> bounded, adding an option to trim the edges after the input phase will help
> reduce this excess memory usage. For mutable graphs, we can also provide an
> option for the same method to be called after each superstep.
> Review request: https://reviews.apache.org/r/21119/
--
This message was sent by Atlassian JIRA
(v6.2#6252)
