[
https://issues.apache.org/jira/browse/HAMA-704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Jungblut updated HAMA-704:
---------------------------------
Attachment: HAMA-704_1337.patch
First scratch in HAMA-704_1337.patch, everything within maven and locally works
so far (even pagerank!).
I will profile this tomorrow or a bit later today to see if it has given any
major improvement on the RAM usage.
Still I'm not completely sure about many things, do you think we should open a
review board thread for it? Please let me know about suggestions and honest
opinions you have.
Also a small drawback: we can't skip reading inactive vertices every iteration
because the values file must be rewritten using all vertex values (otherwise
there are obviously strange results). However, I added the possibility to do so
by a strategy pattern if needed or we have a better idea how to not rewrite the
whole values every iteration.
Also for aggregators to work correctly, I had to replay messages between the
supersteps (between the master and the slave synchronization). Maybe we can get
a better synchronization strategy that doesn't clear the queues so we don't
have to loop-back them again for the next step.
> Optimization of memory usage during message processing
> ------------------------------------------------------
>
> Key: HAMA-704
> URL: https://issues.apache.org/jira/browse/HAMA-704
> Project: Hama
> Issue Type: Improvement
> Components: graph
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Priority: Critical
> Fix For: 0.6.1
>
> Attachments: HAMA-704_1337.patch, HAMA-704.patch-v1,
> hama-704_v05.patch, HAMA-704-v2.patch, localdisk.patch, mytest.patch,
> patch.txt, patch.txt, removeMsgMap.patch
>
>
> <vertex, message> map seems consume a lot of memory. We should figure out an
> efficient way to reduce memory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
