[
https://issues.apache.org/jira/browse/HAMA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Jungblut updated HAMA-642:
---------------------------------
Attachment: HAMA-642_unix_1.patch
Finally unix generated patch, tescases work fine. Extraction of constants.
Please review and cast your vote on adding a submodule for JDBM (we could
rename it if the name is too boring for you).
I would like to use this improved version of JDBM for our messaging queues
instead of sequence files. Also for a potential users algorithm, it could be
beneficial to have structures to store something on disk or in cached memory
areas with the java collections interfaces.
> Make GraphRunner disk based
> ---------------------------
>
> Key: HAMA-642
> URL: https://issues.apache.org/jira/browse/HAMA-642
> Project: Hama
> Issue Type: Improvement
> Components: graph
> Reporter: Thomas Jungblut
> Attachments: HAMA-642_unix_1.patch, HAMA-scale_1.patch,
> HAMA-scale_2.patch, HAMA-scale_3.patch, HAMA-scale_4.patch
>
>
> To improve scalability we can improve the graph runner to be disk based.
> Which basically means:
> - We have just a single Vertex instance that get's refilled.
> - We directly write vertices to disk after partitioning
> - In every superstep we iterate over the vertices on disk, fill the vertex
> instance and call the users compute functions
> Problems:
> - State other than vertex value can't be stored easy
> - How do we deal with random access after messages have arrived?
> So I think we should make the graph runner more hybrid, like using the queues
> we have implemented in the messaging. So the graphrunner can be configured to
> run completely on disk, in cached mode or in in-memory mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
