In conclusion, I would say yes. But, I'm not sure that there's another method at the moment.
On Tue, Jun 28, 2011 at 11:01 PM, Claudio Martella <[email protected]> wrote: > Hi, > > I was considering an issue not discussed in Pregel paper. One > interesting thing about MapReduce is that the memory limit for each job > in a cluster node is the number of task assigned to that node * the > memory allocated by each task (plus of course the framework overhead). > The input data is iteratively loaded (and kind of emitted too) from > storage and doesn't need to be kept in memory or loaded together in advance. > > In Pregel, to me, it looks like the whole graph has to fit in memory > among the cluster nodes. As it is somehow stateful and doesn't have an > iterative job-job nature as MapReduce, at the beginning of the job all > the vertices and their data have to be loaded into the cluster (and kept > in memory). > > So the question is: if i want to run Pagerank over my graph, do I need > to be able to store the whole graph in memory? > > > Thanks! > > -- > Claudio Martella > Digital Technologies > Unit Research & Development - Analyst > > TIS innovation park > Via Siemens 19 | Siemensstr. 19 > 39100 Bolzano | 39100 Bozen > Tel. +39 0471 068 123 > Fax +39 0471 068 129 > [email protected] http://www.tis.bz.it > > Short information regarding use of personal data. According to Section 13 of > Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we > process your personal data in order to fulfil contractual and fiscal > obligations and also to send you information regarding our services and > events. Your personal data are processed with and without electronic means > and by respecting data subjects' rights, fundamental freedoms and dignity, > particularly with regard to confidentiality, personal identity and the right > to personal data protection. At any time and without formalities you can > write an e-mail to [email protected] in order to object the processing of > your personal data for the purpose of sending advertising materials and also > to exercise the right to access personal data and other rights referred to in > Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation > Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete > information on the web site www.tis.bz.it. > > > > > -- Best Regards, Edward J. Yoon @eddieyoon
