Re: FYI, Large-scale graph computing at Google

2009-06-29 Thread Steve Loughran
Patterson, Josh wrote: Steve, I'm a little lost here; Is this a replacement for M/R or is it some new code that sits ontop of M/R that runs an iteration over some sort of graph's vertexes? My quick scan of Google's article didn't seem to yeild a distinction. Either way, I'd say for our data that

Re: FYI, Large-scale graph computing at Google

2009-06-29 Thread Steve Loughran
Edward J. Yoon wrote: I just made a wiki page -- http://wiki.apache.org/hadoop/Hambrug -- Let's discuss about the graph computing framework named Hambrug. ok, first Q, why the Hambrug. To me that's just Hamburg typed wrong, which is going to cause lots of confusion. What about something

Re: FYI, Large-scale graph computing at Google

2009-06-25 Thread Edward J. Yoon
What do you think about another new computation framework on HDFS? On Mon, Jun 22, 2009 at 3:50 PM, Edward J. Yoon edwardy...@apache.org wrote: http://googleresearch.blogspot.com/2009/06/large-scale-graph-computing-at-google.html -- It sounds like Pregel seems, a computing framework based on

Re: FYI, Large-scale graph computing at Google

2009-06-25 Thread Steve Loughran
Edward J. Yoon wrote: What do you think about another new computation framework on HDFS? On Mon, Jun 22, 2009 at 3:50 PM, Edward J. Yoon edwardy...@apache.org wrote: http://googleresearch.blogspot.com/2009/06/large-scale-graph-computing-at-google.html -- It sounds like Pregel seems, a

Re: FYI, Large-scale graph computing at Google

2009-06-25 Thread mike anderson
This would be really useful for my current projects. I'd be more than happy to help out if needed. On Thu, Jun 25, 2009 at 5:57 AM, Steve Loughran ste...@apache.org wrote: Edward J. Yoon wrote: What do you think about another new computation framework on HDFS? On Mon, Jun 22, 2009 at 3:50

Re: FYI, Large-scale graph computing at Google

2009-06-25 Thread Steve Loughran
mike anderson wrote: This would be really useful for my current projects. I'd be more than happy to help out if needed. well the first bit of code to play with then is this http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/extras/citerank/ the standalone.xml file is the one

Re: FYI, Large-scale graph computing at Google

2009-06-25 Thread Amandeep Khurana
I've been working on some graph stuff using MR as well. I'd be more than interested to chip in as well.. I remember exchanging a few mails with Paolo about having an RDF store over HBase and developing graph algorithms over it. Amandeep Khurana Computer Science Graduate Student University of

Re: FYI, Large-scale graph computing at Google

2009-06-25 Thread Edward J. Yoon
To be honest, I was thought the BigTable (HBase) for the map/reduce based graph/matrix operations. The main problems of performance were the sequential algorithm, the cost for MR job building in iterations. and, the locality of adjacent components. As mentioned on Pregel, If some algorithm

FYI, Large-scale graph computing at Google

2009-06-22 Thread Edward J. Yoon
http://googleresearch.blogspot.com/2009/06/large-scale-graph-computing-at-google.html -- It sounds like Pregel seems, a computing framework based on dynamic programming for the graph operations. I guess maybe they removed the file communications/intermediate files during iterations. Anyway, What