Re: Multiple node types in Giraph and doing a selective M/R over one of them

2013-01-29 Thread David Koch
Hello Claudio and Eli, Thank you for your answers. As far as Map/Reduce being a better tool for the job - I was under the impression that Giraph relies on the M/R framework. It seems like it when I check the console output of the examples on the project's Wiki. Again, thank you. /David On Mon,

Re: Multiple node types in Giraph and doing a selective M/R over one of them

2013-01-29 Thread Eli Reisman
Right. If your use case boils down to a join, we're probably not the ideal tool. On Tue, Jan 29, 2013 at 1:26 AM, Claudio Martella claudio.marte...@gmail.com wrote: That is correct, but that is not the reason. We use M/R for resource allocation, but we do not inherit the limits of the M/R

Multiple node types in Giraph and doing a selective M/R over one of them

2013-01-28 Thread David Koch
Hello, In Giraph is it possible to have different node types in a graph and have a Map/Reduce only iterate over nodes of this type and their direct successors? If it sounds a bit cryptic here is something more about our use-case: We have different HBase tables which we want to pseudo-join in

Re: Multiple node types in Giraph and doing a selective M/R over one of them

2013-01-28 Thread Claudio Martella
Giraph does not support multipartite graph in a natural way. But you can try to model your different sets through the vertexvalue. You can then propagate it (by composing with the ID?) to the neighbors, and obtain your join. On Mon, Jan 28, 2013 at 2:52 PM, David Koch ogd...@googlemail.com

Re: Multiple node types in Giraph and doing a selective M/R over one of them

2013-01-28 Thread Eli Reisman
I agree, something like this is possible using the vertex value. In giraph, we now have native support for multigraphs, but before we had that support, I described a kind of cheat to process multigraphs. You could use a variation of that same cheat (its on the site confluence wiki) to do what

Re: Multiple node types in Giraph and doing a selective M/R over one of them

2013-01-28 Thread Claudio Martella
One more general point would be whether giraph is a better tool for your problem. From my understanding, map reduce is probably the way to go. On Monday, January 28, 2013, Eli Reisman wrote: I agree, something like this is possible using the vertex value. In giraph, we now have native support