Re: Running Giraph on YARN

2013-08-19 Thread Milinda Pathirage
Hi Devs, I was able to get Giraph running on Yarn by creating Yarn specific configuration programmatically. I think that it's better if we can have some shell scripts specific to Yarn (or may be modifications to existing shell scripts), so that we can easily deploy Giraph jobs on Yarn clusters.

RE: Workers input splits and MasterCompute communication

2013-08-19 Thread Marco Aurelio Barbosa Fagnani Lotz
Hello all :) I am having problems calling getContext().getInputSplit(); inside the compute() method in the workers. It always returns as if it didn't get any split at all, since inputSplit.getLocations() returns without the hosts that should have that split as local and inputSplit.getLength()

RE: MultiVertexInputFormat

2013-08-19 Thread Yasser Altowim
Hi Guys, Any help on this will be appreciated. I am repeating my question and my code below: I am implementing an algorithm in Giraph that reads the vertex values from two input files, each has its own format. I am not using any EdgeInputFormatClass. I am now using

New vertex allocation and messages

2013-08-19 Thread Marco Aurelio Barbosa Fagnani Lotz
Hello all :) I am programming an application that has to create and destroy a few vertices. I was wondering if there is any protection in Giraph to prevent a vertex to send a message to another vertex that does not exist (i.e. provide a vertex id that is not associated with a vertex yet). Is

Re: Workers input splits and MasterCompute communication

2013-08-19 Thread Avery Ching
That makes sense, since the Context doesn't have a real InputSplit (it's a Giraph one - see BspInputSplit). What information are you trying to get out of the input splits? Giraph workers can process an arbitrary number of input splits (0 or more), so I don't think this will be useful. You

Re: New vertex allocation and messages

2013-08-19 Thread Avery Ching
Yes, you can control this behavior with the VertexResolver. It handles all mutations to the graph and resolves them in a user defined way. Avery On 8/19/13 9:21 AM, Marco Aurelio Barbosa Fagnani Lotz wrote: Hello all :) I am programming an application that has to create and destroy a few