RE: Workers input splits and MasterCompute communication

2013-08-19 Thread Marco Aurelio Barbosa Fagnani Lotz
Hello all :)

I am having problems calling getContext().getInputSplit(); inside the compute() 
method in the workers.

It always returns as if it didn't get any split at all, since 
inputSplit.getLocations() returns without the hosts that should have that split 
as local and inputSplit.getLength() returns 0.

Should there be any initialization to the Workers context so that I can get 
this information?
Is there anyway to access the jobContext from the workers or the Master?

Best Regards,
Marco Lotz


From: Marco Aurelio Barbosa Fagnani Lotz m.a.b.l...@stu12.qmul.ac.uk
Sent: 17 August 2013 20:20
To: user@giraph.apache.org
Subject: Workers input splits and MasterCompute communication

Hello all :)

In what class the workers actually get the input file splits from the file 
system?

Is it possible to a MasterCompute class object to have access/communication 
with the workers in that job? I though about using aggregators, but then I 
assumed that aggregators actually work with vertices compute() (and related 
methods) and not with the worker itself.

When I mean workers I don't mean the vertices in each worker, but the object 
that runs the compute for all the vertices in that worker.

Best Regards,
Marco Lotz


Re: Workers input splits and MasterCompute communication

2013-08-19 Thread Avery Ching
That makes sense, since the Context doesn't have a real InputSplit (it's 
a Giraph one - see BspInputSplit).


What information are you trying to get out of the input splits? Giraph 
workers can process an arbitrary number of input splits (0 or more), so 
I don't think this will be useful.


You can use Configuration if you need to set some information at runtime.

Avery

On 8/19/13 9:14 AM, Marco Aurelio Barbosa Fagnani Lotz wrote:

Hello all :)

I am having problems calling getContext().getInputSplit(); inside the 
compute() method in the workers.


It always returns as if it didn't get any split at all, since 
inputSplit.getLocations() returns without the hosts that should have 
that split as local and inputSplit.getLength() returns 0.


Should there be any initialization to the Workers context so that I 
can get this information?

Is there anyway to access the jobContext from the workers or the Master?

Best Regards,
Marco Lotz


*From:* Marco Aurelio Barbosa Fagnani Lotz m.a.b.l...@stu12.qmul.ac.uk
*Sent:* 17 August 2013 20:20
*To:* user@giraph.apache.org
*Subject:* Workers input splits and MasterCompute communication
Hello all :)

In what class the workers actually get the input file splits from the 
file system?


Is it possible to a MasterCompute class object to have 
access/communication with the workers in that job? I though about 
using aggregators, but then I assumed that aggregators actually work 
with vertices compute() (and related methods) and not with the worker 
itself.


When I mean workers I don't mean the vertices in each worker, but the 
object that runs the compute for all the vertices in that worker.


Best Regards,
Marco Lotz