That sounds great to me, maybe just a mention in the wiki that the two
functionalities are tied together will help the idea click for people.
Either way this will be a big help I think.
On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <majakabi...@fb.com> wrote:
> Eli, that's an interesting idea, we could have some class which user
> extends and which is there only for aggregator registration. Sometimes we
> want to register some aggregators later on during the computation, so we
> need to keep allowing registration from masterCompute too.
>
> But I think for users the biggest problem is to realize that they have
> to extend and set MasterCompute/this new class in order to use aggregators.
> Currently, if user tries to aggregate a value to unregistered aggregator he
> will get an exception, but if he tries to get the value of unregistered
> aggregator he will just get null. So maybe adding a warning message in that
> case, together with a wiki page, might be enough? What do you think?
>
> From: Eli Reisman <apache.mail...@gmail.com>
> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
> Date: Thursday, February 21, 2013 10:25 AM
>
> To: "user@giraph.apache.org" <user@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
> Thanks for the explanation, that makes sense. I would love to see a wiki
> page at some point, you have so much knowledge of this piece of Giraph from
> all your dev work on it and have also the additional bonus of experience
> running big cluster jobs using these features so you have a lot of insight
> to share.
>
> Would there be any point to a future JIRA to break out the aggregator
> registration from the master compute stuff, at least from the user's view?
> Or is it not that confusing once you've used them a bit?
>
>
> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabi...@fb.com> wrote:
>
>> Progressable exception can be caused by many different reasons (it's
>> totally unrelated to aggregators), and when looking at which exception it's
>> caused by users should get better sense about what's going on.
>> What you are suggesting about providing default master compute is not
>> doable, since the part which needs to be done there is aggregator
>> registration. We can't know what kind of aggregators (names and types) an
>> application needs.
>> I remember I was talking about writing a short tutorial for aggregators
>> long time ago, sorry for not doing that, will try to get to it soon.
>>
>> From: Eli Reisman <apache.mail...@gmail.com>
>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
>> Date: Thursday, February 14, 2013 2:23 PM
>>
>> To: "user@giraph.apache.org" <user@giraph.apache.org>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>
>> Other folks on the list are also having this problem with the
>> progressable utile exception & job failures. I don't know much about master
>> compute usage but if it is needed to make the aggregators work, maybe we
>> should have a default dummy class that just handles aggregators if no other
>> master compute is specified? Or a wiki page? The progressable error message
>> does not lead us to this conclusion directly.
>>
>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <mars...@gmail.com>wrote:
>>
>>> Hey,
>>>
>>> I am trying to run the example SimpleMasterComputeVertex, but no
>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>> gives the error:
>>>
>>> Caused by: java.lang.*NullPointerException*
>>> at
>>> org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>
>>> This line 42 is the first line of the compute()
>>> public void compute(Iterable<DoubleWritable> messages){
>>>
>>> So I guess, the initialization is not done correctly, because the
>>> input file does not have the correct format.
>>>
>>> Any help would be appreciated,
>>> Thanks!
>>> Maria
>>> --
>>> Maria Stylianou
>>> Intern at Telefonica, Barcelona, Spain
>>> Master Student of European Master in Distributed
>>> Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>> KTH Royal Institute of Technology, Stockholm, Sweden
>>>
>>>
>>
>
