Thanks for doing this Maja! On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <majakabi...@fb.com> wrote:
> I wrote up a basic info about aggregators: > https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please > take a look, and let me know if something needs to be changed / improved. > > From: Eli Reisman <apache.mail...@gmail.com> > Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> > Date: Thursday, February 21, 2013 4:21 PM > > To: "user@giraph.apache.org" <user@giraph.apache.org> > Subject: Re: InputFormat for the example SimpleMasterComputeVertex > > That sounds great to me, maybe just a mention in the wiki that the two > functionalities are tied together will help the idea click for people. > Either way this will be a big help I think. > > > On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <majakabi...@fb.com> wrote: > >> Eli, that's an interesting idea, we could have some class which user >> extends and which is there only for aggregator registration. Sometimes we >> want to register some aggregators later on during the computation, so we >> need to keep allowing registration from masterCompute too. >> >> But I think for users the biggest problem is to realize that they have >> to extend and set MasterCompute/this new class in order to use aggregators. >> Currently, if user tries to aggregate a value to unregistered aggregator he >> will get an exception, but if he tries to get the value of unregistered >> aggregator he will just get null. So maybe adding a warning message in that >> case, together with a wiki page, might be enough? What do you think? >> >> From: Eli Reisman <apache.mail...@gmail.com> >> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >> Date: Thursday, February 21, 2013 10:25 AM >> >> To: "user@giraph.apache.org" <user@giraph.apache.org> >> Subject: Re: InputFormat for the example SimpleMasterComputeVertex >> >> Thanks for the explanation, that makes sense. I would love to see a >> wiki page at some point, you have so much knowledge of this piece of Giraph >> from all your dev work on it and have also the additional bonus of >> experience running big cluster jobs using these features so you have a lot >> of insight to share. >> >> Would there be any point to a future JIRA to break out the aggregator >> registration from the master compute stuff, at least from the user's view? >> Or is it not that confusing once you've used them a bit? >> >> >> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabi...@fb.com> wrote: >> >>> Progressable exception can be caused by many different reasons (it's >>> totally unrelated to aggregators), and when looking at which exception it's >>> caused by users should get better sense about what's going on. >>> What you are suggesting about providing default master compute is not >>> doable, since the part which needs to be done there is aggregator >>> registration. We can't know what kind of aggregators (names and types) an >>> application needs. >>> I remember I was talking about writing a short tutorial for aggregators >>> long time ago, sorry for not doing that, will try to get to it soon. >>> >>> From: Eli Reisman <apache.mail...@gmail.com> >>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >>> Date: Thursday, February 14, 2013 2:23 PM >>> >>> To: "user@giraph.apache.org" <user@giraph.apache.org> >>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex >>> >>> Other folks on the list are also having this problem with the >>> progressable utile exception & job failures. I don't know much about master >>> compute usage but if it is needed to make the aggregators work, maybe we >>> should have a default dummy class that just handles aggregators if no other >>> master compute is specified? Or a wiki page? The progressable error message >>> does not lead us to this conclusion directly. >>> >>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <mars...@gmail.com>wrote: >>> >>>> Hey, >>>> >>>> I am trying to run the example SimpleMasterComputeVertex, but no >>>> matter which Input Format and graph I give, it doesn't work. Each worker >>>> gives the error: >>>> >>>> Caused by: java.lang.*NullPointerException* >>>> at >>>> org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*) >>>> >>>> This line 42 is the first line of the compute() >>>> public void compute(Iterable<DoubleWritable> messages){ >>>> >>>> So I guess, the initialization is not done correctly, because the >>>> input file does not have the correct format. >>>> >>>> Any help would be appreciated, >>>> Thanks! >>>> Maria >>>> -- >>>> Maria Stylianou >>>> Intern at Telefonica, Barcelona, Spain >>>> Master Student of European Master in Distributed >>>> Computing<http://www.kth.se/en/studies/programmes/master/em/emdc> >>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain >>>> KTH Royal Institute of Technology, Stockholm, Sweden >>>> >>>> >>> >> >
