Its fantastic thanks again! We need more wiki pages like that!!! This will save so many emails on the list too! I need to (and will) update the graph representations page to say we support multigraph, edge and vertex IO, etc. soon. A lot of our wiki info is old enough to be confusing people now.
On Sat, Mar 2, 2013 at 1:57 PM, Eli Reisman <apache.mail...@gmail.com>wrote: > Thanks for doing this Maja! > > > On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <majakabi...@fb.com> wrote: > >> I wrote up a basic info about aggregators: >> https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please >> take a look, and let me know if something needs to be changed / improved. >> >> From: Eli Reisman <apache.mail...@gmail.com> >> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >> Date: Thursday, February 21, 2013 4:21 PM >> >> To: "user@giraph.apache.org" <user@giraph.apache.org> >> Subject: Re: InputFormat for the example SimpleMasterComputeVertex >> >> That sounds great to me, maybe just a mention in the wiki that the two >> functionalities are tied together will help the idea click for people. >> Either way this will be a big help I think. >> >> >> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <majakabi...@fb.com> wrote: >> >>> Eli, that's an interesting idea, we could have some class which user >>> extends and which is there only for aggregator registration. Sometimes we >>> want to register some aggregators later on during the computation, so we >>> need to keep allowing registration from masterCompute too. >>> >>> But I think for users the biggest problem is to realize that they have >>> to extend and set MasterCompute/this new class in order to use aggregators. >>> Currently, if user tries to aggregate a value to unregistered aggregator he >>> will get an exception, but if he tries to get the value of unregistered >>> aggregator he will just get null. So maybe adding a warning message in that >>> case, together with a wiki page, might be enough? What do you think? >>> >>> From: Eli Reisman <apache.mail...@gmail.com> >>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >>> Date: Thursday, February 21, 2013 10:25 AM >>> >>> To: "user@giraph.apache.org" <user@giraph.apache.org> >>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex >>> >>> Thanks for the explanation, that makes sense. I would love to see a >>> wiki page at some point, you have so much knowledge of this piece of Giraph >>> from all your dev work on it and have also the additional bonus of >>> experience running big cluster jobs using these features so you have a lot >>> of insight to share. >>> >>> Would there be any point to a future JIRA to break out the aggregator >>> registration from the master compute stuff, at least from the user's view? >>> Or is it not that confusing once you've used them a bit? >>> >>> >>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabi...@fb.com>wrote: >>> >>>> Progressable exception can be caused by many different reasons (it's >>>> totally unrelated to aggregators), and when looking at which exception it's >>>> caused by users should get better sense about what's going on. >>>> What you are suggesting about providing default master compute is not >>>> doable, since the part which needs to be done there is aggregator >>>> registration. We can't know what kind of aggregators (names and types) an >>>> application needs. >>>> I remember I was talking about writing a short tutorial for aggregators >>>> long time ago, sorry for not doing that, will try to get to it soon. >>>> >>>> From: Eli Reisman <apache.mail...@gmail.com> >>>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org> >>>> Date: Thursday, February 14, 2013 2:23 PM >>>> >>>> To: "user@giraph.apache.org" <user@giraph.apache.org> >>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex >>>> >>>> Other folks on the list are also having this problem with the >>>> progressable utile exception & job failures. I don't know much about master >>>> compute usage but if it is needed to make the aggregators work, maybe we >>>> should have a default dummy class that just handles aggregators if no other >>>> master compute is specified? Or a wiki page? The progressable error message >>>> does not lead us to this conclusion directly. >>>> >>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <mars...@gmail.com>wrote: >>>> >>>>> Hey, >>>>> >>>>> I am trying to run the example SimpleMasterComputeVertex, but no >>>>> matter which Input Format and graph I give, it doesn't work. Each worker >>>>> gives the error: >>>>> >>>>> Caused by: java.lang.*NullPointerException* >>>>> at >>>>> org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*) >>>>> >>>>> This line 42 is the first line of the compute() >>>>> public void compute(Iterable<DoubleWritable> messages){ >>>>> >>>>> So I guess, the initialization is not done correctly, because the >>>>> input file does not have the correct format. >>>>> >>>>> Any help would be appreciated, >>>>> Thanks! >>>>> Maria >>>>> -- >>>>> Maria Stylianou >>>>> Intern at Telefonica, Barcelona, Spain >>>>> Master Student of European Master in Distributed >>>>> Computing<http://www.kth.se/en/studies/programmes/master/em/emdc> >>>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain >>>>> KTH Royal Institute of Technology, Stockholm, Sweden >>>>> >>>>> >>>> >>> >> >
