Eli, that's an interesting idea, we could have some class which user extends and which is there only for aggregator registration. Sometimes we want to register some aggregators later on during the computation, so we need to keep allowing registration from masterCompute too.
But I think for users the biggest problem is to realize that they have to extend and set MasterCompute/this new class in order to use aggregators. Currently, if user tries to aggregate a value to unregistered aggregator he will get an exception, but if he tries to get the value of unregistered aggregator he will just get null. So maybe adding a warning message in that case, together with a wiki page, might be enough? What do you think? From: Eli Reisman <apache.mail...@gmail.com<mailto:apache.mail...@gmail.com>> Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>> Date: Thursday, February 21, 2013 10:25 AM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex Thanks for the explanation, that makes sense. I would love to see a wiki page at some point, you have so much knowledge of this piece of Giraph from all your dev work on it and have also the additional bonus of experience running big cluster jobs using these features so you have a lot of insight to share. Would there be any point to a future JIRA to break out the aggregator registration from the master compute stuff, at least from the user's view? Or is it not that confusing once you've used them a bit? On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabi...@fb.com<mailto:majakabi...@fb.com>> wrote: Progressable exception can be caused by many different reasons (it's totally unrelated to aggregators), and when looking at which exception it's caused by users should get better sense about what's going on. What you are suggesting about providing default master compute is not doable, since the part which needs to be done there is aggregator registration. We can't know what kind of aggregators (names and types) an application needs. I remember I was talking about writing a short tutorial for aggregators long time ago, sorry for not doing that, will try to get to it soon. From: Eli Reisman <apache.mail...@gmail.com<mailto:apache.mail...@gmail.com>> Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>> Date: Thursday, February 14, 2013 2:23 PM To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex Other folks on the list are also having this problem with the progressable utile exception & job failures. I don't know much about master compute usage but if it is needed to make the aggregators work, maybe we should have a default dummy class that just handles aggregators if no other master compute is specified? Or a wiki page? The progressable error message does not lead us to this conclusion directly. On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <mars...@gmail.com<mailto:mars...@gmail.com>> wrote: Hey, I am trying to run the example SimpleMasterComputeVertex, but no matter which Input Format and graph I give, it doesn't work. Each worker gives the error: Caused by: java.lang.NullPointerException at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42) This line 42 is the first line of the compute() public void compute(Iterable<DoubleWritable> messages){ So I guess, the initialization is not done correctly, because the input file does not have the correct format. Any help would be appreciated, Thanks! Maria -- Maria Stylianou Intern at Telefonica, Barcelona, Spain Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain KTH Royal Institute of Technology, Stockholm, Sweden
