[jira] [Commented] (GIRAPH-266) Average aggregators don't calculate real average
[ https://issues.apache.org/jira/browse/GIRAPH-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421326#comment-13421326 ] Maja Kabiljo commented on GIRAPH-266: - The current idea of aggregators doesn't really support functions like average, it supports binary operations. What we can do: 1. Remove all average aggregators, user can still get average by using two sum aggregators 2. Create new classes, like DoubleAverageWritable, with method getAverage(). Then DoubleAverageAggregator extends AggregatorDoubleAverageWritable and user will call getAverage() on aggregated value to get average, and will have to aggregate DoubleAverageWritable(value, 1). 3. If we want to support this and potentially some other complicated aggregators, we could also add a method to combine two aggregators, and make aggregators Writable. Master would get whole aggregator objects from workers, and he would combine them to get the final result. In that case we could also have input (values which we aggregate) and output (aggregated value) of aggregator to be of different types. Although this maybe makes aggregators more complicated than anyone will need. What do you think? Average aggregators don't calculate real average Key: GIRAPH-266 URL: https://issues.apache.org/jira/browse/GIRAPH-266 Project: Giraph Issue Type: Bug Reporter: Maja Kabiljo Assignee: Maja Kabiljo Average aggregators calculate average on workers, and then workers' averages are sent to the master, where we actually calculate the average of averages. For example, if one worker aggregates value 2, the other values 4 and 6, and third values 8, 8 and 8, we'll get the result of 5 instead of 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-266) Average aggregators don't calculate real average
[ https://issues.apache.org/jira/browse/GIRAPH-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421337#comment-13421337 ] Alessandro Presta commented on GIRAPH-266: -- I'm in favor of 1 (let's get aggregators to work first of all) and then possibly 3 (or any other good abstraction) to allow aggregations that are not associative binary operations. Average aggregators don't calculate real average Key: GIRAPH-266 URL: https://issues.apache.org/jira/browse/GIRAPH-266 Project: Giraph Issue Type: Bug Reporter: Maja Kabiljo Assignee: Maja Kabiljo Average aggregators calculate average on workers, and then workers' averages are sent to the master, where we actually calculate the average of averages. For example, if one worker aggregates value 2, the other values 4 and 6, and third values 8, 8 and 8, we'll get the result of 5 instead of 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-266) Average aggregators don't calculate real average
[ https://issues.apache.org/jira/browse/GIRAPH-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421904#comment-13421904 ] Avery Ching commented on GIRAPH-266: +1, committed. Average aggregators don't calculate real average Key: GIRAPH-266 URL: https://issues.apache.org/jira/browse/GIRAPH-266 Project: Giraph Issue Type: Bug Reporter: Maja Kabiljo Assignee: Maja Kabiljo Attachments: GIRAPH-266.patch Average aggregators calculate average on workers, and then workers' averages are sent to the master, where we actually calculate the average of averages. For example, if one worker aggregates value 2, the other values 4 and 6, and third values 8, 8 and 8, we'll get the result of 5 instead of 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-266) Average aggregators don't calculate real average
[ https://issues.apache.org/jira/browse/GIRAPH-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421907#comment-13421907 ] Hudson commented on GIRAPH-266: --- Integrated in Giraph-trunk-Commit #151 (See [https://builds.apache.org/job/Giraph-trunk-Commit/151/]) GIRAPH-266: Average aggregators don't calculate real average (majakabiljo via aching). (Revision 1365372) Result = SUCCESS aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1365372 Files : * /giraph/trunk/CHANGELOG * /giraph/trunk/src/main/java/org/apache/giraph/aggregators/DoubleAverageAggregator.java * /giraph/trunk/src/main/java/org/apache/giraph/aggregators/FloatAverageAggregator.java * /giraph/trunk/src/test/java/org/apache/giraph/aggregators/TestDoubleAggregators.java * /giraph/trunk/src/test/java/org/apache/giraph/aggregators/TestFloatAggregators.java Average aggregators don't calculate real average Key: GIRAPH-266 URL: https://issues.apache.org/jira/browse/GIRAPH-266 Project: Giraph Issue Type: Bug Reporter: Maja Kabiljo Assignee: Maja Kabiljo Attachments: GIRAPH-266.patch Average aggregators calculate average on workers, and then workers' averages are sent to the master, where we actually calculate the average of averages. For example, if one worker aggregates value 2, the other values 4 and 6, and third values 8, 8 and 8, we'll get the result of 5 instead of 6. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
