With 2000 workers, thats 2000 extra connections in the system. We run Giraph/Netty on the same cluster as existing jobs that use Hadoop RPC so network resources are sometimes at a premium. These jobs are often running on the same boxes as our worker mappers are running, and the scheduling is not under our control or particularly suited to Giraph. I'm not too familiar with the aggregator code but it seems like if you have an idea for an implementation that doesn't use a barrier, I agree with Avery that this doesn't preclude the tree option in that scenario either.
On the other hand, if you have a specialized use case, maybe the easiest thing would be to do what it takes to make your map aggregator work however you like and have it be command-line optional, and just leave the existing ZK implementation in place for the rest of the use cases. Have you had problems with needing more standard aggregators and ZK nodes not holding enough data, or is this map aggregator driving your need for this feature? Can I ask what algorithm you're implementing that requires a globally aggregated map at every superstep? Have you guys noticed performance or speed issues with the existing ZK implementation as you add aggregators to an application? Anyway I'm not firmly for or against any of this stuff, just curious. If you find an implementation that works for you that sounds great. If it was optional with the existing version or the tree available, that would probably save us some headache here when we share a cluster (which is almost all the time.) On Fri, Aug 31, 2012 at 12:55 PM, Maja Kabiljo (JIRA) <j...@apache.org>wrote: > > [ > https://issues.apache.org/jira/browse/GIRAPH-273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446301#comment-13446301] > > Maja Kabiljo commented on GIRAPH-273: > ------------------------------------- > > That's true, we can implement several different approaches and decide > which one to use based on the current application needs. > > > Aggregators shouldn't use Zookeeper > > ----------------------------------- > > > > Key: GIRAPH-273 > > URL: https://issues.apache.org/jira/browse/GIRAPH-273 > > Project: Giraph > > Issue Type: Improvement > > Reporter: Maja Kabiljo > > Assignee: Maja Kabiljo > > > > We use Zookeeper znodes to transfer aggregated values from workers to > master and back. Zookeeper is supposed to be used for coordination, and it > also has a memory limit which prevents users from having aggregators with > large value objects. These are the reasons why we should implement > aggregators gathering and distribution in a different way. > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira >
