[ https://issues.apache.org/jira/browse/GIRAPH-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-128: ------------------------------- Attachment: GIRAPH-128.2.patch Updated after GIRAPH-124 was committed. > RPC port from BasicRPCCommunications should be only a starting port, and > retried > -------------------------------------------------------------------------------- > > Key: GIRAPH-128 > URL: https://issues.apache.org/jira/browse/GIRAPH-128 > Project: Giraph > Issue Type: Improvement > Affects Versions: 0.1.0 > Reporter: Avery Ching > Assignee: Avery Ching > Attachments: GIRAPH-128.2.patch > > > Currently Giraph uses a basic port + the task partition to get the RPC port. > This doesn't work well for when there are multiple Giraph jobs running > simultaneously in the same Hadoop cluster (port conflict). At the same time, > it is nice to use this simple algorithm because it makes it very easy to > debug problems (you can find the troublesome mapper from the RPC port name). > I will be proposing a simple scheme to retry with another port. I will round > the total number of mappers up to the nearest power of 10 (let's that that > number Z). Then I will increment the port number by Z, retrying up to 20 > tries. If you have enough ports, this scheme would guarantee that up to 20 > mappers / node would be supported. It should be sufficient for most > clusters. At the same time, we still maintain the easy debugging method > since you it's still easy to figure out the mapper partition from the port > (port % Z = map partition). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira