[ https://issues.apache.org/jira/browse/YARN-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281883#comment-15281883 ]
Jian He commented on YARN-4002: ------------------------------- [~rohithsharma], the patch may not work because hostsReader.refresh() tries to acquire the synchronized lock of HostsFileReader And getHosts / getExcludedHosts also tries to acquire the synchronized lock. So it is no difference than before. > make ResourceTrackerService.nodeHeartbeat more concurrent > --------------------------------------------------------- > > Key: YARN-4002 > URL: https://issues.apache.org/jira/browse/YARN-4002 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Hong Zhiguo > Assignee: Hong Zhiguo > Priority: Critical > Attachments: 0001-YARN-4002.patch, YARN-4002-lockless-read.patch, > YARN-4002-rwlock-v2.patch, YARN-4002-rwlock-v2.patch, > YARN-4002-rwlock-v3-rebase.patch, YARN-4002-rwlock-v3.patch, > YARN-4002-rwlock.patch, YARN-4002-v0.patch > > > We have multiple RPC threads to handle NodeHeartbeatRequest from NMs. By > design the method ResourceTrackerService.nodeHeartbeat should be concurrent > enough to scale for large clusters. > But we have a "BIG" lock in NodesListManager.isValidNode which I think it's > unnecessary. > First, the fields "includes" and "excludes" of HostsFileReader are only > updated on "refresh nodes". All RPC threads handling node heartbeats are > only readers. So RWLock could be used to alow concurrent access by RPC > threads. > Second, since he fields "includes" and "excludes" of HostsFileReader are > always updated by "reference assignment", which is atomic in Java, the reader > side lock could just be skipped. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org