[ https://issues.apache.org/jira/browse/YARN-11285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prabhu Joseph updated YARN-11285: --------------------------------- Attachment: TestConcurrency.java > LocalizedResources are leaked and its LocalPath are not cleared > --------------------------------------------------------------- > > Key: YARN-11285 > URL: https://issues.apache.org/jira/browse/YARN-11285 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager > Affects Versions: 3.2.1 > Reporter: Prabhu Joseph > Assignee: Prabhu Joseph > Priority: Major > Attachments: TestConcurrency.java > > > LocalizedResources are leaked and its LocalPath are not cleared from NM Local > Directories. When multiple containers are initialized at same time, > LocalResourcesTrackerImpl REQUEST handler could create and handle multiple > LocalizedResource object for the same input path due to race condition in > [below > code|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourcesTrackerImpl.java#L149] > {code} > case REQUEST: > LocalResourceRequest req = event.getLocalResourceRequest(); > LocalizedResource rsrc = localrsrc.get(req); > .... > if (null == rsrc) { > rsrc = new LocalizedResource(req, dispatcher); > localrsrc.put(req, rsrc); > } > rsrc.handle(event); > {code} > Each container will have separate LocalizedResource object and separate local > path like below. > {code} > /mnt/yarn/usercache/hive/filecache/6/2552419: > total 28456 > -r-x------ 1 yarn yarn 29135164 Aug 7 10:24 > hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar > /mnt/yarn/usercache/hive/filecache/6/2552420: > total 28456 > -r-x------ 1 yarn yarn 29135164 Aug 7 10:24 > hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar > /mnt/yarn/usercache/hive/filecache/6/2552421: > total 28456 > -r-x------ 1 yarn yarn 29135164 Aug 7 10:24 > hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar > /mnt/yarn/usercache/hive/filecache/6/2552422: > total 28456 > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org