[ 
https://issues.apache.org/jira/browse/YARN-11285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-11285:
---------------------------------
    Attachment: TestConcurrency.java

> LocalizedResources are leaked and its LocalPath are not cleared
> ---------------------------------------------------------------
>
>                 Key: YARN-11285
>                 URL: https://issues.apache.org/jira/browse/YARN-11285
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 3.2.1
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>            Priority: Major
>         Attachments: TestConcurrency.java
>
>
> LocalizedResources are leaked and its LocalPath are not cleared from NM Local 
> Directories. When multiple containers are initialized at same time, 
> LocalResourcesTrackerImpl REQUEST handler could create and handle multiple 
> LocalizedResource object for the same input path due to race condition in 
> [below 
> code|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourcesTrackerImpl.java#L149]
> {code}
>     case REQUEST:
>     LocalResourceRequest req = event.getLocalResourceRequest();
>     LocalizedResource rsrc = localrsrc.get(req);
>      ....
>       if (null == rsrc) {
>         rsrc = new LocalizedResource(req, dispatcher);
>         localrsrc.put(req, rsrc);
>       }
>      rsrc.handle(event);
> {code}
> Each container will have separate LocalizedResource object and separate local 
> path like below.
> {code}
>    /mnt/yarn/usercache/hive/filecache/6/2552419:
>    total 28456
>    -r-x------ 1 yarn yarn 29135164 Aug  7 10:24 
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>    /mnt/yarn/usercache/hive/filecache/6/2552420:
>    total 28456
>    -r-x------ 1 yarn yarn 29135164 Aug  7 10:24 
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>    /mnt/yarn/usercache/hive/filecache/6/2552421:
>    total 28456
>    -r-x------ 1 yarn yarn 29135164 Aug  7 10:24 
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>    /mnt/yarn/usercache/hive/filecache/6/2552422:
>    total 28456
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to