[
https://issues.apache.org/jira/browse/GOBBLIN-2052?focusedWorklogId=916077&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-916077
]
ASF GitHub Bot logged work on GOBBLIN-2052:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Apr/24 16:45
Start Date: 23/Apr/24 16:45
Worklog Time Spent: 10m
Work Description: pradeepppc commented on code in PR #3932:
URL: https://github.com/apache/gobblin/pull/3932#discussion_r1576570408
##########
gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnService.java:
##########
@@ -335,13 +350,15 @@ public void
handleContainerReleaseRequest(ContainerReleaseRequest containerRelea
for (Container container : containerReleaseRequest.getContainers()) {
LOGGER.info(String.format("Releasing container %s running on %s",
container.getId(), container.getNodeId()));
- // Record that this container was explicitly released so that a new one
is not spawned to replace it
- // Put the container id in the releasedContainerCache before releasing
it so that handleContainerCompletion()
- // can check for the container id and skip spawning a replacement
container.
- // Note that this is the best effort since these are asynchronous
operations and a container may abort concurrently
- // with the release call. So in some cases a replacement container may
have already been spawned before
- // the container is put into the black list.
- this.releasedContainerCache.put(container.getId(), "");
+ if(!containerReleaseRequest.isSpinUpReplacementContainers()) {
+ // Record that this container was explicitly released so that a new
one is not spawned to replace it
+ // Put the container id in the releasedContainerCache before releasing
it so that handleContainerCompletion()
+ // can check for the container id and skip spawning a replacement
container.
+ // Note that this is the best effort since these are asynchronous
operations and a container may abort concurrently
+ // with the release call. So in some cases a replacement container may
have already been spawned before
+ // the container is put into the black list.
+ this.releasedContainerCache.put(container.getId(), "");
Review Comment:
this cache is used to maintain the list containers which are released
intentionally. This will be used in onContainersCompleted function within
AMRMClientAsync handler to check whether we want to spinup another container or
not. Having the container id in this cache means new container won't be spin
up.
Issue Time Tracking
-------------------
Worklog Id: (was: 916077)
Time Spent: 1h (was: 50m)
> Release container which is running yarn task that is stuck in INIT state
> ------------------------------------------------------------------------
>
> Key: GOBBLIN-2052
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2052
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-yarn
> Reporter: pradeep pallikila
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
