[
https://issues.apache.org/jira/browse/GOBBLIN-2052?focusedWorklogId=915978&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-915978
]
ASF GitHub Bot logged work on GOBBLIN-2052:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Apr/24 07:23
Start Date: 23/Apr/24 07:23
Worklog Time Spent: 10m
Work Description: aditya1105 commented on code in PR #3932:
URL: https://github.com/apache/gobblin/pull/3932#discussion_r1575772969
##########
gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnService.java:
##########
@@ -335,13 +350,15 @@ public void
handleContainerReleaseRequest(ContainerReleaseRequest containerRelea
for (Container container : containerReleaseRequest.getContainers()) {
LOGGER.info(String.format("Releasing container %s running on %s",
container.getId(), container.getNodeId()));
- // Record that this container was explicitly released so that a new one
is not spawned to replace it
- // Put the container id in the releasedContainerCache before releasing
it so that handleContainerCompletion()
- // can check for the container id and skip spawning a replacement
container.
- // Note that this is the best effort since these are asynchronous
operations and a container may abort concurrently
- // with the release call. So in some cases a replacement container may
have already been spawned before
- // the container is put into the black list.
- this.releasedContainerCache.put(container.getId(), "");
+ if(!containerReleaseRequest.isSpinUpReplacementContainers()) {
+ // Record that this container was explicitly released so that a new
one is not spawned to replace it
+ // Put the container id in the releasedContainerCache before releasing
it so that handleContainerCompletion()
+ // can check for the container id and skip spawning a replacement
container.
+ // Note that this is the best effort since these are asynchronous
operations and a container may abort concurrently
+ // with the release call. So in some cases a replacement container may
have already been spawned before
+ // the container is put into the black list.
+ this.releasedContainerCache.put(container.getId(), "");
Review Comment:
Require some explanation on this part.
1) What is the utility of this cache?
2) Why we do we check "isSpinUpReplacementContainers" and then only put in
cache? while we log and release assigned containers without this check? I think
1st should be the answer to this question.
Issue Time Tracking
-------------------
Worklog Id: (was: 915978)
Time Spent: 0.5h (was: 20m)
> Release container which is running yarn task that is stuck in INIT state
> ------------------------------------------------------------------------
>
> Key: GOBBLIN-2052
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2052
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-yarn
> Reporter: pradeep pallikila
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
