pradeepppc commented on code in PR #3932:
URL: https://github.com/apache/gobblin/pull/3932#discussion_r1576570408
##########
gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnService.java:
##########
@@ -335,13 +350,15 @@ public void
handleContainerReleaseRequest(ContainerReleaseRequest containerRelea
for (Container container : containerReleaseRequest.getContainers()) {
LOGGER.info(String.format("Releasing container %s running on %s",
container.getId(), container.getNodeId()));
- // Record that this container was explicitly released so that a new one
is not spawned to replace it
- // Put the container id in the releasedContainerCache before releasing
it so that handleContainerCompletion()
- // can check for the container id and skip spawning a replacement
container.
- // Note that this is the best effort since these are asynchronous
operations and a container may abort concurrently
- // with the release call. So in some cases a replacement container may
have already been spawned before
- // the container is put into the black list.
- this.releasedContainerCache.put(container.getId(), "");
+ if(!containerReleaseRequest.isSpinUpReplacementContainers()) {
+ // Record that this container was explicitly released so that a new
one is not spawned to replace it
+ // Put the container id in the releasedContainerCache before releasing
it so that handleContainerCompletion()
+ // can check for the container id and skip spawning a replacement
container.
+ // Note that this is the best effort since these are asynchronous
operations and a container may abort concurrently
+ // with the release call. So in some cases a replacement container may
have already been spawned before
+ // the container is put into the black list.
+ this.releasedContainerCache.put(container.getId(), "");
Review Comment:
this cache is used to maintain the list containers which are released
intentionally. This will be used in onContainersCompleted function within
AMRMClientAsync handler to check whether we want to spinup another container or
not. Having the container id in this cache means new container won't be spin
up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
