Sanil15 commented on a change in pull request #1281: SAMZA-2378: Container
Placements support for Standby containers enabled jobs
URL: https://github.com/apache/samza/pull/1281#discussion_r382760254
##########
File path:
samza-core/src/main/java/org/apache/samza/clustermanager/ContainerManager.java
##########
@@ -136,7 +135,22 @@ boolean handleContainerLaunch(SamzaResourceRequest
request, String preferredHost
LOG.info("Waiting for running container to shutdown due to existing
ContainerPlacement action {}", actionMetaData);
return false;
} else if (actionStatus ==
ContainerPlacementMetadata.ContainerStatus.STOPPED) {
- allocator.runStreamProcessor(request, preferredHost);
+ // If the job has standby containers enabled, always check standby
constraints before issuing a start on container
+ // Note: Always check constraints against allocated resource, since
preferred host can be ANY_HOST as well
+ if (standbyContainerManager.isPresent() &&
!standbyContainerManager.get().checkStandbyConstraints(request.getProcessorId(),
allocatedResource.getHost())) {
+ LOG.info(
+ "Starting container {} on host {} does not meet standby
constraints, falling back to source host placement metadata: {}",
+ request.getProcessorId(), preferredHost, actionMetaData);
+ resourceRequestState.releaseUnstartableContainer(allocatedResource,
preferredHost);
+ resourceRequestState.cancelResourceRequest(request);
+ // Fallback to source host since the new allocated resource does not
meet standby constraints
+ allocator.requestResource(processorId,
actionMetaData.getSourceHost());
+ markContainerPlacementActionFailed(actionMetaData,
+ String.format("allocated resource %s does not meet standby
constraints now, falling back to source host", allocatedResource));
Review comment:
> Update failedStandbyAllocations metric?
No, because here we did not initiate a standby failover, user-initiated two
individual requests: move of standby to x-host and move of active to the
standby host, these are two independent requests hence we do not need a metric
update
> Also it may be possible to expose a method on standby-container-manager to
do the
Sure, will do
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
