[
https://issues.apache.org/jira/browse/FLINK-9635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16668539#comment-16668539
]
ASF GitHub Bot commented on FLINK-9635:
---------------------------------------
tillrohrmann commented on a change in pull request #6961: [FLINK-9635] Fix
scheduling for local recovery
URL: https://github.com/apache/flink/pull/6961#discussion_r229255838
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/slotpool/SlotPool.java
##########
@@ -742,40 +761,52 @@ private void stashRequestWaitingForResourceManager(final
PendingRequest pendingR
//
------------------------------------------------------------------------
@Override
- public CompletableFuture<Acknowledge> releaseSlot(SlotRequestId
slotRequestId, @Nullable SlotSharingGroupId slotSharingGroupId, Throwable
cause) {
+ public CompletableFuture<Acknowledge> releaseSlot(
+ SlotRequestId slotRequestId,
+ @Nullable SlotSharingGroupId slotSharingGroupId,
+ Throwable cause) {
+
log.debug("Releasing slot [{}] because: {}", slotRequestId,
cause != null ? cause.getMessage() : "null");
+ return (slotSharingGroupId != null) ?
+ releaseSharedSlot(slotRequestId, slotSharingGroupId,
cause) :
+ releaseSingleSlot(slotRequestId, cause);
+ }
- if (slotSharingGroupId != null) {
- final SlotSharingManager multiTaskSlotManager =
slotSharingManagers.get(slotSharingGroupId);
+ private CompletableFuture<Acknowledge> releaseSharedSlot(
+ SlotRequestId slotRequestId,
+ @Nonnull SlotSharingGroupId slotSharingGroupId, Throwable
cause) {
- if (multiTaskSlotManager != null) {
- final SlotSharingManager.TaskSlot taskSlot =
multiTaskSlotManager.getTaskSlot(slotRequestId);
+ final SlotSharingManager multiTaskSlotManager =
slotSharingManagers.get(slotSharingGroupId);
- if (taskSlot != null) {
- taskSlot.release(cause);
- } else {
- log.debug("Could not find slot [{}] in
slot sharing group {}. Ignoring release slot request.", slotRequestId,
slotSharingGroupId);
- }
+ if (multiTaskSlotManager != null) {
+ final SlotSharingManager.TaskSlot taskSlot =
multiTaskSlotManager.getTaskSlot(slotRequestId);
+
+ if (taskSlot != null) {
+ taskSlot.release(cause);
} else {
- log.debug("Could not find slot sharing group
{}. Ignoring release slot request.", slotSharingGroupId);
+ log.debug("Could not find slot [{}] in slot
sharing group {}. Ignoring release slot request.", slotRequestId,
slotSharingGroupId);
}
} else {
- final PendingRequest pendingRequest =
removePendingRequest(slotRequestId);
+ log.debug("Could not find slot sharing group {}.
Ignoring release slot request.", slotSharingGroupId);
+ }
+ return CompletableFuture.completedFuture(Acknowledge.get());
Review comment:
This line could be moved to `releaseSlot` since this method as well as
`releaseSingleSlot` simply return a completed future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Local recovery scheduling can cause spread out of tasks
> -------------------------------------------------------
>
> Key: FLINK-9635
> URL: https://issues.apache.org/jira/browse/FLINK-9635
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
> Affects Versions: 1.5.0, 1.6.2
> Reporter: Till Rohrmann
> Assignee: Stefan Richter
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.7.0
>
>
> In order to make local recovery work, Flink's scheduling was changed such
> that it tries to be rescheduled to its previous location. In order to not
> occupy slots which have state of other tasks cached, the strategy will
> request a new slot if the old slot identified by the previous allocation id
> is no longer present. This also applies to newly allocated slots because
> there is no distinction between new or already used. This behaviour can cause
> that every tasks gets deployed to its own slot if the {{SlotPool}} has
> released all slots in the meantime, for example. The consequence could be
> that a job can no longer be executed after a failure because it needs more
> slots than before.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
