[GitHub] [flink] rmetzger commented on a change in pull request #14963: [FLINK-21362][coordination] Move State.onEnter() logic into constructor
rmetzger commented on a change in pull request #14963: URL: https://github.com/apache/flink/pull/14963#discussion_r579285044 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/declarative/DeclarativeScheduler.java ## @@ -907,20 +909,37 @@ public void runIfState(State expectedState, Runnable action, Duration delay) { // +/** Note: Do not call this method from a State constructor. */ @VisibleForTesting -void transitionToState(State newState) { -if (state != newState) { -LOG.debug( -"Transition from state {} to {}.", -state.getClass().getSimpleName(), -newState.getClass().getSimpleName()); - -State oldState = state; -oldState.onLeave(newState.getClass()); - -state = newState; -newState.onEnter(); -} + void transitionToState(StateFactory targetState) { +Preconditions.checkState( +state != null, "State transitions are now allowed while construcing a state."); +Preconditions.checkState( +state.getClass() != targetState.getStateClass(), +"Attempted to transition into the very state the scheduler is already in."); + +LOG.debug( +"Transition from state {} to {}.", +state.getClass().getSimpleName(), +targetState.getStateClass().getSimpleName()); + +State oldState = state; +oldState.onLeave(targetState.getStateClass()); + +// Guard against state transitions while constructing state objects. +// +// Consider the following scenario: +// Scheduler is in state Restarting, once the cancellation is complete, we enter the +// transitionToState(WaitingForResources) method. +// In the constructor of WaitingForResources, we call `notifyNewResourcesAvailable()`, which +// finds resources and enters transitionsToState(Executing). We are in state Executing. Then +// we return from the methods and go back in our call stack to the +// transitionToState(WaitingForResources) call, where we overwrite Executing with +// WaitingForResources. And there we have it, a deployed execution graph, and a scheduler +// that is in WaitingForResources. +state = null; Review comment: Wouldn't we achieve this "fail fast" behavior if we'd move the Precondition up in Chesnay's proposal? ``` Preconditions.checkState(state == null); // this will fail if a new state transition is triggered oldState = state state = null; oldState.onLeave() newState = targetState.getState() newState.onEnter() state = newState; ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on a change in pull request #14963: [FLINK-21362][coordination] Move State.onEnter() logic into constructor
rmetzger commented on a change in pull request #14963: URL: https://github.com/apache/flink/pull/14963#discussion_r578983459 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/declarative/DeclarativeScheduler.java ## @@ -907,20 +909,37 @@ public void runIfState(State expectedState, Runnable action, Duration delay) { // +/** Note: Do not call this method from a State constructor. */ @VisibleForTesting -void transitionToState(State newState) { -if (state != newState) { -LOG.debug( -"Transition from state {} to {}.", -state.getClass().getSimpleName(), -newState.getClass().getSimpleName()); - -State oldState = state; -oldState.onLeave(newState.getClass()); - -state = newState; -newState.onEnter(); -} + void transitionToState(StateFactory targetState) { +Preconditions.checkState( +state != null, "State transitions are now allowed while construcing a state."); +Preconditions.checkState( +state.getClass() != targetState.getStateClass(), +"Attempted to transition into the very state the scheduler is already in."); + +LOG.debug( +"Transition from state {} to {}.", +state.getClass().getSimpleName(), +targetState.getStateClass().getSimpleName()); + +State oldState = state; +oldState.onLeave(targetState.getStateClass()); + +// Guard against state transitions while constructing state objects. +// +// Consider the following scenario: +// Scheduler is in state Restarting, once the cancellation is complete, we enter the +// transitionToState(WaitingForResources) method. +// In the constructor of WaitingForResources, we call `notifyNewResourcesAvailable()`, which +// finds resources and enters transitionsToState(Executing). We are in state Executing. Then +// we return from the methods and go back in our call stack to the +// transitionToState(WaitingForResources) call, where we overwrite Executing with +// WaitingForResources. And there we have it, a deployed execution graph, and a scheduler +// that is in WaitingForResources. +state = null; Review comment: It would probably work with the tests, since we collect and manually trigger all scheduled executions in the close() method, before checking if the expected state transitions happened. But I'm not sure if it's worth introducing such behavior for the sake of removing the onEnter() method. One of the reasons why we agreed to remove onEnter() was because the change revealed a few test inaccuracies. Basically all of these inaccuracies have been found and fixed during the review of the respective tests. The onEnter() prototype was just done on an unreviewed version of the tests. Right now, WaitingForResources is the only state that triggers state transitions in the constructor. I would actually propose to revisit the decision to remove the onEnter() method, given the compromises we have to do to make it work (CC @tillrohrmann ) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on a change in pull request #14963: [FLINK-21362][coordination] Move State.onEnter() logic into constructor
rmetzger commented on a change in pull request #14963: URL: https://github.com/apache/flink/pull/14963#discussion_r578980290 ## File path: flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/declarative/RestartingTest.java ## @@ -209,6 +210,7 @@ public void close() throws Exception { NoOpExecutionDeploymentListener.get(), (execution, newState) -> {}, 0L); +setJsonPlan(""); Review comment: This will be removed again once the ExecutionGraph interface PR has been merged. If I don't set the JSON plan, creating an archived execution graph will fail. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org