GitHub user GJL reopened a pull request: https://github.com/apache/flink/pull/5107
[FLINK-8176][flip6] Start SubmittedJobGraphStore in Dispatcher ## What is the purpose of the change The FLIP-6 dispatcher never calls `start()` on its SubmittedJobGraphStore instance. Hence, when a Job is submitted (YARN session mode with HA enabled), an IllegalStateException is thrown. This pull request adds the necessary changes so that jobs can be submitted. ## Brief change log - *Implement SubmittedJobGraphListener interface in Dispatcher* ## Verifying this change - *Added unit tests for new methods in Dispatcher class* - *Verified that jobs can be submitted in FLIP-6 YARN session mode with HA. Did not verify anything else.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (**yes** / no / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) CC: @tillrohrmann You can merge this pull request into a Git repository by running: $ git pull https://github.com/GJL/flink FLINK-8176 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5107.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5107 ---- commit d238ef0c23eea585974929eafdff33af916d19ba Author: gyao <g...@data-artisans.com> Date: 2017-11-30T14:37:30Z [hotfix][tests] Extract SubmittedJobGraphStore implementation from JobManagerHARecoveryTest commit 33b9d2848c767088f43fed2d03e6402695827221 Author: gyao <g...@data-artisans.com> Date: 2017-11-30T14:44:23Z [FLINK-8176][flip6] Implement SubmittedJobGraphListener interface in Dispatcher Call start() on SubmittedJobGraphStore with Dispatcher as listener. To enable this, the dispatcher must implement the SubmittedJobGraphListener interface. Add simple unit tests for the new methods. Refactor DispatcherTest to remove redundancy. commit 88359172e23413aa195177993551613349056b68 Author: gyao <g...@data-artisans.com> Date: 2017-12-04T18:57:29Z [FLINK-8176][flip6] Make InMemorySubmittedJobGraphStore thread-safe commit 9cdb29604e9915c7d6ea60ed6fcee06c9bad57b9 Author: gyao <g...@data-artisans.com> Date: 2017-12-04T18:58:26Z [hotfix][Javadoc] Make first sentence in JobSubmissionException Javadoc end with period commit 53ad1771e8bec063157d69c0f7a187ccb5fb340e Author: gyao <g...@data-artisans.com> Date: 2017-12-04T19:04:52Z [FLINK-8176][flip6] Add method isStarted() to TestingLeaderElectionService commit 0c030fb19d7b5b9dba4df5811a69086906e20ca0 Author: gyao <g...@data-artisans.com> Date: 2017-12-04T19:05:47Z [FLINK-8176][flip6] Return same RunningJobsRegistry instance from TestingHighAvailabilityServices commit 7a04cbe54bcf380684c4e79a4f999b31b650570e Author: gyao <g...@data-artisans.com> Date: 2017-12-04T19:09:36Z [FLINK-8176][flip6] Fix race conditions in Dispatcher and DispatcherTest Check if jobManagerRunner exists before submitting job. Replace JobManagerRunner mock used in tests with real instance. Do not run job graph recovery in actor main thread when job graph is recovered from SubmittedJobGraphListener#onAddedJobGraph(JobID). ---- ---