GitHub user ifndef-SleePy opened a pull request: https://github.com/apache/flink/pull/5048
[Flink-7871] [flip6] SlotPool should release unused slots to RM ## What is the purpose of the change * This pull request makes SlotPool release unused slots back to RM if the slots is idle for some time ## Brief change log * Add a TimerService in SlotPool * Each available slot will register a timer in TimerService, and will unregister when the slot is removed or becomes allocated * If timeout happened, the slot will be released by notifying TM * Add notifySlotUnused method in TaskExecutorGateway * Make some fixed params configurable in SlotPool ## Verifying this change This change added tests in AvailableSlotsTest ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no, but add a new `@Public(Evolving)` class SlotOptions) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) You can merge this pull request into a Git repository by running: $ git pull https://github.com/alibaba/flink FLINK-7871 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5048.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5048 ---- commit b16d206b1ff1a459de10dd5d08094172386d3707 Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-16T10:11:20Z [FLINK-7871] [flip6] SlotPool will release unused slot if it is idle for a period. commit 19e9ccfb69b6e1529ed346a00389b682eecdfcfd Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-16T11:15:16Z [FLINK-7871] [flip6] Stop timer service in SlotPool elegantly. commit 0484f7457e46180fe49ae4f3483afa92c0b20ac5 Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-17T02:58:57Z [FLINK-7871] [flip6] Fix rebasing error. commit 2a012582b1e169a90c5674ed294d95c12029ee8d Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-17T03:19:40Z [FLINK-7871] [flip6] Fix AvailableSlotsTest. commit bb8b8f889735ab7a93c837db3c33b3f0900755db Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-17T08:15:01Z [FLINK-7871] [flip6] Add super.postStop() in SlotPool.postStop() commit 29fe70cec7de136e5959c9f128d32ece97fb68fc Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-17T08:36:41Z [Flink-7871] [flip6] SlotPool should release unused slots to RM Summary: ## What is the purpose of the change * This pull request makes SlotPool release unused slots back to RM if the slots is idle for some time ## Brief change log * Add a TimerService in SlotPool * Each available slot will register a timer in TimerService, and will unregister when the slot is removed or becomes allocated * If timeout happened, the slot will be released by notifying TM * Add notifySlotUnused method in TaskExecutorGateway * Make some fixed params configurable in SlotPool ## Verifying this change This change added tests in AvailableSlotsTest ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no, but add a new '@Public(Evolving)' class SlotOptions) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) Test Plan: UT done Reviewers: æµ·æ¶, è¾ æº Differential Revision: https://aone.alibaba-inc.com/code/D350078 commit 7b144e853df1d9a807464832a3582922c91672d1 Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-17T09:19:40Z [Flink-7871] [flip6] SlotPool should release unused slots to RM Summary: ## What is the purpose of the change * This pull request makes SlotPool release unused slots back to RM if the slots is idle for some time ## Brief change log * Add a TimerService in SlotPool * Each available slot will register a timer in TimerService, and will unregister when the slot is removed or becomes allocated * If timeout happened, the slot will be released by notifying TM * Add notifySlotUnused method in TaskExecutorGateway * Make some fixed params configurable in SlotPool ## Verifying this change This change added tests in AvailableSlotsTest ## Does this pull request potentially affect one of the following parts: * Dependencies (does it add or upgrade a dependency): (no) * The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no, but add a new `@Public(Evolving)` class SlotOptions) * The serializers: (no) * The runtime per-record code paths (performance sensitive): (no) * Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) * The S3 file system connector: (no) ## Documentation * Does this pull request introduce a new feature? (no) * If yes, how is the feature documented? (not applicable) Test Plan: UT done Reviewers: haitao.w, è¾ æº Differential Revision: https://aone.alibaba-inc.com/code/D350105 commit 4f2cddf3e35b4f9203850cfee4122b12bfd1f046 Author: biao.liub <biao.l...@alibaba-inc.com> Date: 2017-11-22T10:57:41Z [FLINK-7871] Fix typos and and some tests. ---- ---