[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16326315#comment-16326315 ] ASF GitHub Bot commented on FLINK-8399: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5271 > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu >Priority: Major > Labels: flip-6 > Fix For: 1.5.0 > > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16325919#comment-16325919 ] ASF GitHub Bot commented on FLINK-8399: --- Github user shuai-xu commented on the issue: https://github.com/apache/flink/pull/5271 @tillrohrmann thank you for reviewing, I have modified it. > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu >Priority: Major > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324117#comment-16324117 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161249788 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") + .defaultValue(3); + + /** +* The timeout for a slot request to be discarded, in milliseconds. +*/ + public static final ConfigOption SLOT_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.slot.request-timeout") --- End diff -- maybe `slotmanager.request-timeout` > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324113#comment-16324113 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161248207 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions --- End diff -- Let's make this option a `Long`. > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324118#comment-16324118 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161249685 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") + .defaultValue(3); + + /** +* The timeout for a slot request to be discarded, in milliseconds. +*/ + public static final ConfigOption SLOT_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.slot.request-timeout") + .defaultValue(60); + + /** +* The timeout for an idle task manager to be released, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.timeout") --- End diff -- maybe `slotmanager.taskmanager-timeout` > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324119#comment-16324119 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161249883 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") --- End diff -- maybe `slotmanager.rpc-timeout` > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324115#comment-16324115 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161248438 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") + .defaultValue(3); + + /** +* The timeout for a slot request to be discarded, in milliseconds. +*/ + public static final ConfigOption SLOT_REQUEST_TIMEOUT = ConfigOptions --- End diff -- `Long` > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324114#comment-16324114 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161248457 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") + .defaultValue(3); + + /** +* The timeout for a slot request to be discarded, in milliseconds. +*/ + public static final ConfigOption SLOT_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.slot.request-timeout") + .defaultValue(60); + + /** +* The timeout for an idle task manager to be released, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_TIMEOUT = ConfigOptions --- End diff -- `Long` > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324116#comment-16324116 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161249010 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") + .defaultValue(3); --- End diff -- Please add a description via `withDescription` > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324120#comment-16324120 ] ASF GitHub Bot commented on FLINK-8399: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5271#discussion_r161249992 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/ResourceManagerOptions.java --- @@ -58,6 +58,27 @@ .defaultValue(600) .withDeprecatedKeys("yarn.heap-cutoff-min"); + /** +* The timeout for requesting slot to a task manager, in milliseconds. +*/ + public static final ConfigOption TASK_MANAGER_REQUEST_TIMEOUT = ConfigOptions + .key("slotmanager.taskmanager.request-timeout") + .defaultValue(3); --- End diff -- For the other options, a description would be great as well. > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-8399) Use independent configurations for the different timeouts in slot manager
[ https://issues.apache.org/jira/browse/FLINK-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319856#comment-16319856 ] ASF GitHub Bot commented on FLINK-8399: --- GitHub user shuai-xu opened a pull request: https://github.com/apache/flink/pull/5271 [FLINK-8399] [runtime] use independent configurations for the different timeouts in slot manager ## What is the purpose of the change *This pull request separate the timeouts for slot request to task manager, slot request to be discarded and task manager to be released in slot manager to three different configurations.* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) You can merge this pull request into a Git repository by running: $ git pull https://github.com/shuai-xu/flink jira-8399 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5271.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5271 commit f7024439ead5e3848c705659bfe221b8ce50f154 Author: shuai.xus Date: 2018-01-10T07:43:20Z [FLINK-8399] [runtime] use independent configurations for the different timeouts in slot manager > Use independent configurations for the different timeouts in slot manager > - > > Key: FLINK-8399 > URL: https://issues.apache.org/jira/browse/FLINK-8399 > Project: Flink > Issue Type: Bug > Components: Cluster Management >Affects Versions: 1.5.0 >Reporter: shuai.xu >Assignee: shuai.xu > Labels: flip-6 > > There are three parameter in slot manager to indicate the timeout for slot > request to task manager, slot request to be discarded and task manager to be > released. But now they all come from the value of AkkaOptions.ASK_TIMEOUT, > need to use independent configurations for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)