[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-17010:
---
Release Note:
When FairCallQueue is enabled, user can specify capacity allocation among all
sub-queues via configuration “ipc..callqueue.capacity.weights”. The value
of this config is a comma-separated list of positive integers, each of which
specifies the weight associated with the sub-queue at that index. This list
length should be IPC scheduler priority levels, defined by
"scheduler.priority.levels".
By default, each sub-queue is associated with weight 1, i.e., all sub-queues
are allocated with the same capacity.
was:
This commit adds a new configuration callqueue.capacity.weights which can be
used to specify capacity allocation among all subqueues when FairCallQueue is
enabled. The value for this configuration is expected to be a list of positive
integers separated by {{,}}, whose length should equal to
scheduler.priority.levels. Each of the element in the list specifies the weight
associated with the subqueue at that index.
In default, each subqueue is associated with weight 1, i.e., all subqueues are
allocated with the same capacity.
Labels: (was: ipc)
Thanks [~csun] and [~fengnanli] for prompt reply.
For the release notes, I edited it a bit to make it clearer. Feel free to
refine it. Specially, {{CallQueue}} related settings are not used directly, but
rather combined with a namespace and port. So how to enable this config? I
guess we should set {{ipc..callqueue.capacity.weights}} instead of
{{callqueue.capacity.weights}}. Is that true, [~fengnanli]? Also if you can add
a unit test reading from configuration, that will be good. It can be a separate
JIRA.
> Add queue capacity weights support in FairCallQueue
> ---
>
> Key: HADOOP-17010
> URL: https://issues.apache.org/jira/browse/HADOOP-17010
> Project: Hadoop Common
> Issue Type: New Feature
> Components: ipc
>Reporter: Fengnan Li
>Assignee: Fengnan Li
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17010.001.patch, HADOOP-17010.002.patch
>
>
> Right now in FairCallQueue all subqueues share the same capacity by evenly
> distributing total capacity. This requested feature is to make subqueues able
> to have different queue capacity where more important queues can have more
> capacity, thus less queue overflow and client backoffs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HADOOP-17010: -- Labels: ipc (was: ) > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature > Components: ipc >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > Labels: ipc > Fix For: 3.4.0 > > Attachments: HADOOP-17010.001.patch, HADOOP-17010.002.patch > > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HADOOP-17010:
--
Release Note:
This commit adds a new configuration callqueue.capacity.weights which can be
used to specify capacity allocation among all subqueues when FairCallQueue is
enabled. The value for this configuration is expected to be a list of positive
integers separated by {{,}}, whose length should equal to
scheduler.priority.levels. Each of the element in the list specifies the weight
associated with the subqueue at that index.
In default, each subqueue is associated with weight 1, i.e., all subqueues are
allocated with the same capacity.
was:
This commit adds a new configuration {{callqueue.capacity.weights}} which can
be used to specify capacity allocation among all subqueues when FairCallQueue
is enabled. The value for this configuration is expected to be a list of
positive integers separated by {{,}}, whose length should equal to
{{scheduler.priority.levels}}. Each of the element in the list specifies the
weight associated with the subqueue at that index.
In default, each subqueue is associated with weight 1, i.e., all subqueues are
allocated with the same capacity.
> Add queue capacity weights support in FairCallQueue
> ---
>
> Key: HADOOP-17010
> URL: https://issues.apache.org/jira/browse/HADOOP-17010
> Project: Hadoop Common
> Issue Type: New Feature
> Components: ipc
>Reporter: Fengnan Li
>Assignee: Fengnan Li
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17010.001.patch, HADOOP-17010.002.patch
>
>
> Right now in FairCallQueue all subqueues share the same capacity by evenly
> distributing total capacity. This requested feature is to make subqueues able
> to have different queue capacity where more important queues can have more
> capacity, thus less queue overflow and client backoffs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HADOOP-17010:
--
Component/s: ipc
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Release Note:
This commit adds a new configuration {{callqueue.capacity.weights}} which can
be used to specify capacity allocation among all subqueues when FairCallQueue
is enabled. The value for this configuration is expected to be a list of
positive integers separated by {{,}}, whose length should equal to
{{scheduler.priority.levels}}. Each of the element in the list specifies the
weight associated with the subqueue at that index.
In default, each subqueue is associated with weight 1, i.e., all subqueues are
allocated with the same capacity.
> Add queue capacity weights support in FairCallQueue
> ---
>
> Key: HADOOP-17010
> URL: https://issues.apache.org/jira/browse/HADOOP-17010
> Project: Hadoop Common
> Issue Type: New Feature
> Components: ipc
>Reporter: Fengnan Li
>Assignee: Fengnan Li
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17010.001.patch, HADOOP-17010.002.patch
>
>
> Right now in FairCallQueue all subqueues share the same capacity by evenly
> distributing total capacity. This requested feature is to make subqueues able
> to have different queue capacity where more important queues can have more
> capacity, thus less queue overflow and client backoffs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HADOOP-17010: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > Attachments: HADOOP-17010.001.patch, HADOOP-17010.002.patch > > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HADOOP-17010: Attachment: HADOOP-17010.002.patch > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > Attachments: HADOOP-17010.001.patch, HADOOP-17010.002.patch > > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HADOOP-17010: Attachment: HADOOP-17010.001.patch > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > Attachments: HADOOP-17010.001.patch > > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HADOOP-17010: Attachment: (was: HADOOP-17010.001.patch) > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HADOOP-17010: Status: Patch Available (was: In Progress) > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > Attachments: HADOOP-17010.001.patch > > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17010) Add queue capacity weights support in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HADOOP-17010: Attachment: HADOOP-17010.001.patch > Add queue capacity weights support in FairCallQueue > --- > > Key: HADOOP-17010 > URL: https://issues.apache.org/jira/browse/HADOOP-17010 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Fengnan Li >Assignee: Fengnan Li >Priority: Major > Attachments: HADOOP-17010.001.patch > > > Right now in FairCallQueue all subqueues share the same capacity by evenly > distributing total capacity. This requested feature is to make subqueues able > to have different queue capacity where more important queues can have more > capacity, thus less queue overflow and client backoffs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
