[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18024380#comment-18024380
]
ASF GitHub Bot commented on HDFS-17341:
---------------------------------------
github-actions[bot] commented on PR #6461:
URL: https://github.com/apache/hadoop/pull/6461#issuecomment-3363698000
We're closing this stale PR because it has been open for 100 days with no
activity. This isn't a judgement on the merit of the PR in any way. It's just a
way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working
on it, please feel free to re-open it and ask for a committer to remove the
stale tag and review again.
Thanks all for your contribution.
> Support dedicated user queues in Namenode FairCallQueue
> -------------------------------------------------------
>
> Key: HDFS-17341
> URL: https://issues.apache.org/jira/browse/HDFS-17341
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.10.0, 3.4.0
> Reporter: Lei Yang
> Priority: Major
> Labels: pull-request-available
>
> Some service users today in namenode like ETL, metrics collection, ad-hoc
> users that are critical to run business critical job accounts for many
> traffic in namenode and shouldn't be throttled the same way as other
> individual users in FCQ.
> There is [feature|https://issues.apache.org/jira/browse/HADOOP-17165] in
> namenode to always prioritize some service users to not subject to FCQ
> scheduling. (Those users are always p0) but it is not perfect and it doesn't
> account for traffic surge from those users.
> The idea is to allocate dedicated rpc queues for those service users with
> bounded queue capacity and allocate processing weight for those users. If
> queue is full, those users are expected to backoff and retry.
>
> New configs:
> {code:java}
> "faircallqueue.reserved.users"; // list of service users that are assigned to
> dedicated queue
> "faircallqueue.reserved.users.max"; // max number of service users allowed
> "faircallqueue.reserved.users.capacities"; // custom queue capacities for
> each service user
> "faircallqueue.multiplexer.reserved.weights"; // processing weights for each
> dedicated queue{code}
> For instance, for a FCQ with 4 priority levels, 2 reserved users(a, b)
> FCQ would look like:
>
> {code:java}
> P0: shared queue
> P1: shared queue
> P2: shared queue
> P3: shared queue
> P4: dedicated for user a
> P5: dedicated for user b{code}
> {color:#172b4d}The Multiplexer would have following weights{color}
> {color:#172b4d}shared queue default weights: [8, 4, 2, 1]{color}
> {color:#172b4d}reserved queue weights=[3, 2]{color}
> {color:#172b4d}So user a gets 15% of total cycles, user b gets 10% of total
> cycles.{color}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
