[
https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815511#comment-16815511
]
Daryn Sharp commented on HDFS-14403:
------------------------------------
Sorry for delay, been a little bit distracted. We should consider
incorporating elements from both patches. My splicing wasn't quite as abstract
as this one. Notables and cheats:
* {{CallQueueManager}}: Cheated by using {{scheduler instanceof
DecayRpcScheduler}} to avoid disrupting the interface.
* {{DecayRpcScheduler}}: Hijacked the "*CallCount" variables to be either count
or cost.
* {{Protobuf/WritableRpcEngine}}: Hoisted redundant metrics update into server
to allowed finer-grain breakdown.
** Dropped bizarre metric update that recorded failed calls as their exception.
Always complicated historical trend analysis of call load or which op was
being flooded.
* {{Server}}: metrics tweaks
** Skipped calls from disconnected client are no longer billed as near-zero
processing time – it worse the backlog, the better processing used to look.
** Add lock wait metric.
** Normal metric for processing time still includes lock wait.
** Detailed per op processing time does not include lock wait. Can finally do
production load analysis of call expense!
* {{RpcMetrics}}: changed timeunits to microseconds because milliseconds
accuracy is near worthless. probably/maybe should be configurable.
* {{FSNamesystemLock}}: callback feels a little clunky but most performant I
could quickly devise.
> Cost-Based RPC FairCallQueue
> ----------------------------
>
> Key: HDFS-14403
> URL: https://issues.apache.org/jira/browse/HDFS-14403
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ipc, namenode
> Reporter: Erik Krogen
> Assignee: Christopher Gregorian
> Priority: Major
> Labels: qos, rpc
> Attachments: CostBasedFairCallQueueDesign_v0.pdf,
> HDFS-14403.001.patch, HDFS-14403.002.patch, HDFS-14403.003.patch
>
>
> HADOOP-15016 initially described extensions to the Hadoop FairCallQueue
> encompassing both cost-based analysis of incoming RPCs, as well as support
> for reservations of RPC capacity for system/platform users. This JIRA intends
> to track the former, as HADOOP-15016 was repurposed to more specifically
> focus on the reservation portion of the work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
