[
https://issues.apache.org/jira/browse/HADOOP-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325015#comment-15325015
]
Konstantin Shvachko commented on HADOOP-13189:
----------------------------------------------
Hey [~mingma] good to hear from you. I should have given a bit more context in
the description. Doing it now.
So we are trying to solve two inter-related issues here:
# Inconsistency between the config value of {{ipc.server.handler.queue.size}},
which is defined as _"How many calls per handler are allowed in the queue."_,
and the actual queue size when {{FairCallQueue}} is used, which lets 4 times
more.
# A need to reconfigure the cluster when switching from one CallQueue
implementation to another.
The latter is important to prevent NN meltdown. Before we learned about this
issue we saw too many clients connecting to the NameNode and making it
unresponsive. This is related to read requests like {{listStatus}} and
{{getFileInfo}}. And it worked fine with the default queue, but not with
FairCallQueue. Looking deeper it was noticed that NN allowed more requests with
FairCallQueue than with the default. We had to reconfigure NN and adjust the
queue size, but now we have problem 1. Hope this makes sense.
With your example when all requests go only into the high priority queue one
should probably switch to the default queue as FairCallQueue functionality is
not utilized.
> FairCallQueue makes callQueue larger than the configured capacity.
> ------------------------------------------------------------------
>
> Key: HADOOP-13189
> URL: https://issues.apache.org/jira/browse/HADOOP-13189
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 2.6.0
> Reporter: Konstantin Shvachko
> Assignee: Vinitha Reddy Gankidi
> Attachments: HADOOP-13189.001.patch, HADOOP-13189.002.patch,
> HADOOP-13189.003.patch
>
>
> {{FairCallQueue}} divides {{callQueue}} into multiple (4 by default)
> sub-queues, with each sub-queue corresponding to a different level of
> priority. The constructor for {{FairCallQueue}} takes the same parameter
> {{capacity}} as the default CallQueue implementation, and allocates all its
> sub-queues of size {{capacity}}. With 4 levels of priority (sub-queues) by
> default it results in the total callQueue size 4 times larger than it should
> be based on the configuration.
> {{capacity}} should be divided by the number of sub-queues at some place.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
