[jira] [Commented] (HDFS-7325) Prevent thundering herd problem in ByteArrayManager by using notify not notifyAll
[
https://issues.apache.org/jira/browse/HDFS-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198776#comment-14198776
]
Colin Patrick McCabe commented on HDFS-7325:
bq. The above should be =.
One tricky thing here is that the patch moves this block after the
{{numAllocated--}}. So I believe this should be correct...
bq. How about simply including the change in HDFS-7358 and resolving this?
OK.
Prevent thundering herd problem in ByteArrayManager by using notify not
notifyAll
-
Key: HDFS-7325
URL: https://issues.apache.org/jira/browse/HDFS-7325
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-7325.001.patch
Currently ByteArrayManager wakes all waiting threads whenever a byte array is
released and count == limit. However, only one thread can proceed.With a
large number of waiters, this will cause a thundering herd problem. (See
http://en.wikipedia.org/wiki/Thundering_herd_problem.) We should avoid this
by only waking a single thread.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7325) Prevent thundering herd problem in ByteArrayManager by using notify not notifyAll
[ https://issues.apache.org/jira/browse/HDFS-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198900#comment-14198900 ] Tsz Wo Nicholas Sze commented on HDFS-7325: --- One tricky thing here is that the patch moves this block after the numAllocated--. ... Ah, you are correct. We actually do not need the if since numAllocated maxAllocated is always true. Prevent thundering herd problem in ByteArrayManager by using notify not notifyAll - Key: HDFS-7325 URL: https://issues.apache.org/jira/browse/HDFS-7325 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.6.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-7325.001.patch Currently ByteArrayManager wakes all waiting threads whenever a byte array is released and count == limit. However, only one thread can proceed.With a large number of waiters, this will cause a thundering herd problem. (See http://en.wikipedia.org/wiki/Thundering_herd_problem.) We should avoid this by only waking a single thread. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7325) Prevent thundering herd problem in ByteArrayManager by using notify not notifyAll
[
https://issues.apache.org/jira/browse/HDFS-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197333#comment-14197333
]
Tsz Wo Nicholas Sze commented on HDFS-7325:
---
Thanks for the advise about the thundering herd problem.
{code}
+ if (numAllocated maxAllocated) {
+notify();
+ }
{code}
The above should be =. When numAllocated == maxAllocated, we definitely
should call notify(). How about simply including the change in HDFS-7358 and
resolving this?
Prevent thundering herd problem in ByteArrayManager by using notify not
notifyAll
-
Key: HDFS-7325
URL: https://issues.apache.org/jira/browse/HDFS-7325
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-7325.001.patch
Currently ByteArrayManager wakes all waiting threads whenever a byte array is
released and count == limit. However, only one thread can proceed.With a
large number of waiters, this will cause a thundering herd problem. (See
http://en.wikipedia.org/wiki/Thundering_herd_problem.) We should avoid this
by only waking a single thread.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7325) Prevent thundering herd problem in ByteArrayManager by using notify not notifyAll
[
https://issues.apache.org/jira/browse/HDFS-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195124#comment-14195124
]
Hadoop QA commented on HDFS-7325:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12678981/HDFS-7325.001.patch
against trunk revision 67f13b5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestEnhancedByteBufferAccess
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8626//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8626//console
This message is automatically generated.
Prevent thundering herd problem in ByteArrayManager by using notify not
notifyAll
-
Key: HDFS-7325
URL: https://issues.apache.org/jira/browse/HDFS-7325
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-7325.001.patch
Currently ByteArrayManager wakes all waiting threads whenever a byte array is
released and count == limit. However, only one thread can proceed.With a
large number of waiters, this will cause a thundering herd problem. (See
http://en.wikipedia.org/wiki/Thundering_herd_problem.) We should avoid this
by only waking a single thread.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
