[
https://issues.apache.org/jira/browse/HDFS-12822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangming Zhang updated HDFS-12822:
-----------------------------------
Description:
Description: Hi, When I ran the HDFS unit test and got a failure in
TestDirectoryScanner.java test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
Running
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
227.046 sec <<< FAILURE! - in
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
Time elapsed: 198.014 sec <<< FAILURE!
java.lang.AssertionError: Throttle is too permissive
at
org.junit.Assert.fail(Assert.java:88)
at
org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)
And below is the failure part of source code TestDirectoryScanner.java:
...........
{quote} while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
scanner = new DirectoryScanner(dataNode, fds, conf);
ratio = runThrottleTest(blocks);
retries -= 1;
}
// Waiting should be about 9x running.
LOG.info("RATIO: " + ratio);
assertTrue("Throttle is too restrictive", ratio <= 10f);
assertTrue("Throttle is too permissive", ratio >= 7f);
............
private float runThrottleTest(int blocks) throws IOException {
scanner.setRetainDiffs(true);
scan(blocks, 0, 0, 0, 0, 0);
scanner.shutdown();
assertFalse(scanner.getRunStatus());
return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
}
.............{quote}
The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the
code thrown out an assertTrue failure.
My questions are:
1. Why the ratio was set between 7f and 10f, is it a empirical value?
2. The ratio is smaller than 7f in AArch64 platform, is this value
within normal range?
Could anyone help? Thanks a lot.
was:
Description: Hi, When I ran the HDFS unit test and got a failure in
TestDirectoryScanner.java test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
Running
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
227.046 sec <<< FAILURE! - in
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
Time elapsed: 198.014 sec <<< FAILURE!
java.lang.AssertionError: Throttle is too permissive
at
org.junit.Assert.fail(Assert.java:88)
at
org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)
And below is the failure part of source code TestDirectoryScanner.java:
...........
while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
scanner = new DirectoryScanner(dataNode, fds, conf);
ratio = runThrottleTest(blocks);
retries -= 1;
}
// Waiting should be about 9x running.
LOG.info("RATIO: " + ratio);
assertTrue("Throttle is too restrictive", ratio <= 10f);
assertTrue("Throttle is too permissive", ratio >= 7f);
............
private float runThrottleTest(int blocks) throws IOException {
scanner.setRetainDiffs(true);
scan(blocks, 0, 0, 0, 0, 0);
scanner.shutdown();
assertFalse(scanner.getRunStatus());
return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
}
.............
The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the
code thrown out an assertTrue failure.
My questions are:
1. Why the ratio was set between 7f and 10f, is it a empirical value?
2. The ratio is smaller than 7f in AArch64 platform, is this value
within normal range?
Could anyone help? Thanks a lot.
> HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling:
> Throttle is too permissive
> --------------------------------------------------------------------------------------------------
>
> Key: HDFS-12822
> URL: https://issues.apache.org/jira/browse/HDFS-12822
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
> Affects Versions: 3.1.0
> Environment: ARMv8 AArch64, Ubuntu16.04
> Reporter: Guangming Zhang
> Priority: Minor
> Labels: dtest, easyfix, maven, test
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Description: Hi, When I ran the HDFS unit test and got a failure in
> TestDirectoryScanner.java test case :
> TestDirectoryScanner.testThrottling:624 Throttle is too permissive
> detail:
> Running
> org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
> Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 227.046 sec <<< FAILURE! - in
> org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
>
> testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
> Time elapsed: 198.014 sec <<< FAILURE!
> java.lang.AssertionError: Throttle is too permissive
> at
> org.junit.Assert.fail(Assert.java:88)
> at
> org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)
> And below is the failure part of source code TestDirectoryScanner.java:
> ...........
> {quote} while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
> scanner = new DirectoryScanner(dataNode, fds, conf);
> ratio = runThrottleTest(blocks);
> retries -= 1;
> }
> // Waiting should be about 9x running.
> LOG.info("RATIO: " + ratio);
> assertTrue("Throttle is too restrictive", ratio <= 10f);
> assertTrue("Throttle is too permissive", ratio >= 7f);
> ............
> private float runThrottleTest(int blocks) throws IOException {
> scanner.setRetainDiffs(true);
> scan(blocks, 0, 0, 0, 0, 0);
> scanner.shutdown();
> assertFalse(scanner.getRunStatus());
> return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
> }
> .............{quote}
> The ratio in my test is 6.0578866, which is smaller than 7f in the code. So
> the code thrown out an assertTrue failure.
> My questions are:
> 1. Why the ratio was set between 7f and 10f, is it a empirical value?
> 2. The ratio is smaller than 7f in AArch64 platform, is this value
> within normal range?
> Could anyone help? Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
