[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HDFS-7471:
-
Resolution: Cannot Reproduce
Status: Resolved (was: Patch Available)
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Binglin Chang
> Labels: BB2015-05-TBR
> Attachments: HDFS-7471.001.patch, PreCommit-HDFS-Build #9898 test -
> testNumVersionsReportedCorrect [Jenkins].html
>
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HDFS-7471:
---
Labels: BB2015-05-TBR (was: )
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Binglin Chang
> Labels: BB2015-05-TBR
> Attachments: HDFS-7471.001.patch, PreCommit-HDFS-Build #9898 test -
> testNumVersionsReportedCorrect [Jenkins].html
>
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HDFS-7471:
-
Description:
>From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
{code}
FAILED:
org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
Error Message:
The map of version counts returned by DatanodeManager was not what it was
expected to be on iteration 237 expected:<0> but was:<1>
Stack Trace:
java.lang.AssertionError: The map of version counts returned by DatanodeManager
was not what it was expected to be on iteration 237 expected:<0> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at
org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
{code}
was:
>From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
{code}
FAILED:
org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
Error Message:
The map of version counts returned by DatanodeManager was not what it was
expected to be on iteration 237 expected:<0> but was:<1>
Stack Trace:
java.lang.AssertionError: The map of version counts returned by DatanodeManager
was not what it was expected to be on iteration 237 expected:<0> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at
org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
{code}
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Binglin Chang
> Attachments: HDFS-7471.001.patch, PreCommit-HDFS-Build #9898 test -
> testNumVersionsReportedCorrect [Jenkins].html
>
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated HDFS-7471:
---
Attachment: PreCommit-HDFS-Build #9898 test -
testNumVersionsReportedCorrect [Jenkins].html
Thanks for bringing this to my attention Kihwal! I'm uploading the jenkins test
run that Kihwal has pointed out (so that we have it after Jenkins deletes old
builds)
For version4, 52 registrations were done (all unique storageIDs). There were 9
removals. I used the same seed and ran the test but it passed for me. I don't
see where there may be a bug in the test so I am inclined to believe that the
DatanodeManager has a race condition. I suggest we print out the counts that
each actor (the test and the DatanodeManager) has and revisit when the test
fails again.
I would guess the tests are slow because our Jenkins slaves are wimps. For me
TestDatanodeManager#testNumVersionsReportedCorrect took 75 seconds. We can
reduce the number of iterations if we think its taking too long, but then
ofcourse we may miss race conditions...
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Binglin Chang
> Attachments: HDFS-7471.001.patch, PreCommit-HDFS-Build #9898 test -
> testNumVersionsReportedCorrect [Jenkins].html
>
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Binglin Chang updated HDFS-7471:
Attachment: HDFS-7471.001.patch
Simple work around to increase expire interval.
After investigate the code, I suspect the current countVersion logic may have
race conditions, maybe someone more familiar with the code can provide a better
fix.
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Binglin Chang
> Attachments: HDFS-7471.001.patch
>
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Binglin Chang updated HDFS-7471:
Target Version/s: 2.7.0
Status: Patch Available (was: Open)
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Binglin Chang
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7471) TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
[
https://issues.apache.org/jira/browse/HDFS-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HDFS-7471:
-
Component/s: test
Priority: Major (was: Minor)
Affects Version/s: 3.0.0
uprating to major as this is currently the sole test blocking jenkins HDFS runs
> TestDatanodeManager#testNumVersionsReportedCorrect occasionally fails
> -
>
> Key: HDFS-7471
> URL: https://issues.apache.org/jira/browse/HDFS-7471
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>
> From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1957/ :
> {code}
> FAILED:
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect
> Error Message:
> The map of version counts returned by DatanodeManager was not what it was
> expected to be on iteration 237 expected:<0> but was:<1>
> Stack Trace:
> java.lang.AssertionError: The map of version counts returned by
> DatanodeManager was not what it was expected to be on iteration 237
> expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager.testNumVersionsReportedCorrect(TestDatanodeManager.java:150)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
