[jira] [Updated] (HDFS-2379) 0.20: Allow block reports to proceed without holding FSDataset lock
[ https://issues.apache.org/jira/browse/HDFS-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2379: -- Attachment: hdfs-2379.txt Sorry about that. Here's a patch against current branch-0.20-security. > 0.20: Allow block reports to proceed without holding FSDataset lock > --- > > Key: HDFS-2379 > URL: https://issues.apache.org/jira/browse/HDFS-2379 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node >Affects Versions: 0.20.206.0 >Reporter: Todd Lipcon >Priority: Critical > Attachments: hdfs-2379.txt, hdfs-2379.txt, hdfs-2379.txt > > > As disks are getting larger and more plentiful, we're seeing DNs with > multiple millions of blocks on a single machine. When page cache space is > tight, block reports can take multiple minutes to generate. Currently, during > the scanning of the data directories to generate a report, the FSVolumeSet > lock is held. This causes writes and reads to block, timeout, etc, causing > big problems especially for clients like HBase. > This JIRA is to explore some of the ideas originally discussed in HADOOP-4584 > for the 0.20.20x series. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123848#comment-13123848 ] Hudson commented on HDFS-2404: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1067 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1067/]) HDFS-2404. webhdfs liststatus json response is not correct. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180757 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0, 0.23.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Fix For: 0.20.205.0, 0.24.0 > > Attachments: HDFS-2404.20.txt, HDFS-2404.txt, HDFS-2404.txt, > HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2119) "hdfs version" should print out information similar to what "hadoop version" prints out
[
https://issues.apache.org/jira/browse/HDFS-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123841#comment-13123841
]
Ravi Prakash commented on HDFS-2119:
This patch applies on branch-0.23. I ran test-patch on my local machine and
this is what I got
{noformat}
+1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version )
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
{noformat}
> "hdfs version" should print out information similar to what "hadoop version"
> prints out
> ---
>
> Key: HDFS-2119
> URL: https://issues.apache.org/jira/browse/HDFS-2119
> Project: Hadoop HDFS
> Issue Type: Improvement
>Affects Versions: 0.23.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Fix For: 0.23.0
>
> Attachments: HDFS-2119.1.patch, HDFS-2119.2.patch, HDFS-2119.3.patch
>
>
> Implement version in hdfs CLI
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated HDFS-1762: - Attachment: HDFS-1762.patch Added missing license. Also, I'd propose to address any regexp issues separately. The reason for two files was to avoid too wide (e.g. generic) expressions in the first place. > Allow TestHDFSCLI to be run against a cluster > - > > Key: HDFS-1762 > URL: https://issues.apache.org/jira/browse/HDFS-1762 > Project: Hadoop HDFS > Issue Type: Test > Components: build, test >Affects Versions: 0.22.0 >Reporter: Tom White >Assignee: Konstantin Boudnik > Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, > HDFS-1762.hdfs.patch, HDFS-1762.patch > > > Currently TestHDFSCLI starts mini clusters to run tests against. It would be > useful to be able to support running against arbitrary clusters for testing > purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2379) 0.20: Allow block reports to proceed without holding FSDataset lock
[ https://issues.apache.org/jira/browse/HDFS-2379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123835#comment-13123835 ] Suresh Srinivas commented on HDFS-2379: --- Todd, this patch does not apply to the latest 20-security branch. > 0.20: Allow block reports to proceed without holding FSDataset lock > --- > > Key: HDFS-2379 > URL: https://issues.apache.org/jira/browse/HDFS-2379 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node >Affects Versions: 0.20.206.0 >Reporter: Todd Lipcon >Priority: Critical > Attachments: hdfs-2379.txt, hdfs-2379.txt > > > As disks are getting larger and more plentiful, we're seeing DNs with > multiple millions of blocks on a single machine. When page cache space is > tight, block reports can take multiple minutes to generate. Currently, during > the scanning of the data directories to generate a report, the FSVolumeSet > lock is held. This causes writes and reads to block, timeout, etc, causing > big problems especially for clients like HBase. > This JIRA is to explore some of the ideas originally discussed in HADOOP-4584 > for the 0.20.20x series. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123834#comment-13123834 ] Hudson commented on HDFS-2404: -- Integrated in Hadoop-Hdfs-trunk-Commit #1125 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1125/]) HDFS-2404. webhdfs liststatus json response is not correct. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180757 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0, 0.23.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Fix For: 0.20.205.0, 0.24.0 > > Attachments: HDFS-2404.20.txt, HDFS-2404.txt, HDFS-2404.txt, > HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123833#comment-13123833 ] Hudson commented on HDFS-2404: -- Integrated in Hadoop-Common-trunk-Commit #1047 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1047/]) HDFS-2404. webhdfs liststatus json response is not correct. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180757 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0, 0.23.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Fix For: 0.20.205.0, 0.24.0 > > Attachments: HDFS-2404.20.txt, HDFS-2404.txt, HDFS-2404.txt, > HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2404: -- Resolution: Fixed Target Version/s: 0.20.205.0, 0.23.0 (was: 0.23.0, 0.20.205.0) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch to trunk. > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0, 0.23.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Fix For: 0.20.205.0 > > Attachments: HDFS-2404.20.txt, HDFS-2404.txt, HDFS-2404.txt, > HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2404: -- Target Version/s: 0.20.205.0, 0.23.0 (was: 0.23.0, 0.20.205.0) Fix Version/s: 0.24.0 > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0, 0.23.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Fix For: 0.20.205.0, 0.24.0 > > Attachments: HDFS-2404.20.txt, HDFS-2404.txt, HDFS-2404.txt, > HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2119) "hdfs version" should print out information similar to what "hadoop version" prints out
[ https://issues.apache.org/jira/browse/HDFS-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-2119: --- Description: Implement version in hdfs CLI (was: Implement version in hdfs) Priority: Major (was: Minor) Target Version/s: 0.23.0 Affects Version/s: (was: 0.22.0) 0.23.0 Fix Version/s: (was: 0.22.0) 0.23.0 Updating JIRA attributes based on current versions > "hdfs version" should print out information similar to what "hadoop version" > prints out > --- > > Key: HDFS-2119 > URL: https://issues.apache.org/jira/browse/HDFS-2119 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.23.0 >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Fix For: 0.23.0 > > Attachments: HDFS-2119.1.patch, HDFS-2119.2.patch, HDFS-2119.3.patch > > > Implement version in hdfs CLI -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2119) "hdfs version" should print out information similar to what "hadoop version" prints out
[ https://issues.apache.org/jira/browse/HDFS-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123807#comment-13123807 ] Hadoop QA commented on HDFS-2119: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12498376/HDFS-2119.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1356//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1356//console This message is automatically generated. > "hdfs version" should print out information similar to what "hadoop version" > prints out > --- > > Key: HDFS-2119 > URL: https://issues.apache.org/jira/browse/HDFS-2119 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.22.0 >Reporter: Ravi Prakash >Assignee: Ravi Prakash >Priority: Minor > Fix For: 0.22.0 > > Attachments: HDFS-2119.1.patch, HDFS-2119.2.patch, HDFS-2119.3.patch > > > Implement version in hdfs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2119) "hdfs version" should print out information similar to what "hadoop version" prints out
[ https://issues.apache.org/jira/browse/HDFS-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-2119: --- Attachment: HDFS-2119.3.patch Rebased to current head. Can someone please commit / reject this? > "hdfs version" should print out information similar to what "hadoop version" > prints out > --- > > Key: HDFS-2119 > URL: https://issues.apache.org/jira/browse/HDFS-2119 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.22.0 >Reporter: Ravi Prakash >Assignee: Ravi Prakash >Priority: Minor > Fix For: 0.22.0 > > Attachments: HDFS-2119.1.patch, HDFS-2119.2.patch, HDFS-2119.3.patch > > > Implement version in hdfs -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1762) Allow TestHDFSCLI to be run against a cluster
[ https://issues.apache.org/jira/browse/HDFS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123796#comment-13123796 ] Konstantin Shvachko commented on HDFS-1762: --- The patch looks good. One thing I would recommend to update regular expressions for host names and paths in the spirit of HDFS-2232 and MAPREDUCE-2304. Also TestConfCluster.xml need ASF license header. > Allow TestHDFSCLI to be run against a cluster > - > > Key: HDFS-1762 > URL: https://issues.apache.org/jira/browse/HDFS-1762 > Project: Hadoop HDFS > Issue Type: Test > Components: build, test >Affects Versions: 0.22.0 >Reporter: Tom White >Assignee: Konstantin Boudnik > Attachments: HDFS-1762-20.patch, HDFS-1762.hdfs.patch, > HDFS-1762.hdfs.patch > > > Currently TestHDFSCLI starts mini clusters to run tests against. It would be > useful to be able to support running against arbitrary clusters for testing > purposes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2420) improve handling of datanode timeouts
[ https://issues.apache.org/jira/browse/HDFS-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123793#comment-13123793 ] Suresh Srinivas commented on HDFS-2420: --- Can you please add Affects Versions field in the jira. Default keep alive timeout is 600 seconds. That is a long time given default heart beat timeout is 3s. Also the datanode does keep retrying to connect to namenode. So I am not sure about the issue you have reported here. Can you upload logs for the scenario in this bug. > improve handling of datanode timeouts > - > > Key: HDFS-2420 > URL: https://issues.apache.org/jira/browse/HDFS-2420 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ron Bodkin > > If a datanode ever times out on a heart beat, it gets marked dead > permanently. I am finding that on AWS this is a periodic occurrence, i.e., > datanodes time out although the datanode process is still alive. The current > solution to this is to kill and restart each such process independently. > It would be good if there were more retry logic (e.g., blacklisting the nodes > but try heartbeats for a longer period before determining they are apparently > dead). It would also be good if refreshNodes would check and attempt to > recover timed out data nodes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2414) TestDFSRollback fails intermittently
[
https://issues.apache.org/jira/browse/HDFS-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123683#comment-13123683
]
Hudson commented on HDFS-2414:
--
Integrated in Hadoop-Mapreduce-trunk #855 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/855/])
HDFS-2414. Fix TestDFSRollback to avoid spurious failures. Contributed by
Todd Lipcon.
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180541
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRollback.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java
> TestDFSRollback fails intermittently
>
>
> Key: HDFS-2414
> URL: https://issues.apache.org/jira/browse/HDFS-2414
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node, test
>Affects Versions: 0.23.0
>Reporter: Robert Joseph Evans
>Assignee: Todd Lipcon
>Priority: Critical
> Fix For: 0.23.0
>
> Attachments: hdfs-2414.txt, hdfs-2414.txt, hdfs-2414.txt,
> run-106-failed.tgz, run-158-failed.tgz
>
>
> When running TestDFSRollback repeatedly in a loop I observed a failure rate
> of about 3%. Two separate stack traces are in the output and it appears to
> have something to do with not writing out a complete snapshot of the data for
> rollback.
> {noformat}
> ---
> Test set: org.apache.hadoop.hdfs.TestDFSRollback
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.514 sec <<<
> FAILURE!
> testRollback(org.apache.hadoop.hdfs.TestDFSRollback) Time elapsed: 8.34 sec
> <<< FAILURE!
> java.lang.AssertionError: File contents differed:
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data2/current/VERSION=5b19197114fad0a254e3f318b7f14aec
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data1/current/VERSION=ea7b000a6a1711169fc7a836b240a991
> at org.junit.Assert.fail(Assert.java:91)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertFileContentsSame(FSImageTestUtil.java:250)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertParallelFilesAreIdentical(FSImageTestUtil.java:236)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.checkResult(TestDFSRollback.java:86)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRollback.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:232)
> at junit.framework.TestSuite.run(TestSuite.java:227)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at
> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145)
> at org.apache.maven.surefire.Surefire.run(Surefire.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:290)
> at
> org.apache.maven.surefire.booter
[jira] [Commented] (HDFS-2414) TestDFSRollback fails intermittently
[
https://issues.apache.org/jira/browse/HDFS-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123680#comment-13123680
]
Hudson commented on HDFS-2414:
--
Integrated in Hadoop-Mapreduce-0.23-Build #41 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/41/])
HDFS-2414. Fix TestDFSRollback to avoid spurious failures. Contributed by
Todd Lipcon.
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180540
Files :
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRollback.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java
> TestDFSRollback fails intermittently
>
>
> Key: HDFS-2414
> URL: https://issues.apache.org/jira/browse/HDFS-2414
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node, test
>Affects Versions: 0.23.0
>Reporter: Robert Joseph Evans
>Assignee: Todd Lipcon
>Priority: Critical
> Fix For: 0.23.0
>
> Attachments: hdfs-2414.txt, hdfs-2414.txt, hdfs-2414.txt,
> run-106-failed.tgz, run-158-failed.tgz
>
>
> When running TestDFSRollback repeatedly in a loop I observed a failure rate
> of about 3%. Two separate stack traces are in the output and it appears to
> have something to do with not writing out a complete snapshot of the data for
> rollback.
> {noformat}
> ---
> Test set: org.apache.hadoop.hdfs.TestDFSRollback
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.514 sec <<<
> FAILURE!
> testRollback(org.apache.hadoop.hdfs.TestDFSRollback) Time elapsed: 8.34 sec
> <<< FAILURE!
> java.lang.AssertionError: File contents differed:
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data2/current/VERSION=5b19197114fad0a254e3f318b7f14aec
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data1/current/VERSION=ea7b000a6a1711169fc7a836b240a991
> at org.junit.Assert.fail(Assert.java:91)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertFileContentsSame(FSImageTestUtil.java:250)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertParallelFilesAreIdentical(FSImageTestUtil.java:236)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.checkResult(TestDFSRollback.java:86)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRollback.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:232)
> at junit.framework.TestSuite.run(TestSuite.java:227)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at
> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145)
> at org.apache.maven.surefire.Surefire.run(Surefire.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.booter.SurefireBooter.runSuite
[jira] [Commented] (HDFS-2414) TestDFSRollback fails intermittently
[
https://issues.apache.org/jira/browse/HDFS-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123677#comment-13123677
]
Hudson commented on HDFS-2414:
--
Integrated in Hadoop-Hdfs-0.23-Build #34 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/34/])
HDFS-2414. Fix TestDFSRollback to avoid spurious failures. Contributed by
Todd Lipcon.
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180540
Files :
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRollback.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java
> TestDFSRollback fails intermittently
>
>
> Key: HDFS-2414
> URL: https://issues.apache.org/jira/browse/HDFS-2414
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node, test
>Affects Versions: 0.23.0
>Reporter: Robert Joseph Evans
>Assignee: Todd Lipcon
>Priority: Critical
> Fix For: 0.23.0
>
> Attachments: hdfs-2414.txt, hdfs-2414.txt, hdfs-2414.txt,
> run-106-failed.tgz, run-158-failed.tgz
>
>
> When running TestDFSRollback repeatedly in a loop I observed a failure rate
> of about 3%. Two separate stack traces are in the output and it appears to
> have something to do with not writing out a complete snapshot of the data for
> rollback.
> {noformat}
> ---
> Test set: org.apache.hadoop.hdfs.TestDFSRollback
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.514 sec <<<
> FAILURE!
> testRollback(org.apache.hadoop.hdfs.TestDFSRollback) Time elapsed: 8.34 sec
> <<< FAILURE!
> java.lang.AssertionError: File contents differed:
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data2/current/VERSION=5b19197114fad0a254e3f318b7f14aec
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data1/current/VERSION=ea7b000a6a1711169fc7a836b240a991
> at org.junit.Assert.fail(Assert.java:91)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertFileContentsSame(FSImageTestUtil.java:250)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertParallelFilesAreIdentical(FSImageTestUtil.java:236)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.checkResult(TestDFSRollback.java:86)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRollback.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:232)
> at junit.framework.TestSuite.run(TestSuite.java:227)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at
> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145)
> at org.apache.maven.surefire.Surefire.run(Surefire.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
[jira] [Commented] (HDFS-2414) TestDFSRollback fails intermittently
[
https://issues.apache.org/jira/browse/HDFS-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123673#comment-13123673
]
Hudson commented on HDFS-2414:
--
Integrated in Hadoop-Hdfs-trunk #825 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/825/])
HDFS-2414. Fix TestDFSRollback to avoid spurious failures. Contributed by
Todd Lipcon.
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1180541
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRollback.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java
> TestDFSRollback fails intermittently
>
>
> Key: HDFS-2414
> URL: https://issues.apache.org/jira/browse/HDFS-2414
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node, test
>Affects Versions: 0.23.0
>Reporter: Robert Joseph Evans
>Assignee: Todd Lipcon
>Priority: Critical
> Fix For: 0.23.0
>
> Attachments: hdfs-2414.txt, hdfs-2414.txt, hdfs-2414.txt,
> run-106-failed.tgz, run-158-failed.tgz
>
>
> When running TestDFSRollback repeatedly in a loop I observed a failure rate
> of about 3%. Two separate stack traces are in the output and it appears to
> have something to do with not writing out a complete snapshot of the data for
> rollback.
> {noformat}
> ---
> Test set: org.apache.hadoop.hdfs.TestDFSRollback
> ---
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.514 sec <<<
> FAILURE!
> testRollback(org.apache.hadoop.hdfs.TestDFSRollback) Time elapsed: 8.34 sec
> <<< FAILURE!
> java.lang.AssertionError: File contents differed:
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data2/current/VERSION=5b19197114fad0a254e3f318b7f14aec
>
> /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data1/current/VERSION=ea7b000a6a1711169fc7a836b240a991
> at org.junit.Assert.fail(Assert.java:91)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertFileContentsSame(FSImageTestUtil.java:250)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertParallelFilesAreIdentical(FSImageTestUtil.java:236)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.checkResult(TestDFSRollback.java:86)
> at
> org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRollback.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:232)
> at junit.framework.TestSuite.run(TestSuite.java:227)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at
> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
> at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:145)
> at org.apache.maven.surefire.Surefire.run(Surefire.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:290)
> at
> org.apache.maven.surefire.booter.SurefireB
