[jira] [Commented] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[
https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065740#comment-13065740
]
Aaron T. Myers commented on HDFS-2152:
--
Hey Uma, patch looks pretty good. One small comment:
# I think you should leave in the {{os.close()}} call, and then just set
{{os}} to null. I say that because relying only on {{IOUtils.cleanup(...)}} to
do the close will swallow a legitimate error.
I'll commit it pending that issue being addressed and Hudson results.
TestWriteConfigurationToDFS causing the random failures
---
Key: HDFS-2152
URL: https://issues.apache.org/jira/browse/HDFS-2152
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
Attachments: HDFS-2152.patch
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2147) Move cluster network topology to block management
[
https://issues.apache.org/jira/browse/HDFS-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2147:
-
Attachment: h2147_20110715.patch
h2147_20110715.patch: renamed {{clusterNetwork}} to {{networktopology}}.
Move cluster network topology to block management
-
Key: HDFS-2147
URL: https://issues.apache.org/jira/browse/HDFS-2147
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Attachments: h2147_20110713.patch, h2147_20110714.patch,
h2147_20110714b.patch, h2147_20110715.patch
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2153) DFSClientAdapter should be put under test
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2153:
-
Status: Patch Available (was: Open)
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2153:
-
Attachment: h2153_20110715.patch
h2153_20110715.patch: moved it to test and fix some javac warnings in
{{OfflineEditsViewerHelper}}.
For committing the patch, please use svn mv.
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1149) Lease reassignment is not persisted to edit log
[ https://issues.apache.org/jira/browse/HDFS-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065756#comment-13065756 ] Tsz Wo (Nicholas), SZE commented on HDFS-1149: -- DFSClientAdapter should be put under test; filed HDFS-2153. Lease reassignment is not persisted to edit log --- Key: HDFS-1149 URL: https://issues.apache.org/jira/browse/HDFS-1149 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0, 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Aaron T. Myers Fix For: 0.23.0 Attachments: editsStored, hdfs-1149.0.patch, hdfs-1149.1.patch, hdfs-1149.1.patch, hdfs-1149.2.patch During lease recovery, the lease gets reassigned to a special NN holder. This is not currently persisted to the edit log, which means that after an NN restart, the original leaseholder could end up allocating more blocks or completing a file that has already started recovery. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2131) Tests for HADOOP-7361
[ https://issues.apache.org/jira/browse/HDFS-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065777#comment-13065777 ] Tsz Wo (Nicholas), SZE commented on HDFS-2131: -- Sorry Todd, filed HDFS-2154. Tests for HADOOP-7361 - Key: HDFS-2131 URL: https://issues.apache.org/jira/browse/HDFS-2131 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 0.20.1, 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.23.0 Attachments: HADOOP-7361-test.patch, HADOOP-7361-test.patch, HADOOP-7361-test.patch, HADOOP-7361-test_v3.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2154) TestDFSShell should use test dir
TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2154: - Status: Patch Available (was: Open) TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2154: - Attachment: h2154_20110715.patch h2154_20110715.patch: used test dir and fixed some deprecated warnings. TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2131) Tests for HADOOP-7361
[ https://issues.apache.org/jira/browse/HDFS-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065783#comment-13065783 ] Uma Maheswara Rao G commented on HDFS-2131: --- Sorry, Just now i have seen it. I was trying to update the patch...Looks Nicholas already updated it. Thanks a lot Nicholas. and Todd for noticing it. Tests for HADOOP-7361 - Key: HDFS-2131 URL: https://issues.apache.org/jira/browse/HDFS-2131 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 0.20.1, 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.23.0 Attachments: HADOOP-7361-test.patch, HADOOP-7361-test.patch, HADOOP-7361-test.patch, HADOOP-7361-test_v3.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2147) Move cluster network topology to block management
[ https://issues.apache.org/jira/browse/HDFS-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065797#comment-13065797 ] Hadoop QA commented on HDFS-2147: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486558/h2147_20110715.patch against trunk revision 1146516. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/938//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/938//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/938//console This message is automatically generated. Move cluster network topology to block management - Key: HDFS-2147 URL: https://issues.apache.org/jira/browse/HDFS-2147 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2147_20110713.patch, h2147_20110714.patch, h2147_20110714b.patch, h2147_20110715.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2155) Umbrella JIRA for wire compatibility
Umbrella JIRA for wire compatibility Key: HDFS-2155 URL: https://issues.apache.org/jira/browse/HDFS-2155 Project: Hadoop HDFS Issue Type: New Feature Reporter: Tsz Wo (Nicholas), SZE Assignee: Sanjay Radia This is HDFS side of HADOOP-7347. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2155) Define a DFSClient interface for wire compatibility
[ https://issues.apache.org/jira/browse/HDFS-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2155: - Description: Define a DFSClient interface so that different versions of client can be loaded for talking to the corresponding version of server. For more details, see HADOOP-7347. (was: This is HDFS side of HADOOP-7347.) Assignee: Tsz Wo (Nicholas), SZE (was: Sanjay Radia) Summary: Define a DFSClient interface for wire compatibility (was: Umbrella JIRA for wire compatibility) Define a DFSClient interface for wire compatibility --- Key: HDFS-2155 URL: https://issues.apache.org/jira/browse/HDFS-2155 Project: Hadoop HDFS Issue Type: New Feature Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Define a DFSClient interface so that different versions of client can be loaded for talking to the corresponding version of server. For more details, see HADOOP-7347. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
[
https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065801#comment-13065801
]
Ivan Kelly commented on HDFS-1073:
--
{quote}
LogHeader has a read method but not a write. Will it make sense to encapsulate
both read and write of the header in the same class?
Agreed - Ivan has opened HDFS-2149 - I'd propose we do that under that JIRA?
{quote}
HDFS-2149 will probably remove LogHeader completely. I plan to add a
getVersion() call to InputStreams and each stream will handle it's own metadata
internally. So EditLogFileInputStream will read it's version on creation, or
first call to read etc. The input and output stream will be packet based, so an
input stream is basically an iterator over FSEditLogOp objects and output
stream is a sink for FSEditLogOp objects. I think the way I've implemented the
FSEditLogOp objects should avoid all extra copies and object creation. Whats
more, there's plenty to room to improve this by removing the creation of
ArrayWritables and DeprecatedUTF8 objects and just write strings and arrays
directly.
Simpler model for Namenode's fs Image and edit Logs
Key: HDFS-1073
URL: https://issues.apache.org/jira/browse/HDFS-1073
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee: Todd Lipcon
Attachments: hdfs-1073-editloading-algos.txt, hdfs-1073.txt,
hdfs1073.pdf, hdfs1073.pdf, hdfs1073.pdf, hdfs1073.tex
The naming and handling of NN's fsImage and edit logs can be significantly
improved resulting simpler and more robust code.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2155) Define a DFSClient interface for wire compatibility
[ https://issues.apache.org/jira/browse/HDFS-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2155: - Component/s: hdfs client Define a DFSClient interface for wire compatibility --- Key: HDFS-2155 URL: https://issues.apache.org/jira/browse/HDFS-2155 Project: Hadoop HDFS Issue Type: New Feature Components: hdfs client Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Define a DFSClient interface so that different versions of client can be loaded for talking to the corresponding version of server. For more details, see HADOOP-7347. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065803#comment-13065803
]
Hadoop QA commented on HDFS-2153:
-
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12486563/h2153_20110715.patch
against trunk revision 1146516.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/940//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/940//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/940//console
This message is automatically generated.
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2025) Go Back to File View link is not working in tail.jsp
[ https://issues.apache.org/jira/browse/HDFS-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065804#comment-13065804 ] sravankorumilli commented on HDFS-2025: --- In the tail.jsp we are using the charset=UTF-8 then the output of out.print will be automatically encoded to UTF-8. For Reference :- http://www.w3.org/International/O-HTTP-charset Go Back to File View link is not working in tail.jsp Key: HDFS-2025 URL: https://issues.apache.org/jira/browse/HDFS-2025 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 0.23.0 Reporter: sravankorumilli Assignee: sravankorumilli Priority: Minor Attachments: HDFS-2025.patch, HDFS-2025_1.patch, ScreenShot_1.jpg While browsing the file system. Click on any file link to go to the page where the file contents are displayed, then when we click on '*Tail this file*' link. The control will go to the tail.jsp here when we Click on '*Go Back to File View*' option. HTTP Error page not found will come. This is because the referrer URL is encoded and the encoded URL is itself being used in the '*Go Back to File View*' hyperlink which will be treated as a relative URL and thus the HTTP request will fail. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2149) Move EditLogOp serialisation and deserialation into one place
[ https://issues.apache.org/jira/browse/HDFS-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065802#comment-13065802 ] Ivan Kelly commented on HDFS-2149: -- REPOST from HDFS-1073: HDFS-2149 will probably remove LogHeader completely. I plan to add a getVersion() call to InputStreams and each stream will handle it's own metadata internally. So EditLogFileInputStream will read it's version on creation, or first call to read etc. The input and output stream will be packet based, so an input stream is basically an iterator over FSEditLogOp objects and output stream is a sink for FSEditLogOp objects. I think the way I've implemented the FSEditLogOp objects should avoid all extra copies and object creation. Whats more, there's plenty to room to improve this by removing the creation of ArrayWritables and DeprecatedUTF8 objects and just write strings and arrays directly. Move EditLogOp serialisation and deserialation into one place - Key: HDFS-2149 URL: https://issues.apache.org/jira/browse/HDFS-2149 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.23.0 Attachments: HDFS-2149.diff On trunk serialisation of editlog ops is in FSEditLog#log* and deserialisation is in FSEditLogOp.*Op . This improvement is to move the serialisation code into one place, i.e under FSEditLogOp.*Op. This is part of HDFS-1580. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065828#comment-13065828 ] Hadoop QA commented on HDFS-2154: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486572/h2154_20110715.patch against trunk revision 1146516. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/942//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/942//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/942//console This message is automatically generated. TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Status: Open (was: Patch Available) TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Status: Patch Available (was: Open) TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Attachment: HDFS-2152-1.patch TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[
https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065908#comment-13065908
]
Uma Maheswara Rao G commented on HDFS-2152:
---
Hi Aaron,
Thanks alot for taking a look.
{quote}
I think you should leave in the os.close() call, and then just set os to
null. I say that because relying only on IOUtils.cleanup(...) to do the close
will swallow a legitimate error.
{quote}
Got you.:-) . you are suggesting the similar(HADOOP-7090) pattern right? I
thought IOUtils.cleanup will be enough for tests. You are right, Even in tests,
this will be good approach to follow.Otherwise we may not know some of the
exceptions while closing, due to suppression in IOUtils..cleanup.
Patch incorporates Aaron comments.
TestWriteConfigurationToDFS causing the random failures
---
Key: HDFS-2152
URL: https://issues.apache.org/jira/browse/HDFS-2152
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
Attachments: HDFS-2152-1.patch, HDFS-2152.patch
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1774) Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.
[ https://issues.apache.org/jira/browse/HDFS-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1774: -- Attachment: HDFS-1774-1.patch Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. --- Key: HDFS-1774 URL: https://issues.apache.org/jira/browse/HDFS-1774 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-1774-1.patch, HDFS-1774.patch Inner class FSDir constructor is doing duplicate iterations over the listed files in the passed directory. We can optimize this to single loop and also we can avoid isDirectory check which will perform some native invocations. Consider a case: one directory has only one child directory and 1 files. 1) First loop will get the number of children directories. 2) if (numChildren 0) , This condition will satisfy and again it will iterate 10001 times and also will check isDirectory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1774) Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.
[ https://issues.apache.org/jira/browse/HDFS-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1774: -- Status: Open (was: Patch Available) Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. --- Key: HDFS-1774 URL: https://issues.apache.org/jira/browse/HDFS-1774 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-1774-1.patch, HDFS-1774.patch Inner class FSDir constructor is doing duplicate iterations over the listed files in the passed directory. We can optimize this to single loop and also we can avoid isDirectory check which will perform some native invocations. Consider a case: one directory has only one child directory and 1 files. 1) First loop will get the number of children directories. 2) if (numChildren 0) , This condition will satisfy and again it will iterate 10001 times and also will check isDirectory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1774) Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.
[ https://issues.apache.org/jira/browse/HDFS-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1774: -- Status: Patch Available (was: Open) Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. --- Key: HDFS-1774 URL: https://issues.apache.org/jira/browse/HDFS-1774 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-1774-1.patch, HDFS-1774.patch Inner class FSDir constructor is doing duplicate iterations over the listed files in the passed directory. We can optimize this to single loop and also we can avoid isDirectory check which will perform some native invocations. Consider a case: one directory has only one child directory and 1 files. 1) First loop will get the number of children directories. 2) if (numChildren 0) , This condition will satisfy and again it will iterate 10001 times and also will check isDirectory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1774) Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.
[ https://issues.apache.org/jira/browse/HDFS-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1774: -- Summary: Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. (was: Optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.) Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. - Key: HDFS-1774 URL: https://issues.apache.org/jira/browse/HDFS-1774 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-1774-1.patch, HDFS-1774.patch Inner class FSDir constructor is doing duplicate iterations over the listed files in the passed directory. We can optimize this to single loop and also we can avoid isDirectory check which will perform some native invocations. Consider a case: one directory has only one child directory and 1 files. 1) First loop will get the number of children directories. 2) if (numChildren 0) , This condition will satisfy and again it will iterate 10001 times and also will check isDirectory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1774) Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.
[ https://issues.apache.org/jira/browse/HDFS-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065910#comment-13065910 ] Uma Maheswara Rao G commented on HDFS-1774: --- Attached the Patch for Review. Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. - Key: HDFS-1774 URL: https://issues.apache.org/jira/browse/HDFS-1774 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-1774-1.patch, HDFS-1774.patch Inner class FSDir constructor is doing duplicate iterations over the listed files in the passed directory. We can optimize this to single loop and also we can avoid isDirectory check which will perform some native invocations. Consider a case: one directory has only one child directory and 1 files. 1) First loop will get the number of children directories. 2) if (numChildren 0) , This condition will satisfy and again it will iterate 10001 times and also will check isDirectory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13065986#comment-13065986 ] Uma Maheswara Rao G commented on HDFS-2154: --- Hi Nicholas, I have verified with this patch, now files are getting created inside TEST_ROOT_DIR. Also you have cleaned the deprecated API usage. - Thanks +1 Patch looks good to me. TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2154: - Resolution: Fixed Fix Version/s: 0.23.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Thanks Uma for reviewing it. I have committed this. TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Fix For: 0.23.0 Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066006#comment-13066006
]
Todd Lipcon commented on HDFS-2153:
---
+1. Looks like a few other cleanups in this patch though?
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2080) Speed up DFS read path by lessening checksum overhead
[ https://issues.apache.org/jira/browse/HDFS-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066019#comment-13066019 ] Kihwal Lee commented on HDFS-2080: -- I agree. We can have something simpler. Are we doing anything in compressor/decompressor? Do we also need something there? Speed up DFS read path by lessening checksum overhead - Key: HDFS-2080 URL: https://issues.apache.org/jira/browse/HDFS-2080 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: hdfs-2080.txt, hdfs-2080.txt I've developed a series of patches that speeds up the HDFS read path by a factor of about 2.5x (~300M/sec to ~800M/sec for localhost reading from buffer cache) and also will make it easier to allow for advanced users (eg hbase) to skip a buffer copy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2150) Use new Trash Emptier methods
[ https://issues.apache.org/jira/browse/HDFS-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066028#comment-13066028 ] Uma Maheswara Rao G commented on HDFS-2150: --- Jitendra, Updated the description.- thanks Use new Trash Emptier methods - Key: HDFS-2150 URL: https://issues.apache.org/jira/browse/HDFS-2150 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Uma Maheswara Rao G Assignee: Usman Masood This issue is for HADOOP-7460 changes which are required in HDFS. Since, we are implementing the pluggable trash policy in Common, changes required in HDFS to use that policy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2080) Speed up DFS read path by lessening checksum overhead
[ https://issues.apache.org/jira/browse/HDFS-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066025#comment-13066025 ] stack commented on HDFS-2080: - @Kihwal Do you mean the hbase compressor/decompressor? Speed up DFS read path by lessening checksum overhead - Key: HDFS-2080 URL: https://issues.apache.org/jira/browse/HDFS-2080 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: hdfs-2080.txt, hdfs-2080.txt I've developed a series of patches that speeds up the HDFS read path by a factor of about 2.5x (~300M/sec to ~800M/sec for localhost reading from buffer cache) and also will make it easier to allow for advanced users (eg hbase) to skip a buffer copy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2080) Speed up DFS read path by lessening checksum overhead
[ https://issues.apache.org/jira/browse/HDFS-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066033#comment-13066033 ] Kihwal Lee commented on HDFS-2080: -- I meant the ones in HDFS. They also use direct byte buffer. I thought they might have a similar issue/solution. Speed up DFS read path by lessening checksum overhead - Key: HDFS-2080 URL: https://issues.apache.org/jira/browse/HDFS-2080 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: hdfs-2080.txt, hdfs-2080.txt I've developed a series of patches that speeds up the HDFS read path by a factor of about 2.5x (~300M/sec to ~800M/sec for localhost reading from buffer cache) and also will make it easier to allow for advanced users (eg hbase) to skip a buffer copy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2080) Speed up DFS read path by lessening checksum overhead
[ https://issues.apache.org/jira/browse/HDFS-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066035#comment-13066035 ] Kihwal Lee commented on HDFS-2080: -- Well, since it's in common, so it might be used by others as well. Does HBase use direct byte buffer for comp/decomp? Speed up DFS read path by lessening checksum overhead - Key: HDFS-2080 URL: https://issues.apache.org/jira/browse/HDFS-2080 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: hdfs-2080.txt, hdfs-2080.txt I've developed a series of patches that speeds up the HDFS read path by a factor of about 2.5x (~300M/sec to ~800M/sec for localhost reading from buffer cache) and also will make it easier to allow for advanced users (eg hbase) to skip a buffer copy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2080) Speed up DFS read path by lessening checksum overhead
[ https://issues.apache.org/jira/browse/HDFS-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066041#comment-13066041 ] stack commented on HDFS-2080: - No (not yet at least). Speed up DFS read path by lessening checksum overhead - Key: HDFS-2080 URL: https://issues.apache.org/jira/browse/HDFS-2080 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: hdfs-2080.txt, hdfs-2080.txt I've developed a series of patches that speeds up the HDFS read path by a factor of about 2.5x (~300M/sec to ~800M/sec for localhost reading from buffer cache) and also will make it easier to allow for advanced users (eg hbase) to skip a buffer copy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2080) Speed up DFS read path by lessening checksum overhead
[ https://issues.apache.org/jira/browse/HDFS-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066047#comment-13066047 ] Todd Lipcon commented on HDFS-2080: --- Well, not directly, which means that when we call the codecs, they end up copying in/out of direct buffers when calling the native functions. Speed up DFS read path by lessening checksum overhead - Key: HDFS-2080 URL: https://issues.apache.org/jira/browse/HDFS-2080 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.0 Attachments: hdfs-2080.txt, hdfs-2080.txt I've developed a series of patches that speeds up the HDFS read path by a factor of about 2.5x (~300M/sec to ~800M/sec for localhost reading from buffer cache) and also will make it easier to allow for advanced users (eg hbase) to skip a buffer copy. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports
[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066071#comment-13066071 ] Suresh Srinivas commented on HDFS-395: -- But for an excessive replica, it remains in the block map and excessiveBlockMap until an ack is back. They are the ones that need explicit acknowledgment. I know that for deleted files, when a previously deleted replica is reported by datanode to namenode, NN can again delete the replicas because the file does not exist. But I wonder why we do not remove excess replica also from the map on scheduling deletion. However, this could come very handy in HA implementation. Currently all namespace operations goes to standby through editlog. However having the delete acks creates a channel to report block deletions also to standby. So I am +1 on delete acks from the perspective of HA. Directory scanner should use the mechanism in this jira to send difference between in memory block map and the disk. This could be done in another jira. DFS Scalability: Incremental block reports -- Key: HDFS-395 URL: https://issues.apache.org/jira/browse/HDFS-395 Project: Hadoop HDFS Issue Type: Sub-task Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: blockReportPeriod.patch, explicitDeleteAcks.patch I have a cluster that has 1800 datanodes. Each datanode has around 5 blocks and sends a block report to the namenode once every hour. This means that the namenode processes a block report once every 2 seconds. Each block report contains all blocks that the datanode currently hosts. This makes the namenode compare a huge number of blocks that practically remains the same between two consecutive reports. This wastes CPU on the namenode. The problem becomes worse when the number of datanodes increases. One proposal is to make succeeding block reports (after a successful send of a full block report) be incremental. This will make the namenode process only those blocks that were added/deleted in the last period. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2154) TestDFSShell should use test dir
[
https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066089#comment-13066089
]
Aaron T. Myers commented on HDFS-2154:
--
Hey Nicholas, two small nits:
# Could you please camel case the variable testdir ?
# To form {{testdir}}, could you please use the two-argument constructor of
{{java.io.File}}, rather than string concatenation?
TestDFSShell should use test dir
Key: HDFS-2154
URL: https://issues.apache.org/jira/browse/HDFS-2154
Project: Hadoop HDFS
Issue Type: Test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Fix For: 0.23.0
Attachments: h2154_20110715.patch
The new test by HDFS-2131 creates files/directories under currently directory.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2156) rpm should only require the same major version as common
rpm should only require the same major version as common Key: HDFS-2156 URL: https://issues.apache.org/jira/browse/HDFS-2156 Project: Hadoop HDFS Issue Type: Bug Reporter: Owen O'Malley The rpm for hdfs should only require the same major version (eg. 0.23) of common. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Affects Version/s: 0.23.0 TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066100#comment-13066100 ] Aaron T. Myers commented on HDFS-2152: -- Thanks a lot, Uma. Latest patch looks good. Hudson seems to be down at the moment, but I'll commit it pending those results. TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2154) TestDFSShell should use test dir
[
https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066101#comment-13066101
]
Uma Maheswara Rao G commented on HDFS-2154:
---
{quote}
2. To form testdir, could you please use the two-argument constructor of
java.io.File, rather than string concatenation?
{quote}
Here testdir is used to create Path object, not java.io.File :-(. Also
testdir used in many places in down the line.
To get the absolute path from Path again, we may need to use toString on Path.
I feel two string concatenation is better than toString construction on Path.
TestDFSShell should use test dir
Key: HDFS-2154
URL: https://issues.apache.org/jira/browse/HDFS-2154
Project: Hadoop HDFS
Issue Type: Test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Fix For: 0.23.0
Attachments: h2154_20110715.patch
The new test by HDFS-2131 creates files/directories under currently directory.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1774) Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class.
[ https://issues.apache.org/jira/browse/HDFS-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1774: -- Fix Version/s: 0.20.1 0.23.0 Small optimization in org.apache.hadoop.hdfs.server.datanode.FSDataset class. - Key: HDFS-1774 URL: https://issues.apache.org/jira/browse/HDFS-1774 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.20.1, 0.23.0 Attachments: HDFS-1774-1.patch, HDFS-1774.patch Inner class FSDir constructor is doing duplicate iterations over the listed files in the passed directory. We can optimize this to single loop and also we can avoid isDirectory check which will perform some native invocations. Consider a case: one directory has only one child directory and 1 files. 1) First loop will get the number of children directories. 2) if (numChildren 0) , This condition will satisfy and again it will iterate 10001 times and also will check isDirectory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2047) 1073: Improve TestNamespace and TestEditLog in 1073 branch
[ https://issues.apache.org/jira/browse/HDFS-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2047: - Summary: 1073: Improve TestNamespace and TestEditLog in 1073 branch (was: Improve TestNamespace and TestEditLog in 1073 branch) Fixed bug title for consistency with other HDFS-1073 sub-tasks. 1073: Improve TestNamespace and TestEditLog in 1073 branch -- Key: HDFS-2047 URL: https://issues.apache.org/jira/browse/HDFS-2047 Project: Hadoop HDFS Issue Type: Sub-task Components: test Affects Versions: Edit log branch (HDFS-1073) Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: Edit log branch (HDFS-1073) Attachments: hdfs-2047.txt These tests currently have some test cases that don't make sense after HDFS-1073. This JIRA is to update these tests to do the equivalent things on 1073. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2102) 1073 Zero pad edits filename to make them lexically sortable
[ https://issues.apache.org/jira/browse/HDFS-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2102: - Issue Type: Sub-task (was: Improvement) Parent: HDFS-1073 1073 Zero pad edits filename to make them lexically sortable Key: HDFS-2102 URL: https://issues.apache.org/jira/browse/HDFS-2102 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: Edit log branch (HDFS-1073) Attachments: HDFS-2102.diff, hdfs-2102.txt Zero pad the edit log filenames so they appear in the correct order when you ls on the filesystem. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2102) 1073: Zero pad edits filename to make them lexically sortable
[ https://issues.apache.org/jira/browse/HDFS-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2102: - Summary: 1073: Zero pad edits filename to make them lexically sortable (was: 1073 Zero pad edits filename to make them lexically sortable) Done in 1073 branch, so converted to sub-task fo HDFS-1073 for ease of tracking. Fixed bug title for consistency with other HDFS-1073 sub-tasks. 1073: Zero pad edits filename to make them lexically sortable - Key: HDFS-2102 URL: https://issues.apache.org/jira/browse/HDFS-2102 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: Edit log branch (HDFS-1073) Attachments: HDFS-2102.diff, hdfs-2102.txt Zero pad the edit log filenames so they appear in the correct order when you ls on the filesystem. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2157) Improve header comment in o.a.h.hdfs.server.namenode.NameNode
[
https://issues.apache.org/jira/browse/HDFS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2157:
-
Attachment: hdfs-2157.0.patch
Patch to clean up the comment.
Improve header comment in o.a.h.hdfs.server.namenode.NameNode
-
Key: HDFS-2157
URL: https://issues.apache.org/jira/browse/HDFS-2157
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Labels: newbie
Fix For: 0.23.0
Attachments: hdfs-2157.0.patch
A developer new to HDFS pointed out to me that the header comment at the top
of {{NameNode.java}} is a little out of date/inaccurate.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2157) Improve header comment in o.a.h.hdfs.server.namenode.NameNode
[
https://issues.apache.org/jira/browse/HDFS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2157:
-
Status: Patch Available (was: Open)
Improve header comment in o.a.h.hdfs.server.namenode.NameNode
-
Key: HDFS-2157
URL: https://issues.apache.org/jira/browse/HDFS-2157
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Labels: newbie
Fix For: 0.23.0
Attachments: hdfs-2157.0.patch
A developer new to HDFS pointed out to me that the header comment at the top
of {{NameNode.java}} is a little out of date/inaccurate.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2157) Improve header comment in o.a.h.hdfs.server.namenode.NameNode
Improve header comment in o.a.h.hdfs.server.namenode.NameNode
-
Key: HDFS-2157
URL: https://issues.apache.org/jira/browse/HDFS-2157
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Fix For: 0.23.0
Attachments: hdfs-2157.0.patch
A developer new to HDFS pointed out to me that the header comment at the top of
{{NameNode.java}} is a little out of date/inaccurate.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2157) Improve header comment in o.a.h.hdfs.server.namenode.NameNode
[
https://issues.apache.org/jira/browse/HDFS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066129#comment-13066129
]
Eli Collins commented on HDFS-2157:
---
+1
Nits, feel free to address w/o another patch.
* s/partial namenode's state/partial namenode state/
* Let's be consistent wrt namenode vs NameNode, this uses both
Improve header comment in o.a.h.hdfs.server.namenode.NameNode
-
Key: HDFS-2157
URL: https://issues.apache.org/jira/browse/HDFS-2157
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Labels: newbie
Fix For: 0.23.0
Attachments: hdfs-2157.0.patch
A developer new to HDFS pointed out to me that the header comment at the top
of {{NameNode.java}} is a little out of date/inaccurate.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov updated HDFS-2116: --- Attachment: HDFS-2116.patch Had to get myself more familiar with Mockito, but I think this patch should do the trick. Tested it on my end and it passed. May still need some improvements as it is slightly hack-ish. Anyway, here is the patch for review! Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov updated HDFS-2116: --- Status: Patch Available (was: Open) Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2158) Add JournalSet to manage the set of journals.
Add JournalSet to manage the set of journals. - Key: HDFS-2158 URL: https://issues.apache.org/jira/browse/HDFS-2158 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.23.0 The management of the collection of journals can be encapsulated in a JournalSet. This will cleanup the FSEditLog code significantly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2018) Move all journal stream management code into one place
[ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2018: - Issue Type: Sub-task (was: Improvement) Parent: HDFS-1073 Move all journal stream management code into one place -- Key: HDFS-2018 URL: https://issues.apache.org/jira/browse/HDFS-2018 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: Edit log branch (HDFS-1073) Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff Currently in the HDFS-1073 branch, the code for creating output streams is in FileJournalManager and the code for input streams is in the inspectors. This change does a number of things. - Input and Output streams are now created by the JournalManager. - FSImageStorageInspectors now deals with URIs when referring to edit logs - Recovery of inprogress logs is performed by counting the number of transactions instead of looking at the length of the file. The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2018) 1073: Move all journal stream management code into one place
[ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2018: - Summary: 1073: Move all journal stream management code into one place (was: Move all journal stream management code into one place) 1073: Move all journal stream management code into one place Key: HDFS-2018 URL: https://issues.apache.org/jira/browse/HDFS-2018 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: Edit log branch (HDFS-1073) Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff Currently in the HDFS-1073 branch, the code for creating output streams is in FileJournalManager and the code for input streams is in the inspectors. This change does a number of things. - Input and Output streams are now created by the JournalManager. - FSImageStorageInspectors now deals with URIs when referring to edit logs - Recovery of inprogress logs is performed by counting the number of transactions instead of looking at the length of the file. The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[
https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066156#comment-13066156
]
Uma Maheswara Rao G commented on HDFS-2116:
---
1. What is the purpose of this assertion?
{code}
assertNull(Seek to 101 should not result in another request, null);
{code}
2.looks your coding formatter is not correct. Hadoop intendation is two spaces.
3.Patch contains hard core tabs, can you pleasse remove them?
4.{code}
} catch (Exception e) {
+ System.out.println(e.getMessage());
+ }
{code}
dont catch the generic exceptions.
5. when i try to apply patch, looks there are some conflicts with the trunk.
can you please update properly?
perhaps you can try applying the parch.
6.what about TestStreamFile cleanup?
Cleanup TestStreamFile and TestByteRangeInputStream
Key: HDFS-2116
URL: https://issues.apache.org/jira/browse/HDFS-2116
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Reporter: Eli Collins
Priority: Minor
Labels: newbie
Attachments: HDFS-2116.patch
TestStreamFile and TestByteRangeInputStream should use mockito. This would
allow the private URLOpener class to be removed from ByteRangeInputStream.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066157#comment-13066157 ] Uma Maheswara Rao G commented on HDFS-2116: --- Hi Plamen, I quickly gone through your patch. Please find my comments above. Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2157) Improve header comment in o.a.h.hdfs.server.namenode.NameNode
[
https://issues.apache.org/jira/browse/HDFS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2157:
-
Attachment: hdfs-2157.1.patch
Thanks a lot for the review, Eli. Here's a patch addressing your comments.
Improve header comment in o.a.h.hdfs.server.namenode.NameNode
-
Key: HDFS-2157
URL: https://issues.apache.org/jira/browse/HDFS-2157
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Labels: newbie
Fix For: 0.23.0
Attachments: hdfs-2157.0.patch, hdfs-2157.1.patch
A developer new to HDFS pointed out to me that the header comment at the top
of {{NameNode.java}} is a little out of date/inaccurate.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066176#comment-13066176 ] Hadoop QA commented on HDFS-2116: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486660/HDFS-2116.patch against trunk revision 1147184. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/947//console This message is automatically generated. Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066190#comment-13066190 ] Hudson commented on HDFS-2154: -- Integrated in Hadoop-Hdfs-trunk-Commit #787 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/787/]) HDFS-2154. In TestDFSShell, use TEST_ROOT_DIR and fix some deprecated warnings. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1147184 Files : * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java * /hadoop/common/trunk/hdfs/CHANGES.txt TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Fix For: 0.23.0 Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2143) dfsclusterhealth: we should link to the live nodes and dead nodes
[ https://issues.apache.org/jira/browse/HDFS-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066193#comment-13066193 ] Suresh Srinivas commented on HDFS-2143: --- Can you please post the web page with this change? dfsclusterhealth: we should link to the live nodes and dead nodes - Key: HDFS-2143 URL: https://issues.apache.org/jira/browse/HDFS-2143 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.22.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-2143.1.patch The dfsclusterhealth page shows the number of live and dead nodes. It would be nice to link those numbers to the page containing the list of those nodes -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2143) dfsclusterhealth: we should link to the live nodes and dead nodes
[
https://issues.apache.org/jira/browse/HDFS-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066197#comment-13066197
]
Suresh Srinivas commented on HDFS-2143:
---
Nevermind, looking at the code you do not need that change. While reviewing I
notices that toXmlItemBlockWithLink() arguments has incorrect names. Could you
change the parameter name {{host}} to {{value}}, and {{nodetag}} to {{label}}.
dfsclusterhealth: we should link to the live nodes and dead nodes
-
Key: HDFS-2143
URL: https://issues.apache.org/jira/browse/HDFS-2143
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 0.22.0
Reporter: Ravi Prakash
Assignee: Ravi Prakash
Attachments: HDFS-2143.1.patch
The dfsclusterhealth page shows the number of live and dead nodes. It would
be nice to link those numbers to the page containing the list of those nodes
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2156) rpm should only require the same major version as common
[ https://issues.apache.org/jira/browse/HDFS-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated HDFS-2156: Attachment: HDFS-2156.patch This patch applies to both hdfs and mapreduce dependency mappings. Dependency mapping is described as x.y.0 = current = x.y., where x.y is the compatible version. rpm should only require the same major version as common Key: HDFS-2156 URL: https://issues.apache.org/jira/browse/HDFS-2156 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0 Reporter: Owen O'Malley Fix For: 0.23.0 Attachments: HDFS-2156.patch The rpm for hdfs should only require the same major version (eg. 0.23) of common. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2156) rpm should only require the same major version as common
[ https://issues.apache.org/jira/browse/HDFS-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated HDFS-2156: Fix Version/s: 0.23.0 Assignee: Eric Yang Affects Version/s: 0.23.0 Status: Patch Available (was: Open) rpm should only require the same major version as common Key: HDFS-2156 URL: https://issues.apache.org/jira/browse/HDFS-2156 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0 Reporter: Owen O'Malley Assignee: Eric Yang Fix For: 0.23.0 Attachments: HDFS-2156.patch The rpm for hdfs should only require the same major version (eg. 0.23) of common. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2156) rpm should only require the same major version as common
[
https://issues.apache.org/jira/browse/HDFS-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066209#comment-13066209
]
Eric Yang commented on HDFS-2156:
-
Correction to my comment, the range mapping should be the invert:
{noformat}
x.y0 = current = x.y.
{noformat}
rpm should only require the same major version as common
Key: HDFS-2156
URL: https://issues.apache.org/jira/browse/HDFS-2156
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.23.0
Reporter: Owen O'Malley
Assignee: Eric Yang
Fix For: 0.23.0
Attachments: HDFS-2156.patch
The rpm for hdfs should only require the same major version (eg. 0.23) of
common.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2156) rpm should only require the same major version as common
[
https://issues.apache.org/jira/browse/HDFS-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066211#comment-13066211
]
Eric Yang commented on HDFS-2156:
-
Correction to my comment:
{noformat}
x.y.0 = current = x.y.
{noformat}
rpm should only require the same major version as common
Key: HDFS-2156
URL: https://issues.apache.org/jira/browse/HDFS-2156
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.23.0
Reporter: Owen O'Malley
Assignee: Eric Yang
Fix For: 0.23.0
Attachments: HDFS-2156.patch
The rpm for hdfs should only require the same major version (eg. 0.23) of
common.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066214#comment-13066214 ] Plamen Jeliazkov commented on HDFS-2116: Thanks I will definitely get on that. Yeah, that assertion was not part of my code though -- I believe it was there prior to my patch as well. Same thing goes with the generic exception catching, they were there prior to my patch. I think the issue here is that I am working from the 0.22.0 branch; where is this is updating straight to trunk. Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2141) Remove NameNode roles Active and Standby
[
https://issues.apache.org/jira/browse/HDFS-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066249#comment-13066249
]
Aaron T. Myers commented on HDFS-2141:
--
+1, this patch looks good to me. If you care to, you might want to line up the
indentation of (NameNode) with the other members of {{NameNodeRole}}, but no
reason to make another patch for it.
We might even consider merging {{HdfsConstants.NodeType}} and
{{HdfsConstants.NameNodeRole}}, since being a DN is certainly mutually
exclusive with all of the {{NameNodeRoles}}, but that should probably be done
as a separate JIRA.
Remove NameNode roles Active and Standby
Key: HDFS-2141
URL: https://issues.apache.org/jira/browse/HDFS-2141
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Reporter: Suresh Srinivas
Assignee: Suresh Srinivas
Attachments: HDFS-2141.patch
In HDFS, following roles are supported in NameNodeRole: ACTIVE, BACKUP,
CHECKPOINT and STANDBY.
Active and Standby are the state of the NameNode. While Backup and CheckPoint
are the name/role of the daemons that are started. This mixes up the run time
state of NameNode with the daemon role. I propose changing the NameNodeRole
to: NAMENODE, BACKUP, CHECKPOINT. HDFS-1974 will introduce the states active
and standby to the daemon that is running in the role NAMENODE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2141) Remove NameNode roles Active and Standby
[ https://issues.apache.org/jira/browse/HDFS-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066255#comment-13066255 ] Aaron T. Myers commented on HDFS-2141: -- Also, I see no reason not to commit this to trunk. There's nothing HA-specific in this change - it just removes a vestigial field and renames another to be more accurate of what it actually represents. Both are used internally so there should be no compatibility concerns. Remove NameNode roles Active and Standby Key: HDFS-2141 URL: https://issues.apache.org/jira/browse/HDFS-2141 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2141.patch In HDFS, following roles are supported in NameNodeRole: ACTIVE, BACKUP, CHECKPOINT and STANDBY. Active and Standby are the state of the NameNode. While Backup and CheckPoint are the name/role of the daemons that are started. This mixes up the run time state of NameNode with the daemon role. I propose changing the NameNodeRole to: NAMENODE, BACKUP, CHECKPOINT. HDFS-1974 will introduce the states active and standby to the daemon that is running in the role NAMENODE. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2154) TestDFSShell should use test dir
[ https://issues.apache.org/jira/browse/HDFS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2154: - Component/s: test Hi Aaron, thanks for the comments. Since the code is in a test, I won't update it for the nits but I will beware them for my future code. TestDFSShell should use test dir Key: HDFS-2154 URL: https://issues.apache.org/jira/browse/HDFS-2154 Project: Hadoop HDFS Issue Type: Test Components: test Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Fix For: 0.23.0 Attachments: h2154_20110715.patch The new test by HDFS-2131 creates files/directories under currently directory. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066276#comment-13066276
]
Tsz Wo (Nicholas), SZE commented on HDFS-2153:
--
Todd, thanks for reviewing it. I also have fixed some javac warnings in
OfflineEditsViewerHelper, which is the class using DFSClientAdapter.
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2153:
-
Resolution: Fixed
Fix Version/s: 0.23.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I have committed this.
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Fix For: 0.23.0
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2159) Deprecate DistributedFileSystem.getClient()
Deprecate DistributedFileSystem.getClient() --- Key: HDFS-2159 URL: https://issues.apache.org/jira/browse/HDFS-2159 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE The DFSClient in DistributedFileSystem should not be accessed directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov updated HDFS-2116: --- Attachment: HDFS-2116.patch Went and grabbed the trunk revision and worked from there. Followed comments -- editted TestStreamFile.java as well; I could not figure out an appropiate way to use Mockito with FSInputStream. Removed hard coded tabs -- space indenting is by 2. Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch, HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2153) DFSClientAdapter should be put under test
[
https://issues.apache.org/jira/browse/HDFS-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066294#comment-13066294
]
Hudson commented on HDFS-2153:
--
Integrated in Hadoop-Hdfs-trunk-Commit #788 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/788/])
HDFS-2153. Move DFSClientAdapter to test and fix some javac warnings in
OfflineEditsViewerHelper.
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1147340
Files :
*
/hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/DFSClientAdapter.java
* /hadoop/common/trunk/hdfs/CHANGES.txt
*
/hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/OfflineEditsViewerHelper.java
*
/hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DFSClientAdapter.java
DFSClientAdapter should be put under test
-
Key: HDFS-2153
URL: https://issues.apache.org/jira/browse/HDFS-2153
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Priority: Minor
Fix For: 0.23.0
Attachments: h2153_20110715.patch
{{DFSClientAdapter}} is a test utility but it is put in src/java.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2159) Deprecate DistributedFileSystem.getClient()
[
https://issues.apache.org/jira/browse/HDFS-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2159:
-
Attachment: h2159_20110715.patch
h2159_20110715.patch: deprecated the method and fixed some deprecated warnings
in {{DFSAdmin}}.
Deprecate DistributedFileSystem.getClient()
---
Key: HDFS-2159
URL: https://issues.apache.org/jira/browse/HDFS-2159
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs client
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Attachments: h2159_20110715.patch
The DFSClient in DistributedFileSystem should not be accessed directly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2159) Deprecate DistributedFileSystem.getClient()
[ https://issues.apache.org/jira/browse/HDFS-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2159: - Status: Patch Available (was: Open) Deprecate DistributedFileSystem.getClient() --- Key: HDFS-2159 URL: https://issues.apache.org/jira/browse/HDFS-2159 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2159_20110715.patch The DFSClient in DistributedFileSystem should not be accessed directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2157) Improve header comment in o.a.h.hdfs.server.namenode.NameNode
[
https://issues.apache.org/jira/browse/HDFS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2157:
-
Component/s: documentation
Improve header comment in o.a.h.hdfs.server.namenode.NameNode
-
Key: HDFS-2157
URL: https://issues.apache.org/jira/browse/HDFS-2157
Project: Hadoop HDFS
Issue Type: Improvement
Components: documentation, name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Labels: newbie
Fix For: 0.23.0
Attachments: hdfs-2157.0.patch, hdfs-2157.1.patch
A developer new to HDFS pointed out to me that the header comment at the top
of {{NameNode.java}} is a little out of date/inaccurate.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2116) Cleanup TestStreamFile and TestByteRangeInputStream
[ https://issues.apache.org/jira/browse/HDFS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066301#comment-13066301 ] Hadoop QA commented on HDFS-2116: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486697/HDFS-2116.patch against trunk revision 1147340. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/950//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/950//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/950//console This message is automatically generated. Cleanup TestStreamFile and TestByteRangeInputStream Key: HDFS-2116 URL: https://issues.apache.org/jira/browse/HDFS-2116 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Eli Collins Priority: Minor Labels: newbie Attachments: HDFS-2116.patch, HDFS-2116.patch TestStreamFile and TestByteRangeInputStream should use mockito. This would allow the private URLOpener class to be removed from ByteRangeInputStream. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2159) Deprecate DistributedFileSystem.getClient()
[ https://issues.apache.org/jira/browse/HDFS-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066313#comment-13066313 ] Aaron T. Myers commented on HDFS-2159: -- Hey Nicholas, patch looks good to me. +1 pending clean Hudson results. Deprecate DistributedFileSystem.getClient() --- Key: HDFS-2159 URL: https://issues.apache.org/jira/browse/HDFS-2159 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2159_20110715.patch The DFSClient in DistributedFileSystem should not be accessed directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2159) Deprecate DistributedFileSystem.getClient()
[ https://issues.apache.org/jira/browse/HDFS-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066328#comment-13066328 ] Hadoop QA commented on HDFS-2159: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486698/h2159_20110715.patch against trunk revision 1147340. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/951//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/951//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/951//console This message is automatically generated. Deprecate DistributedFileSystem.getClient() --- Key: HDFS-2159 URL: https://issues.apache.org/jira/browse/HDFS-2159 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2159_20110715.patch The DFSClient in DistributedFileSystem should not be accessed directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
[ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066338#comment-13066338 ] Jitendra Nath Pandey commented on HDFS-1073: Ivan has opened HDFS-2149 - I'd propose we do that under that JIRA? Sounds good. A few more minor comments: TestEditLog.java#testSimpleEditLog: Exception in the cluster.shutdown is being ignored. TestEditLog.java#testFailedOpen is disabled. TestBackupNode.java : waitCheckpointDone does nothing. Commented out code in a few places with TODOs. Simpler model for Namenode's fs Image and edit Logs Key: HDFS-1073 URL: https://issues.apache.org/jira/browse/HDFS-1073 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Assignee: Todd Lipcon Attachments: hdfs-1073-editloading-algos.txt, hdfs-1073.txt, hdfs1073.pdf, hdfs1073.pdf, hdfs1073.pdf, hdfs1073.tex The naming and handling of NN's fsImage and edit logs can be significantly improved resulting simpler and more robust code. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
[ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066352#comment-13066352 ] Todd Lipcon commented on HDFS-1073: --- good catches in TestEditLog. Are you sure you're looking at the latest version of the branch, regarding TestBackupNode? That function was filled in by HDFS-1979 which I committed yesterday morning I believe. I'll do another sweep for TODOs I might have missed. Simpler model for Namenode's fs Image and edit Logs Key: HDFS-1073 URL: https://issues.apache.org/jira/browse/HDFS-1073 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Assignee: Todd Lipcon Attachments: hdfs-1073-editloading-algos.txt, hdfs-1073.txt, hdfs1073.pdf, hdfs1073.pdf, hdfs1073.pdf, hdfs1073.tex The naming and handling of NN's fsImage and edit logs can be significantly improved resulting simpler and more robust code. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2160) 1073: fix CreateEditsLog test tool in branch
[ https://issues.apache.org/jira/browse/HDFS-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2160: -- Component/s: test Description: This was commented out earlier in the branch. Needs trivial fixing. Affects Version/s: Edit log branch (HDFS-1073) Fix Version/s: Edit log branch (HDFS-1073) 1073: fix CreateEditsLog test tool in branch Key: HDFS-2160 URL: https://issues.apache.org/jira/browse/HDFS-2160 Project: Hadoop HDFS Issue Type: Sub-task Components: test Affects Versions: Edit log branch (HDFS-1073) Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: Edit log branch (HDFS-1073) This was commented out earlier in the branch. Needs trivial fixing. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2160) 1073: fix CreateEditsLog test tool in branch
1073: fix CreateEditsLog test tool in branch Key: HDFS-2160 URL: https://issues.apache.org/jira/browse/HDFS-2160 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Todd Lipcon Assignee: Todd Lipcon -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2160) 1073: fix CreateEditsLog test tool in branch
[
https://issues.apache.org/jira/browse/HDFS-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-2160:
--
Attachment: hdfs-2160.txt
Simple patch to fix this.
I tested using the following command line:
{noformat}
HADOOP_CLASSPATH=build/test/hdfs/classes:build/ivy/lib/hadoop-hdfs/test/mockito-all-1.8.2.jar
./bin/hdfs org.apache.hadoop.hdfs.server.namenode.CreateEditsLog -f 1000 1 1
{noformat}
and then used the oev to verify the resulting edits file.
Note the change to bin/hdfs to be able to run an arbitrary class with HDFS on
the classpath.
1073: fix CreateEditsLog test tool in branch
Key: HDFS-2160
URL: https://issues.apache.org/jira/browse/HDFS-2160
Project: Hadoop HDFS
Issue Type: Sub-task
Components: test
Affects Versions: Edit log branch (HDFS-1073)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Fix For: Edit log branch (HDFS-1073)
Attachments: hdfs-2160.txt
This was commented out earlier in the branch. Needs trivial fixing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2159) Deprecate DistributedFileSystem.getClient()
[ https://issues.apache.org/jira/browse/HDFS-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066363#comment-13066363 ] Hudson commented on HDFS-2159: -- Integrated in Hadoop-Hdfs-trunk-Commit #789 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/789/]) HDFS-2159. Deprecate DistributedFileSystem.getClient() and fixed the deprecated warnings in DFSAdmin. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1147359 Files : * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestBlockUnderConstruction.java * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/DFSClientAdapter.java * /hadoop/common/trunk/hdfs/CHANGES.txt * /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/OfflineEditsViewerHelper.java * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/datanode/TestTransferRbw.java * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestAbandonBlock.java * /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java * /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestLeaseRecovery.java Deprecate DistributedFileSystem.getClient() --- Key: HDFS-2159 URL: https://issues.apache.org/jira/browse/HDFS-2159 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.0 Attachments: h2159_20110715.patch The DFSClient in DistributedFileSystem should not be accessed directly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Attachment: HDFS-2152-1.patch Re Submitted the same patch to trigger Hudson. TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Status: Patch Available (was: Open) TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2152) TestWriteConfigurationToDFS causing the random failures
[ https://issues.apache.org/jira/browse/HDFS-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2152: -- Status: Open (was: Patch Available) TestWriteConfigurationToDFS causing the random failures --- Key: HDFS-2152 URL: https://issues.apache.org/jira/browse/HDFS-2152 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2152-1.patch, HDFS-2152-1.patch, HDFS-2152.patch -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
