[jira] [Updated] (HDFS-1975) HA: Support for sharing the namenode state from active to standby.
[ https://issues.apache.org/jira/browse/HDFS-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1975: - Attachment: HDFS-1975-HDFS-1623.patch Thanks a ton for your review, Todd. Here's an updated patch which addresses all of your comments except the following. bq. In EditLogFileInputStream, why do we need to pass isInProgress as a boolean? If it's in-progress, that means we don't know the lastTxId, so it would be INVALID_TXID, right? So we can implement isInProgress() by just comparing lastTxId. Because in the case of pre-transactional ELFIS, we pass in INVALID_TXID, even though the log is not yet in progress. I'm still working on producing an updated patch which also adds testing for the pending DN message queues. HA: Support for sharing the namenode state from active to standby. -- Key: HDFS-1975 URL: https://issues.apache.org/jira/browse/HDFS-1975 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Suresh Srinivas Assignee: Jitendra Nath Pandey Attachments: HDFS-1975-HA.2.patch, HDFS-1975-HA.patch, HDFS-1975-HDFS-1623.patch, HDFS-1975-HDFS-1623.patch, hdfs-1975.txt, hdfs-1975.txt To enable hot standby namenode, the standby node must have current information for - namenode state (image + edits) and block location information. This jira addresses keeping the namenode state current in the standby node. To do this, the proposed solution in this jira is to use a shared storage to store the namenode state. Note one could also build an alternative solution by augmenting the backup node. A seperate jira could explore this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2562: -- Resolution: Fixed Fix Version/s: 0.23.1 0.24.0 Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 23 (for 23.1). Thanks for the quick review. Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2559) Clean up Storage/StorageInfo classes after federation
[ https://issues.apache.org/jira/browse/HDFS-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152732#comment-13152732 ] Aaron T. Myers commented on HDFS-2559: -- +1, the patch looks good to me. Clean up Storage/StorageInfo classes after federation - Key: HDFS-2559 URL: https://issues.apache.org/jira/browse/HDFS-2559 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2559.txt After federation, there are many different identifiers floating around in a cluster: cluster ID, namespace ID, block pool ID, plus the versioning identifiers cTime and layoutVersion. Some of these are cluster-wide, whereas others may be different within a DN as it talks to different NNs (see the federation design doc for more info). However, the current class heirarchy in trunk has DataStorage (the DN-wide storage class) inherit from Storage which inherits from StorageInfo, which contains a namespaceID. Thus, DataStorage has this unused member which confuses things. This JIRA is to split up StorageInfo into separate classes/interfaces so that only the necessary identifiers are used by each piece of the DN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2563) Some cleanup in BPOfferService
Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2563) Some cleanup in BPOfferService
[
https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-2563:
--
Attachment: hdfs-2563.txt
Here is a prelim patch (haven't run through all the unit tests locally yet)...
we'll see what Hudson has to say.
Summary of the changes:
- Add javadoc for bpRegistration, bpNSInfo fields to clarify when they get set
- Remove blockPoolId member and replace with an accessor, since this is already
stored as part of bpNSInfo
- Don't assign bpRegistration until the block pool is actually determined
during the handshake process. Change use of {{bpRegistration}} in log messages
to use {{this.toString()}} which now has a sensible output. We used to see log
messages like:
{code}
2011-11-18 01:13:24,244 INFO datanode.DataNode (DataNode.java:run(1195)) -
DatanodeRegistration(127.0.0.1:56326, storageID=, infoPort=39362,
ipcPort=38231, storageInfo=lv=0;cid=;nsid=0;c=0)In BPOfferService.run, data =
null;bp=null
{code}
which looks like an error to the user. Now it will say something like:
{code}
2011-11-18 01:19:08,209 INFO datanode.DataNode (DataNode.java:run(1238)) -
Block pool registering (storage id unknown) connecting to
todd-w510/127.0.0.1:9930 starting to offer service
{code}
and later
{code}
2011-11-18 01:19:08,395 INFO datanode.DataNode (DataNode.java:register(1197))
- Block pool BP-428670582-127.0.0.1-1321607947917 (storage id
DS-329594984-127.0.0.1-49361-1321607948276) registered with
todd-w510/127.0.0.1:9930 beginning handshake with NN at todd-w510/127.0.0.1:9930
{code}
when the blockPoolId and storageId become known
- Split the handshake method into two clearer parts: (1)
{{retrieveNamespaceInfo}} which calls {{versionRequest}} to determine the NN's
version info and namespace/cluster ID, and (2) {{register()}} like before,
which actually registers the DN.
- Refactor the code to check layout versions (used to be duplicated in two
places) into a {{checkNNVersion}} class.
- Some other trivial improvements of log messages, etc, to be more user
readable.
- Removed unused test function {{DataNodeTestUtils.setBPNamenodeByIndex}}.
Fixed {{TestDatanodeRegister}} to correspond to above changes.
Some cleanup in BPOfferService
--
Key: HDFS-2563
URL: https://issues.apache.org/jira/browse/HDFS-2563
Project: Hadoop HDFS
Issue Type: Improvement
Components: data-node
Affects Versions: 0.24.0, 0.23.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Attachments: hdfs-2563.txt
BPOfferService is currently rather difficult to follow and not really
commented. This JIRA is to clean up the code a bit, add javadocs/comments
where necessary, and improve the formatting of the log messages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2563: -- Status: Patch Available (was: Open) Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152747#comment-13152747 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Hdfs-trunk-Commit #1358 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1358/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203543 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152748#comment-13152748 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Hdfs-0.23-Commit #181 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/181/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203542 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152751#comment-13152751 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Mapreduce-0.23-Commit #194 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/194/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203542 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152750#comment-13152750 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1309 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1309/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203543 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152753#comment-13152753 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Common-0.23-Commit #182 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/182/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203542 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152755#comment-13152755 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Common-trunk-Commit #1284 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1284/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203543 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152806#comment-13152806 ] Hadoop QA commented on HDFS-2563: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504188/hdfs-2563.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes org.apache.hadoop.hdfs.TestDatanodeRegistration org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.TestDfsOverAvroRpc org.apache.hadoop.hdfs.TestDFSStartupVersions org.apache.hadoop.hdfs.TestDFSUpgrade org.apache.hadoop.hdfs.TestPread +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1568//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1568//console This message is automatically generated. Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152828#comment-13152828 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Hdfs-0.23-Build #80 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/80/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203542 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2560) Refactor BPOfferService to be a static inner class
[ https://issues.apache.org/jira/browse/HDFS-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152825#comment-13152825 ] Hudson commented on HDFS-2560: -- Integrated in Hadoop-Hdfs-0.23-Build #80 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/80/]) HDFS-2560. Refactor BPOfferService to be a static inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203442 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java Refactor BPOfferService to be a static inner class -- Key: HDFS-2560 URL: https://issues.apache.org/jira/browse/HDFS-2560 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2560-final.txt, hdfs-2560.txt, hdfs-2560.txt Currently BPOfferService is a non-static inner class of DataNode. For HA we are adding another inner class inside of this, which makes the scope very hard to understand when reading the code (and has resulted in subtle bugs like HDFS-2529 where a variable is referenced from the wrong scope. Making it a static inner class with a reference to the DN has two advantages: a) scope is now explicit, and b) enables unit testing of the BPOS against a mocked-out DN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2544) Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh.
[ https://issues.apache.org/jira/browse/HDFS-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152827#comment-13152827 ] Hudson commented on HDFS-2544: -- Integrated in Hadoop-Hdfs-0.23-Build #80 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/80/]) Merge -r 1203451:1203452 from trunk to branch-0.23. Fixes: HDFS-2544 tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203453 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/distribute-exclude.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/refresh-namenodes.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-balancer.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-secure-dns.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-balancer.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-secure-dns.sh Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh. - Key: HDFS-2544 URL: https://issues.apache.org/jira/browse/HDFS-2544 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2544.patch It would be nice to be able to specify some other location for hadoop-config.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2543) HADOOP_PREFIX cannot be overriden
[ https://issues.apache.org/jira/browse/HDFS-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152826#comment-13152826 ] Hudson commented on HDFS-2543: -- Integrated in Hadoop-Hdfs-0.23-Build #80 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/80/]) Merge -r 1203485:1203486 from trunk to branch-0.23. Fixes: HDFS-2543 tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203487 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/fuse_dfs_wrapper.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/init.d/hadoop-secondarynamenode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/init.d/hadoop-secondarynamenode HADOOP_PREFIX cannot be overriden - Key: HDFS-2543 URL: https://issues.apache.org/jira/browse/HDFS-2543 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2543.patch hadoop-config.sh forces HADOOP_prefix to a specific value: export HADOOP_PREFIX=`dirname $this`/.. It would be nice to make this overridable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152839#comment-13152839 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Hdfs-trunk #867 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/867/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203543 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2560) Refactor BPOfferService to be a static inner class
[ https://issues.apache.org/jira/browse/HDFS-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152836#comment-13152836 ] Hudson commented on HDFS-2560: -- Integrated in Hadoop-Hdfs-trunk #867 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/867/]) HDFS-2560. Refactor BPOfferService to be a static inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203444 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java Refactor BPOfferService to be a static inner class -- Key: HDFS-2560 URL: https://issues.apache.org/jira/browse/HDFS-2560 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2560-final.txt, hdfs-2560.txt, hdfs-2560.txt Currently BPOfferService is a non-static inner class of DataNode. For HA we are adding another inner class inside of this, which makes the scope very hard to understand when reading the code (and has resulted in subtle bugs like HDFS-2529 where a variable is referenced from the wrong scope. Making it a static inner class with a reference to the DN has two advantages: a) scope is now explicit, and b) enables unit testing of the BPOS against a mocked-out DN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2543) HADOOP_PREFIX cannot be overriden
[ https://issues.apache.org/jira/browse/HDFS-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152837#comment-13152837 ] Hudson commented on HDFS-2543: -- Integrated in Hadoop-Hdfs-trunk #867 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/867/]) HDFS-2543. HADOOP_PREFIX cannot be overridden. Contributed by Bruno Mahé. tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203486 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/fuse_dfs_wrapper.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/init.d/hadoop-secondarynamenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/init.d/hadoop-secondarynamenode HADOOP_PREFIX cannot be overriden - Key: HDFS-2543 URL: https://issues.apache.org/jira/browse/HDFS-2543 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2543.patch hadoop-config.sh forces HADOOP_prefix to a specific value: export HADOOP_PREFIX=`dirname $this`/.. It would be nice to make this overridable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2544) Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh.
[ https://issues.apache.org/jira/browse/HDFS-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152838#comment-13152838 ] Hudson commented on HDFS-2544: -- Integrated in Hadoop-Hdfs-trunk #867 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/867/]) HDFS-2544. Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh. Contributed by Bruno Mahé tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203452 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/distribute-exclude.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/refresh-namenodes.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-balancer.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-secure-dns.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-balancer.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-secure-dns.sh Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh. - Key: HDFS-2544 URL: https://issues.apache.org/jira/browse/HDFS-2544 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2544.patch It would be nice to be able to specify some other location for hadoop-config.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2560) Refactor BPOfferService to be a static inner class
[ https://issues.apache.org/jira/browse/HDFS-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152851#comment-13152851 ] Hudson commented on HDFS-2560: -- Integrated in Hadoop-Mapreduce-0.23-Build #97 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/97/]) HDFS-2560. Refactor BPOfferService to be a static inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203442 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java Refactor BPOfferService to be a static inner class -- Key: HDFS-2560 URL: https://issues.apache.org/jira/browse/HDFS-2560 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2560-final.txt, hdfs-2560.txt, hdfs-2560.txt Currently BPOfferService is a non-static inner class of DataNode. For HA we are adding another inner class inside of this, which makes the scope very hard to understand when reading the code (and has resulted in subtle bugs like HDFS-2529 where a variable is referenced from the wrong scope. Making it a static inner class with a reference to the DN has two advantages: a) scope is now explicit, and b) enables unit testing of the BPOS against a mocked-out DN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2543) HADOOP_PREFIX cannot be overriden
[ https://issues.apache.org/jira/browse/HDFS-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152852#comment-13152852 ] Hudson commented on HDFS-2543: -- Integrated in Hadoop-Mapreduce-0.23-Build #97 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/97/]) Merge -r 1203485:1203486 from trunk to branch-0.23. Fixes: HDFS-2543 tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203487 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/fuse_dfs_wrapper.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/init.d/hadoop-secondarynamenode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/init.d/hadoop-secondarynamenode HADOOP_PREFIX cannot be overriden - Key: HDFS-2543 URL: https://issues.apache.org/jira/browse/HDFS-2543 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2543.patch hadoop-config.sh forces HADOOP_prefix to a specific value: export HADOOP_PREFIX=`dirname $this`/.. It would be nice to make this overridable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152854#comment-13152854 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Mapreduce-0.23-Build #97 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/97/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203542 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2544) Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh.
[ https://issues.apache.org/jira/browse/HDFS-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152853#comment-13152853 ] Hudson commented on HDFS-2544: -- Integrated in Hadoop-Mapreduce-0.23-Build #97 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/97/]) Merge -r 1203451:1203452 from trunk to branch-0.23. Fixes: HDFS-2544 tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203453 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/distribute-exclude.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/refresh-namenodes.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-balancer.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-secure-dns.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-balancer.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-secure-dns.sh Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh. - Key: HDFS-2544 URL: https://issues.apache.org/jira/browse/HDFS-2544 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2544.patch It would be nice to be able to specify some other location for hadoop-config.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2562) Refactor DN configuration variables out of DataNode class
[ https://issues.apache.org/jira/browse/HDFS-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152870#comment-13152870 ] Hudson commented on HDFS-2562: -- Integrated in Hadoop-Mapreduce-trunk #901 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/901/]) HDFS-2562. Refactor DN configuration variables out of DataNode class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203543 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestInterDatanodeProtocol.java Refactor DN configuration variables out of DataNode class - Key: HDFS-2562 URL: https://issues.apache.org/jira/browse/HDFS-2562 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2562.txt Right now there are many member variables in DataNode.java which are just read from configuration when the DN is started. Similar to what we did with DFSClient, we should refactor them into a new DNConf class which can be passed around - the motivation is to remove the many references we have throughout the code that read package-protected members of DataNode and reduce the number of members in DataNode itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2560) Refactor BPOfferService to be a static inner class
[ https://issues.apache.org/jira/browse/HDFS-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152867#comment-13152867 ] Hudson commented on HDFS-2560: -- Integrated in Hadoop-Mapreduce-trunk #901 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/901/]) HDFS-2560. Refactor BPOfferService to be a static inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203444 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java Refactor BPOfferService to be a static inner class -- Key: HDFS-2560 URL: https://issues.apache.org/jira/browse/HDFS-2560 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2560-final.txt, hdfs-2560.txt, hdfs-2560.txt Currently BPOfferService is a non-static inner class of DataNode. For HA we are adding another inner class inside of this, which makes the scope very hard to understand when reading the code (and has resulted in subtle bugs like HDFS-2529 where a variable is referenced from the wrong scope. Making it a static inner class with a reference to the DN has two advantages: a) scope is now explicit, and b) enables unit testing of the BPOS against a mocked-out DN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2543) HADOOP_PREFIX cannot be overriden
[ https://issues.apache.org/jira/browse/HDFS-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152868#comment-13152868 ] Hudson commented on HDFS-2543: -- Integrated in Hadoop-Mapreduce-trunk #901 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/901/]) HDFS-2543. HADOOP_PREFIX cannot be overridden. Contributed by Bruno Mahé. tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203486 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/fuse_dfs_wrapper.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/deb/init.d/hadoop-secondarynamenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/rpm/init.d/hadoop-secondarynamenode HADOOP_PREFIX cannot be overriden - Key: HDFS-2543 URL: https://issues.apache.org/jira/browse/HDFS-2543 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2543.patch hadoop-config.sh forces HADOOP_prefix to a specific value: export HADOOP_PREFIX=`dirname $this`/.. It would be nice to make this overridable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2544) Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh.
[ https://issues.apache.org/jira/browse/HDFS-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13152869#comment-13152869 ] Hudson commented on HDFS-2544: -- Integrated in Hadoop-Mapreduce-trunk #901 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/901/]) HDFS-2544. Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh. Contributed by Bruno Mahé tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203452 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/distribute-exclude.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/refresh-namenodes.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-balancer.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-secure-dns.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-balancer.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-secure-dns.sh Hadoop scripts unconditionally source $bin/../libexec/hadoop-config.sh. - Key: HDFS-2544 URL: https://issues.apache.org/jira/browse/HDFS-2544 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2544.patch It would be nice to be able to specify some other location for hadoop-config.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Status: Patch Available (was: Open) Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: HDFS-2564.txt Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153030#comment-13153030 ] Hadoop QA commented on HDFS-2564: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504240/HDFS-2564.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1569//console This message is automatically generated. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2565) hdfs command does not explicitely complain that the command I am trying to run is not valid
hdfs command does not explicitely complain that the command I am trying to run is not valid --- Key: HDFS-2565 URL: https://issues.apache.org/jira/browse/HDFS-2565 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0, 0.23.1 Reporter: Bruno Mahé [root@bigtop-fedora-15 ~]# hdfs foobar Exception in thread main java.lang.NoClassDefFoundError: foobar Caused by: java.lang.ClassNotFoundException: foobar at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: foobar. Program will exit. Instead of loading any class, it would be nice to explain the command is not valid and to call print_usage() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude reassigned HDFS-2564: -- Assignee: Hari Mankude Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: HDFS-2564.txt Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt, HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153073#comment-13153073 ] Hadoop QA commented on HDFS-2564: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504252/HDFS-2564.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1570//console This message is automatically generated. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt, HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2563: -- Attachment: hdfs-2563.txt The previous version of the patch had an issue where the block pool wasn't getting registered in blockPoolManager early enough. New rev cleans this up, and the tests that failed seem to pass now locally. Also fixed a few of the log messages which were including the nnAddr twice (once as part of toString() and once in the log message) Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: HDFS-2564-1.txt Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, HDFS-2564.txt, HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153194#comment-13153194 ] Hadoop QA commented on HDFS-2564: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504279/HDFS-2564-1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1572//console This message is automatically generated. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, HDFS-2564.txt, HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: (was: HDFS-2564.txt) Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: (was: HDFS-2564.txt) Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153213#comment-13153213 ] Hadoop QA commented on HDFS-2563: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504269/hdfs-2563.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDfsOverAvroRpc +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1571//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1571//console This message is automatically generated. Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: hadoop-2564.trunk.patch Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153215#comment-13153215 ] Hari Mankude commented on HDFS-2564: Seperate out hadoop-common files into hdfs-7840 jira so that different projects can be applied. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153218#comment-13153218 ] Hadoop QA commented on HDFS-2564: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504288/hadoop-2564.trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1573//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1573//console This message is automatically generated. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: hadoop-2564.trunk.patch Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2517) Protobuf - Add protocol service for JournalProtocol
[ https://issues.apache.org/jira/browse/HDFS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153279#comment-13153279 ] Jitendra Nath Pandey commented on HDFS-2517: +1 Protobuf - Add protocol service for JournalProtocol --- Key: HDFS-2517 URL: https://issues.apache.org/jira/browse/HDFS-2517 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2517.txt, HDFS-2517.txt This jira adds protobuf service for implementing journal protocol using protocol buffers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153281#comment-13153281 ] Jitendra Nath Pandey commented on HDFS-2518: +1 Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153286#comment-13153286 ] Hadoop QA commented on HDFS-2564: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504297/hadoop-2564.trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDfsOverAvroRpc org.apache.hadoop.hdfs.TestFileAppend4 +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1574//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1574//console This message is automatically generated. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153290#comment-13153290 ] Eli Collins commented on HDFS-2563: --- +1 lgtm Nit: lines 846 and 1267 can fold onto the previous line Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2565) hdfs command does not explicitely complain that the command I am trying to run is not valid
[ https://issues.apache.org/jira/browse/HDFS-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-2565: -- Labels: bigtop noob (was: bigtop) hdfs command does not explicitely complain that the command I am trying to run is not valid --- Key: HDFS-2565 URL: https://issues.apache.org/jira/browse/HDFS-2565 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0, 0.23.1 Reporter: Bruno Mahé Labels: bigtop, noob [root@bigtop-fedora-15 ~]# hdfs foobar Exception in thread main java.lang.NoClassDefFoundError: foobar Caused by: java.lang.ClassNotFoundException: foobar at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: foobar. Program will exit. Instead of loading any class, it would be nice to explain the command is not valid and to call print_usage() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153311#comment-13153311 ] Eli Collins commented on HDFS-2564: --- * Why not just remove the catch clause entirely from refreshNamenodes? Ie no need to create a new exception of the same type * Why the new null checks for curBlock and descriptors, found a case where they can be null? * The formatting of getRpcServerAddress should match the previous method (of the same name) Otherwise looks good. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2563: -- Resolution: Fixed Fix Version/s: 0.23.1 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed for 23 and trunk, thanks Eli. Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2559) Clean up Storage/StorageInfo classes after federation
[
https://issues.apache.org/jira/browse/HDFS-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153317#comment-13153317
]
Eli Collins commented on HDFS-2559:
---
Maybe something more descriptive here? =)
{code}
if (nsInfo.getNamespaceID() == 0) { throw new RuntimeException(BARF: +
this); }
{code}
* In NNInfo looks like TODO wrt matchesStorage can be removed
Otherwise looks great
Clean up Storage/StorageInfo classes after federation
-
Key: HDFS-2559
URL: https://issues.apache.org/jira/browse/HDFS-2559
Project: Hadoop HDFS
Issue Type: Improvement
Components: data-node
Affects Versions: 0.23.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Attachments: hdfs-2559.txt
After federation, there are many different identifiers floating around in a
cluster: cluster ID, namespace ID, block pool ID, plus the versioning
identifiers cTime and layoutVersion. Some of these are cluster-wide, whereas
others may be different within a DN as it talks to different NNs (see the
federation design doc for more info). However, the current class heirarchy in
trunk has DataStorage (the DN-wide storage class) inherit from Storage which
inherits from StorageInfo, which contains a namespaceID. Thus, DataStorage
has this unused member which confuses things.
This JIRA is to split up StorageInfo into separate classes/interfaces so that
only the necessary identifiers are used by each piece of the DN.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153323#comment-13153323 ] Hudson commented on HDFS-2563: -- Integrated in Hadoop-Mapreduce-0.23-Commit #195 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/195/]) HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203942 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2559) Clean up Storage/StorageInfo classes after federation
[ https://issues.apache.org/jira/browse/HDFS-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2559: -- Status: Open (was: Patch Available) Woops re BARF! :) I need to update this a bit after the other patches that have gone into DN in the last couple days. Canceling patch, will get a new one up soon. Clean up Storage/StorageInfo classes after federation - Key: HDFS-2559 URL: https://issues.apache.org/jira/browse/HDFS-2559 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2559.txt After federation, there are many different identifiers floating around in a cluster: cluster ID, namespace ID, block pool ID, plus the versioning identifiers cTime and layoutVersion. Some of these are cluster-wide, whereas others may be different within a DN as it talks to different NNs (see the federation design doc for more info). However, the current class heirarchy in trunk has DataStorage (the DN-wide storage class) inherit from Storage which inherits from StorageInfo, which contains a namespaceID. Thus, DataStorage has this unused member which confuses things. This JIRA is to split up StorageInfo into separate classes/interfaces so that only the necessary identifiers are used by each piece of the DN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[
https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153330#comment-13153330
]
Hari Mankude commented on HDFS-2564:
{quote}
Why not just remove the catch clause entirely from refreshNamenodes? Ie
no need to create a new exception of the same type
{quote}
done. good catch.
{quote}
Why the new null checks for curBlock and descriptors, found a case where
they can be null?
{quote}
compiler warning about null ptr deref.
{quote}
The formatting of getRpcServerAddress should match the previous method (of
the same name)
{quote}
done
Cleanup unnecessary exceptions thrown and unnecessary casts
---
Key: HDFS-2564
URL: https://issues.apache.org/jira/browse/HDFS-2564
Project: Hadoop HDFS
Issue Type: Improvement
Components: data-node, hdfs client, name-node
Affects Versions: 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch,
hadoop-2564.trunk.patch
Cleaning up some of the java files with unnecessary exceptions and casts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2564: --- Attachment: hadoop-2564.trunk.patch Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153339#comment-13153339 ] Hudson commented on HDFS-2563: -- Integrated in Hadoop-Common-trunk-Commit #1285 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1285/]) HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203943 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153342#comment-13153342 ] Hudson commented on HDFS-2563: -- Integrated in Hadoop-Hdfs-trunk-Commit #1359 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1359/]) HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203943 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2566: -- Attachment: hdfs-2566.txt Straightforward class move. I added one or two simple accessors to avoid having to make some DN members accessible, but nothing complicated. Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2566: -- Status: Patch Available (was: Open) Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153347#comment-13153347 ] Hudson commented on HDFS-2563: -- Integrated in Hadoop-Hdfs-0.23-Commit #182 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/182/]) HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203942 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2566) Move BPOfferService to be a non-inner class
Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153349#comment-13153349 ] Hudson commented on HDFS-2563: -- Integrated in Hadoop-Common-0.23-Commit #183 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/183/]) HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203942 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153352#comment-13153352 ] Hadoop QA commented on HDFS-2564: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504325/hadoop-2564.trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1576//console This message is automatically generated. Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153355#comment-13153355 ] Eli Collins commented on HDFS-2566: --- +1 lgtm Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-2564: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1 I updated the patch to reflect a minor conflict with HDFS-2563 and committed it. Thanks Hari! Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2563) Some cleanup in BPOfferService
[ https://issues.apache.org/jira/browse/HDFS-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153356#comment-13153356 ] Hudson commented on HDFS-2563: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1311 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1311/]) HDFS-2563. Some cleanup in BPOfferService. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203943 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java Some cleanup in BPOfferService -- Key: HDFS-2563 URL: https://issues.apache.org/jira/browse/HDFS-2563 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2563.txt, hdfs-2563.txt BPOfferService is currently rather difficult to follow and not really commented. This JIRA is to clean up the code a bit, add javadocs/comments where necessary, and improve the formatting of the log messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153357#comment-13153357 ] Hudson commented on HDFS-2564: -- Integrated in Hadoop-Common-trunk-Commit #1286 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1286/]) HDFS-2564. Cleanup unnecessary exceptions thrown and unnecessary casts. Contributed by Hari Mankude eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203950 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153359#comment-13153359 ] Hudson commented on HDFS-2564: -- Integrated in Hadoop-Hdfs-trunk-Commit #1360 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1360/]) HDFS-2564. Cleanup unnecessary exceptions thrown and unnecessary casts. Contributed by Hari Mankude eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203950 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
[ https://issues.apache.org/jira/browse/HDFS-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153362#comment-13153362 ] Hudson commented on HDFS-2564: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1312 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1312/]) HDFS-2564. Cleanup unnecessary exceptions thrown and unnecessary casts. Contributed by Hari Mankude eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1203950 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564-1.txt, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch, hadoop-2564.trunk.patch Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2430) The number of failed or low-resource volumes the NN can tolerate should be configurable
[ https://issues.apache.org/jira/browse/HDFS-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153363#comment-13153363 ] Eli Collins commented on HDFS-2430: --- The api for NNResourcePolicy#areResourcesAvailable is a little strange, if a required resource is not available we return false, but if there are not enough redundant resources we throw a RTE. Do callers need to distinguish between these two cases? If not, we should return false in both cases. If so, there should be two separate apis. I'd make the method name better reflect the purpose in the comment (whether we can log edits), the method is generic but the particular policy is edits-specific. Does the dir checking use canonical paths so we're sure the checks pass in case of symlinks? Nits: * NNResourcePolicy needs param javadocs * NNResourcePolicy, NNResourceChecker, and CheckableNNResource need @InterfaceAudience.Private annotations Otherwise looks great. The number of failed or low-resource volumes the NN can tolerate should be configurable --- Key: HDFS-2430 URL: https://issues.apache.org/jira/browse/HDFS-2430 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.24.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Attachments: HDFS-2430.patch, HDFS-2430.patch, HDFS-2430.patch Currently the number of failed or low-resource volumes the NN can tolerate is effectively hard-coded at 1. It would be nice if this were configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153372#comment-13153372 ] Hadoop QA commented on HDFS-2566: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504328/hdfs-2566.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDfsOverAvroRpc org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1575//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1575//console This message is automatically generated. Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13153387#comment-13153387 ] Eli Collins commented on HDFS-2246: --- What 205.1 deadline are you referring to? According to Matt the code freeze for 205.1 was last Friday. Per the long discussion on list (and all the bugs Todd found with the patch after it was reviewed) this isn't an appropriate change for a dot release of a dot release, especially at such a late date in the release. How about we let this stabilize in 206 first? Shortcut a local client reads to a Datanodes files directly --- Key: HDFS-2246 URL: https://issues.apache.org/jira/browse/HDFS-2246 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Attachments: 0001-HDFS-347.-Local-reads.patch, HDFS-2246-branch-0.20-security-205.1.patch, HDFS-2246-branch-0.20-security-205.2.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security.3.patch, HDFS-2246-branch-0.20-security.no-softref.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt, HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java, localReadShortcut20-security.2patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
