[jira] [Updated] (HDFS-3013) HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure
[ https://issues.apache.org/jira/browse/HDFS-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingjie Lai updated HDFS-3013: -- Attachment: HDFS-3013.patch HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure - Key: HDFS-3013 URL: https://issues.apache.org/jira/browse/HDFS-3013 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Mingjie Lai Attachments: HDFS-3013.patch One problem I observed at HA branch is that namenode -format doesn't pick up the right configured dfs.namenode.name.dir.NameServiceId. In this case, ``namenode -format'' will format the default directory instead of the configured one, while namenode will use the correct one. The root cause is that NameNode.initializeGenericKeys() is only invoked at the NameNode constructor. So it doesn't get called by static methods like format and finalize. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sho Shimauchi reassigned HDFS-3014: --- Assignee: Sho Shimauchi FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-309) FSEditLog should log progress during replay
[ https://issues.apache.org/jira/browse/HDFS-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sho Shimauchi reassigned HDFS-309: -- Assignee: Sho Shimauchi FSEditLog should log progress during replay --- Key: HDFS-309 URL: https://issues.apache.org/jira/browse/HDFS-309 Project: Hadoop HDFS Issue Type: Improvement Reporter: Todd Lipcon Assignee: Sho Shimauchi Labels: newbie When the NameNode is replaying a long edit log, it's handy to have reports on how far through it is, so you can judge how much time it is remaining. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sho Shimauchi updated HDFS-3014: Attachment: HDFS-3014.txt attached FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi Attachments: HDFS-3014.txt FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-309) FSEditLog should log progress during replay
[
https://issues.apache.org/jira/browse/HDFS-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sho Shimauchi updated HDFS-309:
---
Attachment: HDFS-309.txt
attached patch.
replay log will be generated like this:
{quote}
2012-02-25 19:40:30,768 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(118)) - Acquiring write lock to replay
edit log
2012-02-25 19:40:30,768 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(134)) - replaying edit log: LogSegmentOp
[opCode=OP_START_LOG_SEGMENT, txid=1]
2012-02-25 19:40:30,769 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(134)) - replaying edit log: SetGenstampOp
[genStamp=1001, opCode=OP_SET_GENSTAMP, txid=2]
2012-02-25 19:40:30,769 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(134)) - replaying edit log: AddOp
[length=0, path=/testfile, replication=1, mtime=1330166430268,
atime=1330166430268, blockSize=67108864, blocks=[],
permissions=sho:supergroup:rw-r--r--,
clientName=DFSClient_NONMAPREDUCE_1990794058_1, clientMachine=127.0.0.1,
opCode=OP_ADD, txid=3]
2012-02-25 19:40:30,771 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(134)) - replaying edit log: CloseOp
[length=0, path=/testfile, replication=1, mtime=1330166430481,
atime=1330166430268, blockSize=67108864, blocks=[blk_1906532225227800470_1001],
permissions=sho:supergroup:rw-r--r--, clientName=, clientMachine=,
opCode=OP_CLOSE, txid=4]
2012-02-25 19:40:30,771 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(134)) - replaying edit log: LogSegmentOp
[opCode=OP_END_LOG_SEGMENT, txid=5]
2012-02-25 19:40:30,771 DEBUG namenode.FSEditLogLoader
(FSEditLogLoader.java:loadEditRecords(436)) - replaying edit log finished
{quote}
FSEditLog should log progress during replay
---
Key: HDFS-309
URL: https://issues.apache.org/jira/browse/HDFS-309
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Todd Lipcon
Assignee: Sho Shimauchi
Labels: newbie
Attachments: HDFS-309.txt
When the NameNode is replaying a long edit log, it's handy to have reports on
how far through it is, so you can judge how much time it is remaining.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework
[ https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216415#comment-13216415 ] Hudson commented on HDFS-3002: -- Integrated in Hadoop-Hdfs-trunk #966 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/966/]) HDFS-3002. TestNameNodeMetrics need not wait for metrics update. Contributed by Suresh Srinivas. (Revision 1293482) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293482 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java TestNameNodeMetrics need not wait for metrics update with new metrics framework --- Key: HDFS-3002 URL: https://issues.apache.org/jira/browse/HDFS-3002 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Priority: Trivial Attachments: HDFS-3002.patch With older metrics framework, the namenode metrics was updated by replication thread. This required test having to wait for replication interval. This is no longer necessary with metrics2 framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type
[ https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216416#comment-13216416 ] Hudson commented on HDFS-3006: -- Integrated in Hadoop-Hdfs-trunk #966 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/966/]) HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type to application/octet-stream instead of application/json. (Revision 1293487) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293487 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Webhdfs SETOWNER call returns incorrect content-type -- Key: HDFS-3006 URL: https://issues.apache.org/jira/browse/HDFS-3006 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.2 Reporter: bc Wong Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2 Attachments: h3006_20120223.patch, h3006_20120224.patch, h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch The SETOWNER call returns an empty body. But the header has Content-Type: application/json, which is a contradiction (empty string is not valid json). This appears to happen for SETTIMES and SETPERMISSION as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()
[ https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216417#comment-13216417 ] Hudson commented on HDFS-3003: -- Integrated in Hadoop-Hdfs-trunk #966 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/966/]) HDFS-3003. Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString(). Contributed by Brandon Li. (Revision 1293338) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293338 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileChecksumServlets.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileDataServlet.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeJsp.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStreamFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestTransferFsImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString() -- Key: HDFS-3003 URL: https://issues.apache.org/jira/browse/HDFS-3003 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Brandon Li Assignee: Brandon Li Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch The method getHostPortString() is not directly related with NameNode, and it's more of a utility method. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils
[ https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216419#comment-13216419 ] Hudson commented on HDFS-3009: -- Integrated in Hadoop-Hdfs-trunk #966 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/966/]) HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using NetUtils. Contributed by Hari Mankude. (Revision 1293390) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293390 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java DFSClient islocaladdress() can use similar routine in netutils -- Key: HDFS-3009 URL: https://issues.apache.org/jira/browse/HDFS-3009 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.0, 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-3009.patch isLocalAddress() in dfsclient can use similar function in netutils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work
[ https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216421#comment-13216421 ] Hudson commented on HDFS-3008: -- Integrated in Hadoop-Hdfs-trunk #966 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/966/]) HDFS-3008. Negative caching of local addrs doesn't work. Contributed by Eli Collins (Revision 1293419) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293419 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java Negative caching of local addrs doesn't work Key: HDFS-3008 URL: https://issues.apache.org/jira/browse/HDFS-3008 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.1, 1.1.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 1.1.0, 0.23.2 Attachments: hdfs-3008.txt, hdfs-3008.txt HDFS-2653 added negative caching of local addrs, however it still goes through the fall through path every time if the address is non-local. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type
[ https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216425#comment-13216425 ] Hudson commented on HDFS-3006: -- Integrated in Hadoop-Hdfs-0.23-Build #179 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/179/]) svn merge -c 1293487 from trunk for HDFS-3006. (Revision 1293489) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293489 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Webhdfs SETOWNER call returns incorrect content-type -- Key: HDFS-3006 URL: https://issues.apache.org/jira/browse/HDFS-3006 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.2 Reporter: bc Wong Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2 Attachments: h3006_20120223.patch, h3006_20120224.patch, h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch The SETOWNER call returns an empty body. But the header has Content-Type: application/json, which is a contradiction (empty string is not valid json). This appears to happen for SETTIMES and SETPERMISSION as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work
[ https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216428#comment-13216428 ] Hudson commented on HDFS-3008: -- Integrated in Hadoop-Hdfs-0.23-Build #179 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/179/]) HDFS-3008. svn merge -c 1293419 from trunk (Revision 1293421) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293421 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job * /hadoop/common/branches/branch-0.23/hadoop-project * /hadoop/common/branches/branch-0.23/hadoop-project/src/site Negative caching of local addrs doesn't work Key: HDFS-3008 URL: https://issues.apache.org/jira/browse/HDFS-3008 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.1, 1.1.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 1.1.0, 0.23.2 Attachments: hdfs-3008.txt, hdfs-3008.txt HDFS-2653 added negative caching of local addrs, however it still goes through the fall through path every time if the address is non-local. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For
[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216433#comment-13216433 ] Hudson commented on HDFS-2904: -- Integrated in Hadoop-Hdfs-HAbranch-build #88 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/88/]) HDFS-2904. Client support for getting delegation tokens. Contributed by Todd Lipcon. (Revision 1293486) Result = UNSTABLE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293486 Files : * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-1623.txt * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HAUtil.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSelector.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ConfiguredFailoverProxyProvider.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDelegationTokensWithHA.java HA: Client support for getting delegation tokens to an HA cluster - Key: HDFS-2904 URL: https://issues.apache.org/jira/browse/HDFS-2904 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, hdfs client, name-node, security Affects Versions: HA branch (HDFS-1623) Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: HA branch (HDFS-1623) Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh Currently we have server-side support for delegation tokens in HA, and some tests to verify it, but the client throws NPEs when trying to fetch a DT. This is because the cluster doesn't have a single hostname, but instead a logical nameservice name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type
[ https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216437#comment-13216437 ] Hudson commented on HDFS-3006: -- Integrated in Hadoop-Mapreduce-0.23-Build #207 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/207/]) svn merge -c 1293487 from trunk for HDFS-3006. (Revision 1293489) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293489 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Webhdfs SETOWNER call returns incorrect content-type -- Key: HDFS-3006 URL: https://issues.apache.org/jira/browse/HDFS-3006 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.2 Reporter: bc Wong Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2 Attachments: h3006_20120223.patch, h3006_20120224.patch, h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch The SETOWNER call returns an empty body. But the header has Content-Type: application/json, which is a contradiction (empty string is not valid json). This appears to happen for SETTIMES and SETPERMISSION as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework
[ https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216449#comment-13216449 ] Hudson commented on HDFS-3002: -- Integrated in Hadoop-Mapreduce-trunk #1001 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1001/]) HDFS-3002. TestNameNodeMetrics need not wait for metrics update. Contributed by Suresh Srinivas. (Revision 1293482) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293482 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java TestNameNodeMetrics need not wait for metrics update with new metrics framework --- Key: HDFS-3002 URL: https://issues.apache.org/jira/browse/HDFS-3002 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Priority: Trivial Attachments: HDFS-3002.patch With older metrics framework, the namenode metrics was updated by replication thread. This required test having to wait for replication interval. This is no longer necessary with metrics2 framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type
[ https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216450#comment-13216450 ] Hudson commented on HDFS-3006: -- Integrated in Hadoop-Mapreduce-trunk #1001 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1001/]) HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type to application/octet-stream instead of application/json. (Revision 1293487) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293487 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Webhdfs SETOWNER call returns incorrect content-type -- Key: HDFS-3006 URL: https://issues.apache.org/jira/browse/HDFS-3006 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.2 Reporter: bc Wong Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2 Attachments: h3006_20120223.patch, h3006_20120224.patch, h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch The SETOWNER call returns an empty body. But the header has Content-Type: application/json, which is a contradiction (empty string is not valid json). This appears to happen for SETTIMES and SETPERMISSION as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils
[ https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216453#comment-13216453 ] Hudson commented on HDFS-3009: -- Integrated in Hadoop-Mapreduce-trunk #1001 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1001/]) HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using NetUtils. Contributed by Hari Mankude. (Revision 1293390) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293390 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java DFSClient islocaladdress() can use similar routine in netutils -- Key: HDFS-3009 URL: https://issues.apache.org/jira/browse/HDFS-3009 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.0, 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-3009.patch isLocalAddress() in dfsclient can use similar function in netutils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work
[ https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216455#comment-13216455 ] Hudson commented on HDFS-3008: -- Integrated in Hadoop-Mapreduce-trunk #1001 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1001/]) HDFS-3008. Negative caching of local addrs doesn't work. Contributed by Eli Collins (Revision 1293419) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293419 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java Negative caching of local addrs doesn't work Key: HDFS-3008 URL: https://issues.apache.org/jira/browse/HDFS-3008 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.1, 1.1.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 1.1.0, 0.23.2 Attachments: hdfs-3008.txt, hdfs-3008.txt HDFS-2653 added negative caching of local addrs, however it still goes through the fall through path every time if the address is non-local. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-3013) HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure
[ https://issues.apache.org/jira/browse/HDFS-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers reassigned HDFS-3013: Assignee: Mingjie Lai HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure - Key: HDFS-3013 URL: https://issues.apache.org/jira/browse/HDFS-3013 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Mingjie Lai Assignee: Mingjie Lai Attachments: HDFS-3013.patch One problem I observed at HA branch is that namenode -format doesn't pick up the right configured dfs.namenode.name.dir.NameServiceId. In this case, ``namenode -format'' will format the default directory instead of the configured one, while namenode will use the correct one. The root cause is that NameNode.initializeGenericKeys() is only invoked at the NameNode constructor. So it doesn't get called by static methods like format and finalize. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216549#comment-13216549 ] Aaron T. Myers commented on HDFS-3014: -- Hey Sho, instead of using a StringBuilder and repeatedly appending, have you considered using String.format(...)? I think that might be a little bit clearer. FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi Attachments: HDFS-3014.txt FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216551#comment-13216551 ] Aaron T. Myers commented on HDFS-2978: -- bq. @Aaron, the current patch should be fine as all the data currently exposed follows the same json serialize approach. However, at some point, it would be good to look at exactly how this is being used or meant to be used and fix the mbean output if needed. Sure, makes sense. Want to file a JIRA for that? The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Attachments: HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216560#comment-13216560 ] Hudson commented on HDFS-2978: -- Integrated in Hadoop-Common-trunk-Commit #1781 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1781/]) HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293707 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Attachments: HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216558#comment-13216558 ] Hudson commented on HDFS-2978: -- Integrated in Hadoop-Common-0.23-Commit #596 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/596/]) HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293709 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Attachments: HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216561#comment-13216561 ] Hudson commented on HDFS-2978: -- Integrated in Hadoop-Hdfs-trunk-Commit #1855 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1855/]) HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293707 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Attachments: HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-2978: - Resolution: Fixed Fix Version/s: 0.23.3 1.1.0 Target Version/s: 1.1.0, 0.23.3 (was: 0.23.3, 1.1.0) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've just committed this to branch-1, branch-0.23, and trunk. Thanks a lot for the review, Suresh. The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 1.1.0, 0.23.3 Attachments: HDFS-2978-branch-1.patch, HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-2978: - Attachment: HDFS-2978-branch-1.patch Here's the patch I just committed to branch-1. The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 1.1.0, 0.23.3 Attachments: HDFS-2978-branch-1.patch, HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216574#comment-13216574 ] Hudson commented on HDFS-2978: -- Integrated in Hadoop-Mapreduce-0.23-Commit #597 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/597/]) HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293709 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 1.1.0, 0.23.3 Attachments: HDFS-2978-branch-1.patch, HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216573#comment-13216573 ] Hudson commented on HDFS-2978: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1792 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1792/]) HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293707 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 1.1.0, 0.23.3 Attachments: HDFS-2978-branch-1.patch, HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216577#comment-13216577 ] Hadoop QA commented on HDFS-2978: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516063/HDFS-2978.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1911//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1911//console This message is automatically generated. The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 1.1.0, 0.23.3 Attachments: HDFS-2978-branch-1.patch, HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216615#comment-13216615 ] Sho Shimauchi commented on HDFS-3014: - Can you tell me why String.format() is better than StringBuilder in this case? StringBuilder is better than String.format() in performance I think. Additionally, StringBuilder is used 456 times in trunk while String.format() is used 242 times. That mean StringBuilder seems to be more common in code. I don't think StringBuilder + append() is less readable than String.format(), but if you feel so, I'll change the format. FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi Attachments: HDFS-3014.txt FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3005) ConcurrentModificationException in FSDataset$FSVolume.getDfsUsed(..)
[
https://issues.apache.org/jira/browse/HDFS-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
VINAYAKUMAR B updated HDFS-3005:
Target Version/s: 0.24.0
Affects Version/s: 0.24.0
Status: Patch Available (was: Open)
Just changed HashMap to ConcurrentHashMap. So Not adding any Tests. Tested
Concurrent Operations on ConcurrentHashMap with Sample Program.
ConcurrentModificationException in FSDataset$FSVolume.getDfsUsed(..)
Key: HDFS-3005
URL: https://issues.apache.org/jira/browse/HDFS-3005
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node
Affects Versions: 0.24.0
Reporter: Tsz Wo (Nicholas), SZE
Attachments: HDFS-3005.patch
Saw this in [build
#1888|https://builds.apache.org/job/PreCommit-HDFS-Build/1888//testReport/org.apache.hadoop.hdfs.server.datanode/TestMulitipleNNDataBlockScanner/testBlockScannerAfterRestart/].
{noformat}
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$EntryIterator.next(HashMap.java:834)
at java.util.HashMap$EntryIterator.next(HashMap.java:832)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.getDfsUsed(FSDataset.java:557)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolumeSet.getDfsUsed(FSDataset.java:809)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolumeSet.access$1400(FSDataset.java:774)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset.getDfsUsed(FSDataset.java:1124)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.sendHeartBeat(BPOfferService.java:406)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.offerService(BPOfferService.java:490)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.run(BPOfferService.java:635)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3005) ConcurrentModificationException in FSDataset$FSVolume.getDfsUsed(..)
[
https://issues.apache.org/jira/browse/HDFS-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
VINAYAKUMAR B updated HDFS-3005:
Attachment: HDFS-3005.patch
ConcurrentModificationException in FSDataset$FSVolume.getDfsUsed(..)
Key: HDFS-3005
URL: https://issues.apache.org/jira/browse/HDFS-3005
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node
Affects Versions: 0.24.0
Reporter: Tsz Wo (Nicholas), SZE
Attachments: HDFS-3005.patch
Saw this in [build
#1888|https://builds.apache.org/job/PreCommit-HDFS-Build/1888//testReport/org.apache.hadoop.hdfs.server.datanode/TestMulitipleNNDataBlockScanner/testBlockScannerAfterRestart/].
{noformat}
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$EntryIterator.next(HashMap.java:834)
at java.util.HashMap$EntryIterator.next(HashMap.java:832)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.getDfsUsed(FSDataset.java:557)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolumeSet.getDfsUsed(FSDataset.java:809)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolumeSet.access$1400(FSDataset.java:774)
at
org.apache.hadoop.hdfs.server.datanode.FSDataset.getDfsUsed(FSDataset.java:1124)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.sendHeartBeat(BPOfferService.java:406)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.offerService(BPOfferService.java:490)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.run(BPOfferService.java:635)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[
https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216627#comment-13216627
]
Aaron T. Myers commented on HDFS-3014:
--
It's mostly personal preference. I just find this:
{code}
builder.append(AddOp [length=);
builder.append(length);
builder.append(, path=);
builder.append(path);
builder.append(, replication=);
builder.append(replication);
builder.append(, mtime=);
builder.append(mtime);
builder.append(, atime=);
builder.append(atime);
builder.append(, blockSize=);
builder.append(blockSize);
builder.append(, blocks=);
builder.append(Arrays.toString(blocks));
builder.append(, permissions=);
builder.append(permissions);
builder.append(, clientName=);
builder.append(clientName);
builder.append(, clientMachine=);
builder.append(clientMachine);
builder.append(, opCode=);
builder.append(opCode);
builder.append(, txid=);
builder.append(txid);
builder.append(]);
{code}
to be less readable than this:
{code}
String.format(AddOp [length=%d, path=%s, replication=%d, mtime=%d, atime=%d,
blocksize=%d, +
blocks=%s, permissions=%s, clientName=%s, clientMachine=%s, opCode=%d,
txid=%d],
length, path, replication, mtime, atime, blockSize, Arrays.toString(blocks),
permissions,
clientName, clientMachine, opCode, txid);
{code}
I also suspect that if one were to do a detailed analysis of StringBuilder vs.
String.format usage in the code base, you'd probably find that StringBuilder is
used mostly when there is an open-ended number of items to be included in the
string (e.g. when stringifying a list of indeterminate length), and therefore
String.format couldn't possibly be used.
Regardless, I don't feel very strongly about this point. If you think it's
better to use StringBuilder, then I don't mind committing it as you've
implemented it.
Thoughts?
FSEditLogOp and its subclasses should have toString() method
Key: HDFS-3014
URL: https://issues.apache.org/jira/browse/HDFS-3014
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 0.23.0
Reporter: Sho Shimauchi
Assignee: Sho Shimauchi
Attachments: HDFS-3014.txt
FSEditLogOp and its subclasses should have toString() method.
It's helpful for investigating editlog related issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216634#comment-13216634 ] Sho Shimauchi commented on HDFS-3014: - The reason why I prefer to use StringBuilder is that it's faster than String.format(). These toString() will be called only when developer debugs fsimage/edits related issues. This method won't be called not so often, therefore most user will never see these outputs. However, once this method is called, it's probably called many times. For example, I'll use this method to print debug log of replaying edit log (see HDFS-309 ). Using StringBuilder will reduce time to replay edit log. That means developer will reduce time to diagnose corrupted edit log and therefore the debeloper can seek the root cause of the issue faster. Notes that I don't have strong opinion about this either. I'll change the patch if it's better. FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi Attachments: HDFS-3014.txt FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216637#comment-13216637 ] Aaron T. Myers commented on HDFS-3014: -- OK then, fine by me. One other comment then: I think we should refactor the toString methods of AddOp and CloseOp, since they're almost identical and those classes have a common superclass. Perhaps AddCloseOp could have a stringifyMembers() method which the subclasses could call? +1 once this is addressed. FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi Attachments: HDFS-3014.txt FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
