[jira] [Commented] (HDFS-229) Show last checkpoint time on the webUI
[ https://issues.apache.org/jira/browse/HDFS-229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13177981#comment-13177981 ] Sho Shimauchi commented on HDFS-229: dfsadmin -report is too painful, because it's difficult to show checkpoint in dfsadmin -report without adding unnecessary DistributedFileSystem/DFSClient APIs. I'll implement WebUI and metrics instead. Show last checkpoint time on the webUI -- Key: HDFS-229 URL: https://issues.apache.org/jira/browse/HDFS-229 Project: Hadoop HDFS Issue Type: New Feature Reporter: Koji Noguchi Assignee: Sho Shimauchi Priority: Minor We've had a couple of occasions where secondary namenode fail/die without anyone noticing. (HADOOP-1695) It would be nice if the last checkpoint time can be shown on the namenode webui. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2716) HA: Configuration needs to allow different dfs.http.addresses for each HA NN
[ https://issues.apache.org/jira/browse/HDFS-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13177996#comment-13177996 ] Hudson commented on HDFS-2716: -- Integrated in Hadoop-Hdfs-HAbranch-build #33 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/33/]) HDFS-2716. Configuration needs to allow different dfs.http.addresses for each HA NN. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1226020 Files : * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-1623.txt * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUtil.java HA: Configuration needs to allow different dfs.http.addresses for each HA NN Key: HDFS-2716 URL: https://issues.apache.org/jira/browse/HDFS-2716 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: HA branch (HDFS-1623) Attachments: hdfs-2716.txt, hdfs-2716.txt Earlier on the HA branch we expanded the configuration so that different IPC addresses can be specified for each of the HA NNs in a cluster. But we didn't do this for the HTTP address. This has proved problematic while working on HDFS-2291 (checkpointing in HA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2720) HA : TestStandbyIsHot is failing while copying in_use.lock file from NN1 nameSpaceDirs to NN2 nameSpaceDirs
[ https://issues.apache.org/jira/browse/HDFS-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178029#comment-13178029 ] Uma Maheswara Rao G commented on HDFS-2720: --- Todd, Thanks a lot for the Review! I am presently on leave, will update patch soon. Thanks Uma HA : TestStandbyIsHot is failing while copying in_use.lock file from NN1 nameSpaceDirs to NN2 nameSpaceDirs Key: HDFS-2720 URL: https://issues.apache.org/jira/browse/HDFS-2720 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, test Affects Versions: HA branch (HDFS-1623) Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2720.patch To maintain the clusterID same , we are copying the namespaceDirs from 1st NN to other NNs. While copying this files, in_use.lock file may not allow to copy in all the OSs since it has aquired the lock on it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2739) SecondaryNameNode doesn't start up
[
https://issues.apache.org/jira/browse/HDFS-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas reassigned HDFS-2739:
-
Assignee: Suresh Srinivas
SecondaryNameNode doesn't start up
--
Key: HDFS-2739
URL: https://issues.apache.org/jira/browse/HDFS-2739
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.24.0
Reporter: Sho Shimauchi
Assignee: Suresh Srinivas
Priority: Critical
Built a 0.24-SNAPSHOT tar from today, used a general config, started NN/DN,
but SNN won't come up with following error:
{code}
11/12/31 12:13:14 ERROR namenode.SecondaryNameNode: Throwable Exception in
doCheckpoint
java.lang.RuntimeException: java.lang.NoSuchFieldException: versionID
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:154)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invocation.init(WritableRpcEngine.java:112)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:226)
at $Proxy9.getTransationId(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB.getTransactionID(NamenodeProtocolTranslatorPB.java:185)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.countUncheckpointedTxns(SecondaryNameNode.java:625)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.shouldCheckpointBasedOnCount(SecondaryNameNode.java:633)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:386)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:356)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NoSuchFieldException: versionID
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:150)
... 9 more
java.lang.RuntimeException: java.lang.NoSuchFieldException: versionID
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:154)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invocation.init(WritableRpcEngine.java:112)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:226)
at $Proxy9.getTransationId(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB.getTransactionID(NamenodeProtocolTranslatorPB.java:185)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.countUncheckpointedTxns(SecondaryNameNode.java:625)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.shouldCheckpointBasedOnCount(SecondaryNameNode.java:633)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:386)
at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:356)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.NoSuchFieldException: versionID
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:150)
... 9 more
11/12/31 12:13:14 INFO namenode.SecondaryNameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down SecondaryNameNode at sho-mba.local/192.168.11.2
/
{code}
full error log: http://pastebin.com/mSaVbS34
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Issue with Namenode storage
Hi, Namenode storage on our cluster went out suddenly with the following error 2011-12-30 18:24:59,857 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Unable to open edit log file /data/d1/hadoop-data/hadoop-hdfs/dfs/name/current/edits 2011-12-30 18:24:59,858 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Unable to open edit log file /data/d2/hadoop-data/hadoop-hdfs/dfs/name/current/edits dfshealth.jsp reports the Storage to be unhealthy NameNode Storage: Storage Directory Type State /data/d1/hadoop-data/hadoop-hdfs/dfs/name IMAGE_AND_EDITS Failed /data/d2/hadoop-data/hadoop-hdfs/dfs/name IMAGE_AND_EDITS Failed While the cluster is functional the edit logs are not being actively written to. Looks like if the cluster were to be restarted, we would loose changes before this error occurred. It seems like image update from the secondary namenode caused this. Previous pushes from the SNN were OK though. 2011-12-30 18:24:59,855 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll FSImage from 10.3.0.161 2011-12-30 18:24:59,855 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of transactions: 5552 Total time for transactions(ms): 32Number of transactions batched in Syncs: 694 Number of syncs: 2690 SyncTimes(ms): 1789 1669 2011-12-30 18:24:59,857 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Unable to open edit log file /data/d1/hadoop-data/hadoop-hdfs/dfs/name/current/edits 2011-12-30 18:24:59,858 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Unable to open edit log file /data/d2/hadoop-data/hadoop-hdfs/dfs/name/current/edits dfs.name.dir contents: /data/d1/hadoop-data/hadoop-hdfs/dfs/name: total 12 drwxr-xr-x 2 hdfs hdfs 4096 2011-12-30 18:24 current drwxr-xr-x 2 hdfs hdfs 4096 2011-11-10 18:41 image -rw-r--r-- 1 hdfs hdfs 0 2011-11-28 18:43 in_use.lock drwxr-xr-x 2 hdfs hdfs 4096 2011-11-23 10:51 previous.checkpoint /data/d1/hadoop-data/hadoop-hdfs/dfs/name/current: total 1938828 -rw-r--r-- 1 hdfs hdfs 660736 2011-12-30 18:24 edits -rw-r--r-- 1 hdfs hdfs 991278946 2011-12-30 18:17 fsimage -rw-r--r-- 1 hdfs hdfs 991452881 2011-12-30 18:24 fsimage.ckpt -rw-r--r-- 1 hdfs hdfs 8 2011-12-30 18:17 fstime -rw-r--r-- 1 hdfs hdfs 101 2011-12-30 18:17 VERSION /data/d1/hadoop-data/hadoop-hdfs/dfs/name/image: total 4 -rw-r--r-- 1 hdfs hdfs 157 2011-12-30 18:17 fsimage /data/d1/hadoop-data/hadoop-hdfs/dfs/name/previous.checkpoint: total 9013076 -rw-r--r-- 1 hdfs hdfs 8705120607 2011-11-28 17:31 edits -rw-r--r-- 1 hdfs hdfs 516045025 2011-11-23 10:51 fsimage -rw-r--r-- 1 hdfs hdfs 8 2011-11-23 10:51 fstime -rw-r--r-- 1 hdfs hdfs 101 2011-11-23 10:51 VERSION getImage servlet on NN returns the following error 2011-12-31 13:12:34,686 WARN org.mortbay.log: /getimage: java.io.IOException: GetImage failed. java.lang.NullPointerException at org.apache.hadoop.hdfs.server.namenode.FSImage.getImageFile(FSImage.java:219) at org.apache.hadoop.hdfs.server.namenode.FSImage.getFsImageName(FSImage.java:1584) at org.apache.hadoop.hdfs.server.namenode.GetImageServlet$1.run(GetImageServlet.java:75) at org.apache.hadoop.hdfs.server.namenode.GetImageServlet$1.run(GetImageServlet.java:70) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) What are the causes for this error? What can be done to restart the cluster without loss of data? Any solutions/pointers are greatly appreciated. Regards Srikanth Sundarrajan
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178102#comment-13178102 ] Harsh J commented on HDFS-2574: --- Hey Joe, sorry about all that delay, got busy. Patch LGTM, and being doc-only doesn't require a build. Committing in a bit. remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-2574: -- Target Version/s: 0.24.0, 0.23.1 remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178103#comment-13178103 ] Harsh J commented on HDFS-2574: --- There seems to be some issue applying the dfs.datanode.data.dir.perm stuff. I went ahead and resolved it locally since the rejected hunk just had prop replacements. Targeting 0.23 and trunk. remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178104#comment-13178104 ] Harsh J commented on HDFS-2574: --- Hrm, merge always seems to do these prop changes. But no harm caused I believe. remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Fix For: 0.23.1 Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-2574: -- Resolution: Fixed Fix Version/s: 0.23.1 Target Version/s: 0.24.0, 0.23.1 (was: 0.23.1, 0.24.0) Status: Resolved (was: Patch Available) Committed to branch-0.23 and trunk. Thanks very much for your patches Joe! remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Fix For: 0.23.1 Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178108#comment-13178108 ] Hudson commented on HDFS-2574: -- Integrated in Hadoop-Common-0.23-Commit #329 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/329/]) merge HDFS-2574 harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1226212 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job * /hadoop/common/branches/branch-0.23/hadoop-project * /hadoop/common/branches/branch-0.23/hadoop-project/src/site remove references to deprecated properties in hdfs-site.xml template
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178110#comment-13178110 ] Hudson commented on HDFS-2574: -- Integrated in Hadoop-Hdfs-0.23-Commit #318 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/318/]) merge HDFS-2574 harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1226212 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job * /hadoop/common/branches/branch-0.23/hadoop-project * /hadoop/common/branches/branch-0.23/hadoop-project/src/site remove references to deprecated properties in hdfs-site.xml template and
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178111#comment-13178111 ] Hudson commented on HDFS-2574: -- Integrated in Hadoop-Common-trunk-Commit #1483 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1483/]) HDFS-2574. Remove references to some deprecated properties in conf templates and defaults files. (Joe Crobak via harsh) harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1226211 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Fix For: 0.23.1 Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2319) Add test cases for FSshell -stat
[ https://issues.apache.org/jira/browse/HDFS-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178112#comment-13178112 ] Harsh J commented on HDFS-2319: --- +1, thanks for tests. Running TestHDFSCLI after HADOOP-7574 and will commit if it passes now. Add test cases for FSshell -stat Key: HDFS-2319 URL: https://issues.apache.org/jira/browse/HDFS-2319 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 0.24.0 Reporter: XieXianshan Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-2319.patch Add test cases for HADOOP-7574. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-2574: -- Target Version/s: 0.24.0, 0.23.1 (was: 0.23.1, 0.24.0) Hadoop Flags: Reviewed remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Fix For: 0.23.1 Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178115#comment-13178115 ] Hudson commented on HDFS-2574: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1504 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1504/]) HDFS-2574. Remove references to some deprecated properties in conf templates and defaults files. (Joe Crobak via harsh) harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1226211 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Joe Crobak Priority: Trivial Fix For: 0.23.1 Attachments: HDFS-2574.patch, HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178114#comment-13178114 ] Hudson commented on HDFS-2574: -- Integrated in Hadoop-Mapreduce-0.23-Commit #340 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/340/]) merge HDFS-2574 harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1226212 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job * /hadoop/common/branches/branch-0.23/hadoop-project * /hadoop/common/branches/branch-0.23/hadoop-project/src/site remove references to deprecated properties in hdfs-site.xml
[jira] [Commented] (HDFS-929) DFSClient#getBlockSize is unused
[
https://issues.apache.org/jira/browse/HDFS-929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178116#comment-13178116
]
Harsh J commented on HDFS-929:
--
Thanks for the careful eye Jim!
This is provided by the ClientProtocol. If it is being deprecated, it should be
deprecated all the way, and removed at a subsequent release. This'd incur an
unwanted change in compatibility and will make HDFS require major version
upgrade. So far, I only see server-side mechanisms use this via the
{{INodeFile#getPreferredBlockSize(…)}} method that eventually gets called, and
this is called pretty directly.
I do not know why its published into the client (seems to be available since
very long ago) but there's no harm in keeping it as DFSClient was never meant
for user consumption. Perhaps a feature that may get added later may use it?
If you agree with this assessment, lets resolve this one.
DFSClient#getBlockSize is unused
Key: HDFS-929
URL: https://issues.apache.org/jira/browse/HDFS-929
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 0.21.0
Reporter: Eli Collins
Assignee: Jim Plush
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-929-take1.txt
DFSClient#getBlockSize is unused. Since it's a public class internal to HDFS
we just remove it? If not then we should add a unit test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-962) Make DFSOutputStream MAX_PACKETS configurable
[ https://issues.apache.org/jira/browse/HDFS-962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178117#comment-13178117 ] Harsh J commented on HDFS-962: -- Nicholas, Is the test required to check different values of packets and their impact or just config reading in general? Make DFSOutputStream MAX_PACKETS configurable - Key: HDFS-962 URL: https://issues.apache.org/jira/browse/HDFS-962 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Affects Versions: 0.22.0 Reporter: Todd Lipcon Assignee: Justin Joseph Priority: Minor Attachments: HDFS-962.1.patch, HDFS-962.patch HDFS-959 suggests that the MAX_PACKETS variable (which determines how many outstanding data packets the DFSOutputStream will permit) may have an impact on performance. If so, we should make it configurable to trade off between memory and performance. I think it ought to be a secret/undocumented config for now - this will make it easier to benchmark without confusing users. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-979) FSImage should specify which dirs are missing when refusing to come up
[
https://issues.apache.org/jira/browse/HDFS-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178118#comment-13178118
]
Harsh J commented on HDFS-979:
--
Hey Jim, did you get a chance to apply Matt's review comments to the patch? :)
FSImage should specify which dirs are missing when refusing to come up
--
Key: HDFS-979
URL: https://issues.apache.org/jira/browse/HDFS-979
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.22.0
Reporter: Steve Loughran
Assignee: Jim Plush
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-979-take1.txt, HDFS-979-take2.txt,
HDFS-979-take3.txt
When {{FSImage}} can't come up as either it has no data or edit dirs, it
tells me this
{code}
java.io.IOException: All specified directories are not accessible or do not
exist.
{code}
What it doesn't do is say which of the two attributes are missing. This would
be beneficial to anyone trying to track down the problem. Also, I don't think
the message is correct. It's bailing out because dataDirs.size() == 0 ||
editsDirs.size() == 0 , because a list is empty -not because the dirs aren't
there, as there hasn't been any validation yet.
More useful would be
# Explicit mention of which attributes are null
# Declare that this is because they are not in the config
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-402) Display the server version in dfsadmin -report
[ https://issues.apache.org/jira/browse/HDFS-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13178119#comment-13178119 ] Harsh J commented on HDFS-402: -- -1 on that patch, its not the 'required' fix for this JIRA. An easy misleader with that would be that the client running report will print 0.23.1 while the real server version may just be 0.23.0. This is not what is required. But FWIW, version is present in the web UI and in metrics today. (name : hadoop:service=NameNode,name=NameNodeInfo carries a 'Version' param.) Also, if the client is incompatible, a better message is thrown (in 0.23+) indicating the protocol version differences (not version string, however). Display the server version in dfsadmin -report -- Key: HDFS-402 URL: https://issues.apache.org/jira/browse/HDFS-402 Project: Hadoop HDFS Issue Type: Wish Reporter: Jakob Homan Priority: Minor Labels: newbie Attachments: HDFS-402.patch As part of HADOOP-5094, it was requested to include the server version in the dfsadmin -report, to avoid the need to screen scrape to get this information: bq. Please do provide the server version, so there is a quick and non-taxing way of determine what is the current running version on the namenode. Currently there is nothing in the dfs client protocol to query this information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1339) NameNodeMetrics should use MetricsTimeVaryingLong
[ https://issues.apache.org/jira/browse/HDFS-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-1339: -- Resolution: Duplicate Status: Resolved (was: Patch Available) This should be resolved with HDFS-1117 on 0.23+. Please reopen if not. NameNodeMetrics should use MetricsTimeVaryingLong -- Key: HDFS-1339 URL: https://issues.apache.org/jira/browse/HDFS-1339 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Scott Chen Assignee: Scott Chen Priority: Minor Attachments: HDFS-1339.txt NameNodeMetrics uses MetricsTimeVaryingInt. We see that FileInfoOps and GetBlockLocations overflow in our cluster. Using MetricsTimeVaryingLong will easily solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
