[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154079#comment-13154079 ] Hadoop QA commented on HDFS-2575: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504474/hdfs-2575.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1592//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1592//console This message is automatically generated. DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2415) Move MiniDFS out of test JAR and into the main hadoop-hdfs JAR
[ https://issues.apache.org/jira/browse/HDFS-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154107#comment-13154107 ] Steve Loughran commented on HDFS-2415: -- # if the test jar includes log4j.properties it causes problems downstream # putting it in the main jar formalises its existence as a stable tool for external use. Which I think it should be, though you may disagree. The key point is that these mini clusters are ideal for downstream testing, as well as in-Hadoop. Move MiniDFS out of test JAR and into the main hadoop-hdfs JAR -- Key: HDFS-2415 URL: https://issues.apache.org/jira/browse/HDFS-2415 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Steve Loughran Priority: Minor This is just an idea: move the MiniDFS cluster out of the Hadoop test JAR and into the main redistributable. This would make it easier for people downstream to use it. It is the easiest way to bring up a DFS cluster in a single JVM, and with the MiniMR cluster is a common way to test MR jobs in the IDE against small datasets. Moving the file while keeping the package name should not cause any problems; all it will do is make applying outstanding patches to it slightly harder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-54) datanode not failing when read-only filesystem
[ https://issues.apache.org/jira/browse/HDFS-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154138#comment-13154138 ] Harsh J commented on HDFS-54: - Although there is a check in BlockReceiverinit now for disk errors upon IOE, it seems to be limited to errors that only go like: Possible disk error on file creation: I guess this is hence still not covered today. Should we get rid of this check, and simply issue disk error checks each time we plainly hit an IOE? Other places do this directly, but perhaps for the constructor here its too expensive to. datanode not failing when read-only filesystem -- Key: HDFS-54 URL: https://issues.apache.org/jira/browse/HDFS-54 Project: Hadoop HDFS Issue Type: Bug Reporter: Koji Noguchi Priority: Minor (This is not directly related to dfs -put hanging, but thought it should get fixed.) Datanode is catching IOException but not shutting itself down. 2008-02-02 00:10:24,237 ERROR org.apache.hadoop.dfs.DataNode: DataXceiver: java.io.IOException: Read-only file system at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createNewFile(File.java:883) at org.apache.hadoop.dfs.FSDataset$FSVolume.createTmpFile(FSDataset.java:329) at org.apache.hadoop.dfs.FSDataset.createTmpFile(FSDataset.java:606) at org.apache.hadoop.dfs.FSDataset.writeToBlock(FSDataset.java:582) at org.apache.hadoop.dfs.DataNode$BlockReceiver.init(DataNode.java:1257) at org.apache.hadoop.dfs.DataNode$DataXceiver.writeBlock(DataNode.java:901) at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:804) at java.lang.Thread.run(Thread.java:619) 2008-02-02 00:16:29,996 INFO org.apache.hadoop.dfs.DataNode: Received block blk_-7723120264171092160 from /11.11.11.11 2008-02-02 00:41:40,409 INFO org.apache.hadoop.dfs.DataNode: Received block blk_1939877544554342517 from /22.22.22.22 2008-02-02 00:46:53,925 INFO org.apache.hadoop.dfs.DataNode: Received block blk_-4102605170938551016 from /33.33.33.33 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2514) Link resolution bug for intermediate symlinks with relative targets
[ https://issues.apache.org/jira/browse/HDFS-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154152#comment-13154152 ] Hudson commented on HDFS-2514: -- Integrated in Hadoop-Mapreduce-0.23-Build #100 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/100/]) HDFS-2514. svn merge -c 1204370 from trunk eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204373 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/UnresolvedPathException.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestFcHdfsSymlink.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java *
[jira] [Commented] (HDFS-2514) Link resolution bug for intermediate symlinks with relative targets
[ https://issues.apache.org/jira/browse/HDFS-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154168#comment-13154168 ] Hudson commented on HDFS-2514: -- Integrated in Hadoop-Mapreduce-trunk #904 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/904/]) HDFS-2514. Link resolution bug for intermediate symlinks with relative targets. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204370 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/UnresolvedPathException.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestFcHdfsSymlink.java Link resolution bug for intermediate symlinks with relative targets --- Key: HDFS-2514 URL: https://issues.apache.org/jira/browse/HDFS-2514 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0, 0.22.0, 0.23.0 Reporter: Eli Collins Assignee: Eli Collins Attachments: hdfs-2514-1.patch, hdfs-2514-2.patch, hdfs-2514-3.patch There's a bug in the way the Namenode resolves intermediate symlinks (ie the symlink is not the final path component) in paths when the symlink's target is a relative path. Will post the full description in the first comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2532) TestDfsOverAvroRpc timing out in trunk
[ https://issues.apache.org/jira/browse/HDFS-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154164#comment-13154164 ] Hudson commented on HDFS-2532: -- Integrated in Hadoop-Mapreduce-trunk #904 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/904/]) HDFS-2532. TestDfsOverAvroRpc timing out in trunk. Contributed by Uma Maheswara Rao G. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204366 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java TestDfsOverAvroRpc timing out in trunk -- Key: HDFS-2532 URL: https://issues.apache.org/jira/browse/HDFS-2532 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.24.0 Reporter: Todd Lipcon Assignee: Uma Maheswara Rao G Priority: Critical Fix For: 0.24.0 Attachments: HDFS-2532.patch, hdfs-2532-make-timeout.txt java.io.IOException: java.io.IOException: Unknown protocol: org.apache.hadoop.ipc.AvroRpcEngine$TunnelProtocol occurs while starting up the DN, and then it hangs waiting for the MiniCluster to start. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2572) Unnecessary double-check in DN#getHostName
[ https://issues.apache.org/jira/browse/HDFS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154169#comment-13154169 ] Hudson commented on HDFS-2572: -- Integrated in Hadoop-Mapreduce-trunk #904 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/904/]) HDFS-2572. Remove Unnecessary double-check in DN#getHostName (harsh) harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204177 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java Unnecessary double-check in DN#getHostName -- Key: HDFS-2572 URL: https://issues.apache.org/jira/browse/HDFS-2572 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0 Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-2572.patch, HDFS-2572.patch We do a double config.get unnecessarily inside DN#getHostName(...). Can be removed by this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-71) HTTP ERROR: 500 when using Go back to dir listing link in NameNode web interface
[ https://issues.apache.org/jira/browse/HDFS-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HDFS-71. - Resolution: Cannot Reproduce Can't reproduce on 0.23/0.20-security today. Resolving for now, lets reopen when it surfaces again (although it looks like a wrong root dir string, probably a bug fixed already.) HTTP ERROR: 500 when using Go back to dir listing link in NameNode web interface -- Key: HDFS-71 URL: https://issues.apache.org/jira/browse/HDFS-71 Project: Hadoop HDFS Issue Type: Bug Reporter: Albert Strasheim Priority: Minor I'm running Hadoop 0.13.0 on Windows XP. Steps to reproduce HTTP 500 error: 1. Go to http://localhost:50070/dfshealth.jsp 2. Click Browse the filesystem 3. Click on any file 4. Click Go back to dir listing If I do this, I get the following error: HTTP ERROR: 500 java.io.IOException: Cannot open filename \ at org.apache.hadoop.dfs.NameNode.open(NameNode.java:263) at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:573) RequestURI=/browseDirectory.jsp -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-111) UnderReplicationBlocks should use generic types
[ https://issues.apache.org/jira/browse/HDFS-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HDFS-111. -- Resolution: Not A Problem Both comments seem to have been addressed already today. Not a problem anymore, resolving. UnderReplicationBlocks should use generic types --- Key: HDFS-111 URL: https://issues.apache.org/jira/browse/HDFS-111 Project: Hadoop HDFS Issue Type: Bug Reporter: Konstantin Shvachko Priority: Minor 1. HADOOP-659 introduced new class UnderReplicationBlocks, and two new warnings about use of generic types TreeSetBlock[] priorityQueues = new TreeSet[LEVEL]; IteratorBlock[] iterator = new Iterator[LEVEL]; should resp. read TreeSetBlock[] priorityQueues = new TreeSetBlock[LEVEL]; IteratorBlock[] iterator = new IteratorBlock[LEVEL]; 2. I'd rename class UnderReplicationBlocks to UnderReplicatedBlocks while it is still internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-131) null pointer exception while accessing secondaryname web interface (servlet dfshealth.jsp should not be served from the
secondary Namenode) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HDFS-131. -- Resolution: Cannot Reproduce Resolved, after HADOOP-3741 where it now redirects / to /status.jsp and shows useful info. null pointer exception while accessing secondaryname web interface (servlet dfshealth.jsp should not be served from the secondary Namenode) --- Key: HDFS-131 URL: https://issues.apache.org/jira/browse/HDFS-131 Project: Hadoop HDFS Issue Type: Bug Environment: Hadoop 0.16.2, and before Reporter: Yuri Pradkin Priority: Minor when I go to the secondary namenode HTTP (dfs.secondary.http.address) in my browser I see something like this: HTTP ERROR: 500 init RequestURI=/dfshealth.jsp Powered by Jetty:// And in secondary's log I find these lines: 2008-04-02 11:26:25,357 WARN /: /dfshealth.jsp: java.lang.NullPointerException at org.apache.hadoop.dfs.dfshealth_jsp.init(dfshealth_jsp.java:21) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:539) at java.lang.Class.newInstance0(Class.java:373) at java.lang.Class.newInstance(Class.java:326) at org.mortbay.jetty.servlet.Holder.newInstance(Holder.java:199) at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:326) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:405) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567) at org.mortbay.http.HttpContext.handle(HttpContext.java:1565) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635) at org.mortbay.http.HttpContext.handle(HttpContext.java:1517) at org.mortbay.http.HttpServer.service(HttpServer.java:954) at org.mortbay.http.HttpConnection.service(HttpConnection.java:814) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) Responses from core-u...@hadoop.apache.org: dhruba Borthakur: The secondary Namenode uses the HTTP interface to pull the fsimage from the primary. Similarly, the primary Namenode uses the dfs.secondary.http.address to pull the checkpointed-fsimage back from the secondary to the primary. So, the definition of dfs.secondary.http.address is needed. However, the servlet dfshealth.jsp should not be served from the secondary Namenode. This servet should be setup in such a way that only the primary Namenode invokes this servlet. -- Konstantin Shvachko: We do not have any secondary nn web interface as of today. The http server is used for transferring data between the primary and the secondary. I don't see we can display anything useful on the secondary web UI except for the current status, config values, and the last checkpoint date/time. -- -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-497) One of the DFSClient::create functions ignores parameter
[
https://issues.apache.org/jira/browse/HDFS-497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved HDFS-497.
--
Resolution: Cannot Reproduce
The create(Str, bool, Progressable) seems to pass what it gets properly.
Resolving as can't reproduce today.
From 0.23:
{code}
/**
* Call {@link #create(String, boolean, short, long, Progressable)} with
* default codereplication/code and codeblockSizecode.
*/
public OutputStream create(String src,
boolean overwrite,
Progressable progress) throws IOException {
return create(src, overwrite, dfsClientConf.defaultReplication,
dfsClientConf.defaultBlockSize, progress);
}
{code}
One of the DFSClient::create functions ignores parameter
Key: HDFS-497
URL: https://issues.apache.org/jira/browse/HDFS-497
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.20.1
Reporter: Bill Zeller
Priority: Minor
Labels: newbie
DFSClient::create(String src, boolean overwrite, Progressable progress)
ignores progress parameter
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2514) Link resolution bug for intermediate symlinks with relative targets
[ https://issues.apache.org/jira/browse/HDFS-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154197#comment-13154197 ] Hudson commented on HDFS-2514: -- Integrated in Hadoop-Hdfs-0.23-Build #83 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/83/]) HDFS-2514. svn merge -c 1204370 from trunk eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204373 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/UnresolvedPathException.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestFcHdfsSymlink.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++ * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java *
[jira] [Commented] (HDFS-2514) Link resolution bug for intermediate symlinks with relative targets
[ https://issues.apache.org/jira/browse/HDFS-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154203#comment-13154203 ] Hudson commented on HDFS-2514: -- Integrated in Hadoop-Hdfs-22-branch #112 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/112/]) HDFS-2514. svn merge -c 1204370 from trunk eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204378 Files : * /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/protocol/UnresolvedPathException.java * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/INode.java * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/fs/TestFcHdfsSymlink.java Link resolution bug for intermediate symlinks with relative targets --- Key: HDFS-2514 URL: https://issues.apache.org/jira/browse/HDFS-2514 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0, 0.22.0, 0.23.0 Reporter: Eli Collins Assignee: Eli Collins Attachments: hdfs-2514-1.patch, hdfs-2514-2.patch, hdfs-2514-3.patch There's a bug in the way the Namenode resolves intermediate symlinks (ie the symlink is not the final path component) in paths when the symlink's target is a relative path. Will post the full description in the first comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-554) BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
[
https://issues.apache.org/jira/browse/HDFS-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-554:
-
Attachment: HDFS-554.patch
The difference is apparently very clear, in terms of speed, from even a silly
test:
{code}
public class TestSpeed {
public static void main(String[] args) {
// Load about a million Integers.
Object[] arr = new Object[100];
for (Integer i = 0; i 100; i++) {
arr[i] = i;
}
long now = System.currentTimeMillis();
// Copy iteratively into a new sized array.
Object[] arr2 = new Object[300];
for (Integer i = 0; i arr.length; i++) {
arr2[i] = arr[i];
}
System.out.println(System.currentTimeMillis() - now);
now = System.currentTimeMillis();
// arraycopy into a new sized array.
Object[] arr3 = new Object[300];
System.arraycopy(arr, 0, arr3, 0, arr.length);
System.out.println(System.currentTimeMillis() - now);
}
}
{code}
A few runs do, for example:
||Loop||System.arraycopy||
|59|17|
|54|14|
|52|14|
|52|15|
BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
--
Key: HDFS-554
URL: https://issues.apache.org/jira/browse/HDFS-554
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.21.0
Reporter: Steve Loughran
Priority: Minor
Attachments: HDFS-554.patch
BlockInfo.ensureCapacity() uses a for() loop to copy the old array data into
the expanded array. {{System.arraycopy()}} is generally much faster for
this, as it can do a bulk memory copy. There is also the typesafe Java6
{{Arrays.copyOf()}} to consider, though here it offers no tangible benefit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2514) Link resolution bug for intermediate symlinks with relative targets
[ https://issues.apache.org/jira/browse/HDFS-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154215#comment-13154215 ] Hudson commented on HDFS-2514: -- Integrated in Hadoop-Hdfs-trunk #870 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/870/]) HDFS-2514. Link resolution bug for intermediate symlinks with relative targets. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204370 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/UnresolvedPathException.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestFcHdfsSymlink.java Link resolution bug for intermediate symlinks with relative targets --- Key: HDFS-2514 URL: https://issues.apache.org/jira/browse/HDFS-2514 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0, 0.22.0, 0.23.0 Reporter: Eli Collins Assignee: Eli Collins Attachments: hdfs-2514-1.patch, hdfs-2514-2.patch, hdfs-2514-3.patch There's a bug in the way the Namenode resolves intermediate symlinks (ie the symlink is not the final path component) in paths when the symlink's target is a relative path. Will post the full description in the first comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2572) Unnecessary double-check in DN#getHostName
[ https://issues.apache.org/jira/browse/HDFS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154216#comment-13154216 ] Hudson commented on HDFS-2572: -- Integrated in Hadoop-Hdfs-trunk #870 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/870/]) HDFS-2572. Remove Unnecessary double-check in DN#getHostName (harsh) harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204177 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java Unnecessary double-check in DN#getHostName -- Key: HDFS-2572 URL: https://issues.apache.org/jira/browse/HDFS-2572 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0 Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-2572.patch, HDFS-2572.patch We do a double config.get unnecessarily inside DN#getHostName(...). Can be removed by this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-554) BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
[
https://issues.apache.org/jira/browse/HDFS-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-554:
-
Fix Version/s: 0.24.0
Status: Patch Available (was: Open)
BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
--
Key: HDFS-554
URL: https://issues.apache.org/jira/browse/HDFS-554
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.21.0
Reporter: Steve Loughran
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-554.patch
BlockInfo.ensureCapacity() uses a for() loop to copy the old array data into
the expanded array. {{System.arraycopy()}} is generally much faster for
this, as it can do a bulk memory copy. There is also the typesafe Java6
{{Arrays.copyOf()}} to consider, though here it offers no tangible benefit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-554) BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
[
https://issues.apache.org/jira/browse/HDFS-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J reassigned HDFS-554:
Assignee: Harsh J
BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
--
Key: HDFS-554
URL: https://issues.apache.org/jira/browse/HDFS-554
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.21.0
Reporter: Steve Loughran
Assignee: Harsh J
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-554.patch
BlockInfo.ensureCapacity() uses a for() loop to copy the old array data into
the expanded array. {{System.arraycopy()}} is generally much faster for
this, as it can do a bulk memory copy. There is also the typesafe Java6
{{Arrays.copyOf()}} to consider, though here it offers no tangible benefit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2517) Protobuf - Add protocol service for JournalProtocol
[ https://issues.apache.org/jira/browse/HDFS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154248#comment-13154248 ] Suresh Srinivas commented on HDFS-2517: --- Test failure is unrelated. Tests not included, as this adds just proto definitions. Existing tests will test this functionality on switching to protobuf. Protobuf - Add protocol service for JournalProtocol --- Key: HDFS-2517 URL: https://issues.apache.org/jira/browse/HDFS-2517 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2517.txt, HDFS-2517.txt This jira adds protobuf service for implementing journal protocol using protocol buffers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2518: -- Status: Patch Available (was: Open) Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2517) Protobuf - Add protocol service for JournalProtocol
[ https://issues.apache.org/jira/browse/HDFS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2517: -- Resolution: Fixed Fix Version/s: 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch. Protobuf - Add protocol service for JournalProtocol --- Key: HDFS-2517 URL: https://issues.apache.org/jira/browse/HDFS-2517 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2517.txt, HDFS-2517.txt This jira adds protobuf service for implementing journal protocol using protocol buffers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2517) Protobuf - Add protocol service for JournalProtocol
[ https://issues.apache.org/jira/browse/HDFS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154258#comment-13154258 ] Hudson commented on HDFS-2517: -- Integrated in Hadoop-Hdfs-trunk-Commit #1374 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1374/]) HDFS-2517. Add protobuf service for JounralProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204544 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/HdfsProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/JournalProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/JournalProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/hdfs.proto Protobuf - Add protocol service for JournalProtocol --- Key: HDFS-2517 URL: https://issues.apache.org/jira/browse/HDFS-2517 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2517.txt, HDFS-2517.txt This jira adds protobuf service for implementing journal protocol using protocol buffers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-554) BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
[
https://issues.apache.org/jira/browse/HDFS-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154260#comment-13154260
]
Hadoop QA commented on HDFS-554:
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504505/HDFS-554.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.fs.viewfs.TestViewFsHdfs
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1593//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1593//console
This message is automatically generated.
BlockInfo.ensureCapacity may get a speedup from System.arraycopy()
--
Key: HDFS-554
URL: https://issues.apache.org/jira/browse/HDFS-554
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.21.0
Reporter: Steve Loughran
Assignee: Harsh J
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-554.patch
BlockInfo.ensureCapacity() uses a for() loop to copy the old array data into
the expanded array. {{System.arraycopy()}} is generally much faster for
this, as it can do a bulk memory copy. There is also the typesafe Java6
{{Arrays.copyOf()}} to consider, though here it offers no tangible benefit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2517) Protobuf - Add protocol service for JournalProtocol
[ https://issues.apache.org/jira/browse/HDFS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154261#comment-13154261 ] Hudson commented on HDFS-2517: -- Integrated in Hadoop-Common-trunk-Commit #1301 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1301/]) HDFS-2517. Add protobuf service for JounralProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204544 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/HdfsProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/JournalProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/JournalProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/hdfs.proto Protobuf - Add protocol service for JournalProtocol --- Key: HDFS-2517 URL: https://issues.apache.org/jira/browse/HDFS-2517 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2517.txt, HDFS-2517.txt This jira adds protobuf service for implementing journal protocol using protocol buffers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2517) Protobuf - Add protocol service for JournalProtocol
[ https://issues.apache.org/jira/browse/HDFS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154272#comment-13154272 ] Hudson commented on HDFS-2517: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1324 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1324/]) HDFS-2517. Add protobuf service for JounralProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204544 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/HdfsProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/JournalProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/JournalProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/hdfs.proto Protobuf - Add protocol service for JournalProtocol --- Key: HDFS-2517 URL: https://issues.apache.org/jira/browse/HDFS-2517 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2517.txt, HDFS-2517.txt This jira adds protobuf service for implementing journal protocol using protocol buffers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154275#comment-13154275 ] Owen O'Malley commented on HDFS-2518: - Please don't include the generated code in subversion. We already require protoc in order to compile Hadoop, so there isn't any need to avoid it here. Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2519) Protobuf - Add protobuf service for DatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2519: -- Status: Patch Available (was: Open) Protobuf - Add protobuf service for DatanodeProtocol Key: HDFS-2519 URL: https://issues.apache.org/jira/browse/HDFS-2519 Project: Hadoop HDFS Issue Type: New Feature Components: data-node, name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2519.txt, HDFS-2519.txt, HDFS-2519.txt This jira adds a protobuf service definition for DatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2520) Protobuf - Add protobuf service for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2520: -- Status: Patch Available (was: Open) Protobuf - Add protobuf service for InterDatanodeProtocol - Key: HDFS-2520 URL: https://issues.apache.org/jira/browse/HDFS-2520 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2520.txt, HDFS-2520.txt This jira adds a protobuf service definition for InterDatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2520) Protobuf - Add protobuf service for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154318#comment-13154318 ] Hadoop QA commented on HDFS-2520: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501855/HDFS-2520.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.fs.viewfs.TestViewFsHdfs org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1595//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1595//console This message is automatically generated. Protobuf - Add protobuf service for InterDatanodeProtocol - Key: HDFS-2520 URL: https://issues.apache.org/jira/browse/HDFS-2520 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2520.txt, HDFS-2520.txt This jira adds a protobuf service definition for InterDatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2415) Move MiniDFS out of test JAR and into the main hadoop-hdfs JAR
[ https://issues.apache.org/jira/browse/HDFS-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154320#comment-13154320 ] Alejandro Abdelnur commented on HDFS-2415: -- So far downstream projects has been using Mini DFS/Cluster successfully for testing by including the Hadoop TEST JARs in their classpath. The Hadoop TEST JARs are being published to the Maven repos, thus easily available. If we move the miniclusters somewhere it should be to a 'test tools' module where they would live on their own without all the Hadoop testcases, I would not move it to the main JARs. If there is a problem with logging with the Mini clusters we should fix that. Move MiniDFS out of test JAR and into the main hadoop-hdfs JAR -- Key: HDFS-2415 URL: https://issues.apache.org/jira/browse/HDFS-2415 Project: Hadoop HDFS Issue Type: Improvement Components: test Reporter: Steve Loughran Priority: Minor This is just an idea: move the MiniDFS cluster out of the Hadoop test JAR and into the main redistributable. This would make it easier for people downstream to use it. It is the easiest way to bring up a DFS cluster in a single JVM, and with the MiniMR cluster is a common way to test MR jobs in the IDE against small datasets. Moving the file while keeping the package name should not cause any problems; all it will do is make applying outstanding patches to it slightly harder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2519) Protobuf - Add protobuf service for DatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154336#comment-13154336 ] Hadoop QA commented on HDFS-2519: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501852/HDFS-2519.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1596//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1596//console This message is automatically generated. Protobuf - Add protobuf service for DatanodeProtocol Key: HDFS-2519 URL: https://issues.apache.org/jira/browse/HDFS-2519 Project: Hadoop HDFS Issue Type: New Feature Components: data-node, name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2519.txt, HDFS-2519.txt, HDFS-2519.txt This jira adds a protobuf service definition for DatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2576) Namenode should have a favored nodes hint to enable clients to have control over block placement.
Namenode should have a favored nodes hint to enable clients to have control over block placement. - Key: HDFS-2576 URL: https://issues.apache.org/jira/browse/HDFS-2576 Project: Hadoop HDFS Issue Type: New Feature Reporter: Pritam Damania Sometimes Clients like HBase are required to dynamically compute the datanodes it wishes to place the blocks for a file for higher level of locality. For this purpose there is a need of a way to give the Namenode a hint in terms of a favoredNodes parameter about the locations where the client wants to put each block. The proposed solution is a favored nodes parameter in the addBlock() method and in the create() file method to enable the clients to give the hints to the NameNode about the locations of each replica of the block. Note that this would be just a hint and finally the NameNode would look at disk usage, datanode load etc. and decide whether it can respect the hints or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HDFS-2246: --- Target Version/s: 0.20.205.1, 0.24.0, 0.23.1 (was: 0.23.1, 0.24.0, 0.20.205.1) Status: Open (was: Patch Available) Shortcut a local client reads to a Datanodes files directly --- Key: HDFS-2246 URL: https://issues.apache.org/jira/browse/HDFS-2246 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Attachments: 0001-HDFS-347.-Local-reads.patch, HDFS-2246-branch-0.20-security-205.1.patch, HDFS-2246-branch-0.20-security-205.2.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security.3.patch, HDFS-2246-branch-0.20-security.no-softref.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt, HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java, localReadShortcut20-security.2patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HDFS-2246: --- Attachment: HDFS-2246-trunk.patch Updated patch fixing the merge issue. Shortcut a local client reads to a Datanodes files directly --- Key: HDFS-2246 URL: https://issues.apache.org/jira/browse/HDFS-2246 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Attachments: 0001-HDFS-347.-Local-reads.patch, HDFS-2246-branch-0.20-security-205.1.patch, HDFS-2246-branch-0.20-security-205.2.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security.3.patch, HDFS-2246-branch-0.20-security.no-softref.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt, HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java, localReadShortcut20-security.2patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HDFS-2246: --- Target Version/s: 0.20.205.1, 0.24.0, 0.23.1 (was: 0.23.1, 0.24.0, 0.20.205.1) Status: Patch Available (was: Open) Shortcut a local client reads to a Datanodes files directly --- Key: HDFS-2246 URL: https://issues.apache.org/jira/browse/HDFS-2246 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Attachments: 0001-HDFS-347.-Local-reads.patch, HDFS-2246-branch-0.20-security-205.1.patch, HDFS-2246-branch-0.20-security-205.2.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security.3.patch, HDFS-2246-branch-0.20-security.no-softref.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt, HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java, localReadShortcut20-security.2patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2575: -- Resolution: Fixed Fix Version/s: 0.23.1 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154377#comment-13154377 ] Hudson commented on HDFS-2575: -- Integrated in Hadoop-Hdfs-trunk-Commit #1376 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1376/]) HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204652 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154379#comment-13154379 ] Hudson commented on HDFS-2575: -- Integrated in Hadoop-Common-0.23-Commit #198 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/198/]) HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204651 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154384#comment-13154384 ] Hudson commented on HDFS-2575: -- Integrated in Hadoop-Hdfs-0.23-Commit #195 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/195/]) HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204651 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154381#comment-13154381 ] Hudson commented on HDFS-2575: -- Integrated in Hadoop-Common-trunk-Commit #1303 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1303/]) HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204652 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154383#comment-13154383 ] Hudson commented on HDFS-2575: -- Integrated in Hadoop-Mapreduce-0.23-Commit #209 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/209/]) HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204651 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2566: -- Attachment: hdfs-2566-branch-0.23.txt Here's the patch for 0.23 (a little different since trunk has the Deletion reports feature). I inspected the diff between the two patches for trunk vs 23 and made sure there's nothing else differing. Also checked that the - lines from DataNode.java diff cleanly against the + lines in BPOfferService.java. Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2566: -- Resolution: Fixed Fix Version/s: 0.23.1 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 23 and trunk (the reason to put this in 23 is so future backports and the HA merge are easier) Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2575) DFSTestUtil may create empty files
[ https://issues.apache.org/jira/browse/HDFS-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154398#comment-13154398 ] Hudson commented on HDFS-2575: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1326 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1326/]) HDFS-2575. DFSTestUtil may create empty files. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204652 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java DFSTestUtil may create empty files -- Key: HDFS-2575 URL: https://issues.apache.org/jira/browse/HDFS-2575 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2575.txt DFSTestUtil creates files with random sizes, but there is no minimum size. So, sometimes, it can make a file with length 0. This will cause tests that use this functionality to fail - for example, TestListCorruptFileBlocks assumes that each of the created files has at least one block. We should add a minSize parameter to prevent this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154402#comment-13154402 ] Hudson commented on HDFS-2566: -- Integrated in Hadoop-Common-0.23-Commit #199 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/199/]) HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204660 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataBlockScanner.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestRefreshNamenodes.java Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154400#comment-13154400 ] Hudson commented on HDFS-2566: -- Integrated in Hadoop-Hdfs-0.23-Commit #196 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/196/]) HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204660 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataBlockScanner.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestRefreshNamenodes.java Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154404#comment-13154404 ] Hudson commented on HDFS-2566: -- Integrated in Hadoop-Hdfs-trunk-Commit #1377 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1377/]) HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204659 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataBlockScanner.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestRefreshNamenodes.java Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154405#comment-13154405 ] Hudson commented on HDFS-2566: -- Integrated in Hadoop-Common-trunk-Commit #1304 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1304/]) HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204659 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataBlockScanner.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestRefreshNamenodes.java Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154406#comment-13154406 ] Hudson commented on HDFS-2566: -- Integrated in Hadoop-Mapreduce-0.23-Commit #210 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/210/]) HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204660 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataBlockScanner.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestRefreshNamenodes.java Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2454) Move maxXceiverCount check to before starting the thread in dataXceiver
[ https://issues.apache.org/jira/browse/HDFS-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154409#comment-13154409 ] Uma Maheswara Rao G commented on HDFS-2454: --- Thanks Harsh, for the patch. I think patch should be updated for trunk as well. Move maxXceiverCount check to before starting the thread in dataXceiver --- Key: HDFS-2454 URL: https://issues.apache.org/jira/browse/HDFS-2454 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Harsh J Priority: Minor Fix For: 0.23.1 Attachments: HDFS-2454.patch We can hoist the maxXceiverCount out of DataXceiverServer#run, there's no need to check each time we accept a connection, we can accept when we create a thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2454) Move maxXceiverCount check to before starting the thread in dataXceiver
[ https://issues.apache.org/jira/browse/HDFS-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154415#comment-13154415 ] Harsh J commented on HDFS-2454: --- Thank *you* for reporting this :) The patch was applied already to trunk and branch-0.23. Not sure if I understood what you are asking for? Move maxXceiverCount check to before starting the thread in dataXceiver --- Key: HDFS-2454 URL: https://issues.apache.org/jira/browse/HDFS-2454 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.0 Reporter: Uma Maheswara Rao G Assignee: Harsh J Priority: Minor Fix For: 0.23.1 Attachments: HDFS-2454.patch We can hoist the maxXceiverCount out of DataXceiverServer#run, there's no need to check each time we accept a connection, we can accept when we create a thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2566) Move BPOfferService to be a non-inner class
[ https://issues.apache.org/jira/browse/HDFS-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154416#comment-13154416 ] Hudson commented on HDFS-2566: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1327 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1327/]) HDFS-2566. Move BPOfferService to be a non-inner class. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204659 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataBlockScanner.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeRegister.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestRefreshNamenodes.java Move BPOfferService to be a non-inner class --- Key: HDFS-2566 URL: https://issues.apache.org/jira/browse/HDFS-2566 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2566-branch-0.23.txt, hdfs-2566.txt, hdfs-2566.txt, hdfs-2566.txt Rounding out the cleanup of BPOfferService, it would be good to move it to its own file, so it's no longer an inner class. DataNode.java is really large and hard to navigate. BPOfferService itself is ~700 lines, so seems like a large enough unit to merit its own file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2573) TestFiDataXceiverServer is failing, not testing OOME
[ https://issues.apache.org/jira/browse/HDFS-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2573: -- Attachment: HDFS-2573.patch TestFiDataXceiverServer is failing, not testing OOME Key: HDFS-2573 URL: https://issues.apache.org/jira/browse/HDFS-2573 Project: Hadoop HDFS Issue Type: Bug Components: data-node, test Affects Versions: 0.22.0 Reporter: Konstantin Shvachko Assignee: Konstantin Boudnik Fix For: 0.22.0 Attachments: HDFS-2573.patch, HDFS-2573.patch TestFiDataXceiverServer is failing occasionally. It turns out also that the test is not testing the desired condition, because OOME is not caught in DataXceiverServer.run() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2573) TestFiDataXceiverServer is failing, not testing OOME
[ https://issues.apache.org/jira/browse/HDFS-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154419#comment-13154419 ] Uma Maheswara Rao G commented on HDFS-2573: --- Thanks Cos, for the patch. This needs to be corrected in trunk as well. Here is a patch for trunk. TestFiDataXceiverServer is failing, not testing OOME Key: HDFS-2573 URL: https://issues.apache.org/jira/browse/HDFS-2573 Project: Hadoop HDFS Issue Type: Bug Components: data-node, test Affects Versions: 0.22.0 Reporter: Konstantin Shvachko Assignee: Konstantin Boudnik Fix For: 0.22.0 Attachments: HDFS-2573.patch, HDFS-2573.patch TestFiDataXceiverServer is failing occasionally. It turns out also that the test is not testing the desired condition, because OOME is not caught in DataXceiverServer.run() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: h2552_2021.pdf h2552_2021.patch h2552_2021.patch h2552_2021.pdf Here is the final patch. Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2577) HA: NN fails to start since it tries to start secret manager in safemode
HA: NN fails to start since it tries to start secret manager in safemode Key: HDFS-2577 URL: https://issues.apache.org/jira/browse/HDFS-2577 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Todd Lipcon -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2577) HA: NN fails to start since it tries to start secret manager in safemode
[ https://issues.apache.org/jira/browse/HDFS-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2577: -- Component/s: name-node Description: After HDFS-2301, the NN fails to start with the following: Caused by: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot log master key update in safe mode. Name node is in safe mode. The reported blocks 0 needs additional 5 blocks to reach the threshold 1. of total blocks 4. Safe mode will be turned off automatically. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.logUpdateMasterKey(FSNamesystem.java:4259) at org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSecretManager.logUpdateMasterKey(DelegationTokenSecretManager.java:285) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.updateCurrentKey(AbstractDelegationTokenSecretManager.java:143) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.startThreads(AbstractDelegationTokenSecretManager.java:98) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startSecretManager(FSNamesystem.java:386) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:440) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:937) at org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:57) Affects Version/s: HA branch (HDFS-1623) Assignee: Todd Lipcon HA: NN fails to start since it tries to start secret manager in safemode Key: HDFS-2577 URL: https://issues.apache.org/jira/browse/HDFS-2577 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: HA branch (HDFS-1623) Reporter: Todd Lipcon Assignee: Todd Lipcon After HDFS-2301, the NN fails to start with the following: Caused by: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot log master key update in safe mode. Name node is in safe mode. The reported blocks 0 needs additional 5 blocks to reach the threshold 1. of total blocks 4. Safe mode will be turned off automatically. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.logUpdateMasterKey(FSNamesystem.java:4259) at org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSecretManager.logUpdateMasterKey(DelegationTokenSecretManager.java:285) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.updateCurrentKey(AbstractDelegationTokenSecretManager.java:143) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.startThreads(AbstractDelegationTokenSecretManager.java:98) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startSecretManager(FSNamesystem.java:386) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:440) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:937) at org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:57) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J reassigned HDFS-2574: - Assignee: Harsh J remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Harsh J Priority: Trivial Attachments: HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-2574: -- Attachment: HDFS-2574.patch Done and done. Please lemme know if this covers this one! remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Priority: Trivial Attachments: HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154477#comment-13154477 ] Joe Crobak commented on HDFS-2574: -- I think there are a few other ill-references, but I have that info on another computer. can get back to you tonight. remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Harsh J Priority: Trivial Attachments: HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HDFS-2574: -- Attachment: HDFS-2574.patch Missed adding the default xml changeset. remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Harsh J Priority: Trivial Attachments: HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2574) remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154481#comment-13154481 ] Harsh J commented on HDFS-2574: --- Joe, Sure, would be good to have that if you have some noted down already! remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml - Key: HDFS-2574 URL: https://issues.apache.org/jira/browse/HDFS-2574 Project: Hadoop HDFS Issue Type: Task Components: documentation Affects Versions: 0.23.0 Reporter: Joe Crobak Assignee: Harsh J Priority: Trivial Attachments: HDFS-2574.patch, HDFS-2574.patch Some examples: hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml contains an entry for dfs.name.dir rather than dfs.namenode.name.dir and hdfs-default.xml references dfs.name.dir twice in description tags rather than using dfs.namenode.name.dir. List of deprecated properties is here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2578) Use protoc to generate protobuf artifacts instead of checking it in.
Use protoc to generate protobuf artifacts instead of checking it in. Key: HDFS-2578 URL: https://issues.apache.org/jira/browse/HDFS-2578 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154495#comment-13154495 ] Suresh Srinivas commented on HDFS-2518: --- Test failure TestViewFsHdfs is unrelated to this patch. Also this patch introduces protobuf definitions. It will be tested when protobuf mechanism is used for RPCs, using the existing tests. Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2518: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch. Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2536) Remove unused imports
[
https://issues.apache.org/jira/browse/HDFS-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-2536:
--
Attachment: 0.23-HDFS-2536.patch
Patch for 0.23. Thanks for review+commit of the previous one Eli!
Verified that {{mvn clean compile}} passes.
Remove unused imports
-
Key: HDFS-2536
URL: https://issues.apache.org/jira/browse/HDFS-2536
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.24.0
Reporter: Aaron T. Myers
Assignee: Harsh J
Priority: Trivial
Labels: newbie
Attachments: 0.23-HDFS-2536.patch,
HDFS-2536.FSImageTransactionalStorageInspector.patch, HDFS-2536.patch
Looks like it has 11 unused imports by my count.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2579) Starting delegation token manager during safemode fails
Starting delegation token manager during safemode fails
---
Key: HDFS-2579
URL: https://issues.apache.org/jira/browse/HDFS-2579
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node, security
Affects Versions: 0.23.0
Reporter: Todd Lipcon
I noticed this on the HA branch, but it seems to actually affect non-HA branch
0.23 if security is enabled. When the NN starts up, if security is enabled, we
start the delegation token secret manager, which then tries to call
{{logUpdateMasterKey}}. This fails because the edit logs may not be written
while in safe-mode.
It seems to me that there's not any necessary reason that you have to make a
new master key at startup, since you've loaded the old key when you load the
FSImage. You'd only be lacking a DT master key on a fresh cluster, in which
case we could have it generate one at format time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[
https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154511#comment-13154511
]
Tsz Wo (Nicholas), SZE commented on HDFS-2246:
--
- The value should also be updated
{code}
+ public static final String DFS_CLIENT_READ_SKIP_CHECKSUM_KEY =
dfs.client.read.shortcircuit.checksum;
{code}
- Some public methods do not have javadoc.
Shortcut a local client reads to a Datanodes files directly
---
Key: HDFS-2246
URL: https://issues.apache.org/jira/browse/HDFS-2246
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Attachments: 0001-HDFS-347.-Local-reads.patch,
HDFS-2246-branch-0.20-security-205.1.patch,
HDFS-2246-branch-0.20-security-205.2.patch,
HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security.3.patch,
HDFS-2246-branch-0.20-security.no-softref.patch,
HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch,
HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch,
HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch,
HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt,
HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java,
localReadShortcut20-security.2patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154517#comment-13154517 ] Suresh Srinivas commented on HDFS-2518: --- Owen, HDFS-2578 is created to address your comment. Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154523#comment-13154523 ] Hudson commented on HDFS-2518: -- Integrated in Hadoop-Hdfs-trunk-Commit #1380 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1380/]) HDFS-2518. Add protobuf service for NamenodeProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204707 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/NamenodeProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/NamenodeProtocol.proto Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[
https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154521#comment-13154521
]
Harsh J commented on HDFS-2246:
---
Could boolean-valued config names be suffixed with a {{.enabled}} for clarity?
Shortcut a local client reads to a Datanodes files directly
---
Key: HDFS-2246
URL: https://issues.apache.org/jira/browse/HDFS-2246
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Attachments: 0001-HDFS-347.-Local-reads.patch,
HDFS-2246-branch-0.20-security-205.1.patch,
HDFS-2246-branch-0.20-security-205.2.patch,
HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security.3.patch,
HDFS-2246-branch-0.20-security.no-softref.patch,
HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch,
HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch,
HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch,
HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt,
HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java,
localReadShortcut20-security.2patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2520) Protobuf - Add protobuf service for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2520: -- Resolution: Fixed Fix Version/s: 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) The test failures are unrelated to this patch. Also this functionality will be tested as part of tests that already exists when RPC transitions to protobuf. I committed the patch. Protobuf - Add protobuf service for InterDatanodeProtocol - Key: HDFS-2520 URL: https://issues.apache.org/jira/browse/HDFS-2520 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2520.txt, HDFS-2520.txt This jira adds a protobuf service definition for InterDatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2536) Remove unused imports
[ https://issues.apache.org/jira/browse/HDFS-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154524#comment-13154524 ] Hadoop QA commented on HDFS-2536: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504556/0.23-HDFS-2536.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 93 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1598//console This message is automatically generated. Remove unused imports - Key: HDFS-2536 URL: https://issues.apache.org/jira/browse/HDFS-2536 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Aaron T. Myers Assignee: Harsh J Priority: Trivial Labels: newbie Attachments: 0.23-HDFS-2536.patch, HDFS-2536.FSImageTransactionalStorageInspector.patch, HDFS-2536.patch Looks like it has 11 unused imports by my count. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154527#comment-13154527 ] Hudson commented on HDFS-2518: -- Integrated in Hadoop-Common-trunk-Commit #1307 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1307/]) HDFS-2518. Add protobuf service for NamenodeProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204707 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/NamenodeProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/NamenodeProtocol.proto Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2577) HA: NN fails to start since it tries to start secret manager in safemode
[ https://issues.apache.org/jira/browse/HDFS-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2577: -- Attachment: hdfs-2577.txt Attached patch changes back to the trunk behavior: only start the DT manager if security is enabled. Also fixed another related bug I noticed while testing this fix -- with HDFS-2301 the lease manager stop was always timing out, since it was looping on namesystem.isRunning() which continues to be true even in standby mode. Added a flag to LeaseManager to decide whether the monitor should run. Tests seem to be passing now. HA: NN fails to start since it tries to start secret manager in safemode Key: HDFS-2577 URL: https://issues.apache.org/jira/browse/HDFS-2577 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: HA branch (HDFS-1623) Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2577.txt After HDFS-2301, the NN fails to start with the following: Caused by: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot log master key update in safe mode. Name node is in safe mode. The reported blocks 0 needs additional 5 blocks to reach the threshold 1. of total blocks 4. Safe mode will be turned off automatically. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.logUpdateMasterKey(FSNamesystem.java:4259) at org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSecretManager.logUpdateMasterKey(DelegationTokenSecretManager.java:285) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.updateCurrentKey(AbstractDelegationTokenSecretManager.java:143) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.startThreads(AbstractDelegationTokenSecretManager.java:98) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startSecretManager(FSNamesystem.java:386) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:440) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:937) at org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:57) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2580) NameNode#main(...) can make use of GenericOptionsParser.
NameNode#main(...) can make use of GenericOptionsParser.
Key: HDFS-2580
URL: https://issues.apache.org/jira/browse/HDFS-2580
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Harsh J
Priority: Minor
Fix For: 0.24.0
DataNode supports passing generic opts when calling via {{hdfs datanode}}.
NameNode can support the same thing as well, but doesn't right now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2520) Protobuf - Add protobuf service for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154545#comment-13154545 ] Hudson commented on HDFS-2520: -- Integrated in Hadoop-Common-trunk-Commit #1308 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1308/]) HDFS-2520. Add protobuf service for InterDatanodeProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204709 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/InterDatanodeProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/InterDatanodeProtocol.proto Protobuf - Add protobuf service for InterDatanodeProtocol - Key: HDFS-2520 URL: https://issues.apache.org/jira/browse/HDFS-2520 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2520.txt, HDFS-2520.txt This jira adds a protobuf service definition for InterDatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2520) Protobuf - Add protobuf service for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154541#comment-13154541 ] Hudson commented on HDFS-2520: -- Integrated in Hadoop-Hdfs-trunk-Commit #1381 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1381/]) HDFS-2520. Add protobuf service for InterDatanodeProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204709 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/InterDatanodeProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/InterDatanodeProtocol.proto Protobuf - Add protobuf service for InterDatanodeProtocol - Key: HDFS-2520 URL: https://issues.apache.org/jira/browse/HDFS-2520 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2520.txt, HDFS-2520.txt This jira adds a protobuf service definition for InterDatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2580) NameNode#main(...) can make use of GenericOptionsParser.
[
https://issues.apache.org/jira/browse/HDFS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-2580:
--
Status: Patch Available (was: Open)
NameNode#main(...) can make use of GenericOptionsParser.
Key: HDFS-2580
URL: https://issues.apache.org/jira/browse/HDFS-2580
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-2580.patch
DataNode supports passing generic opts when calling via {{hdfs datanode}}.
NameNode can support the same thing as well, but doesn't right now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2580) NameNode#main(...) can make use of GenericOptionsParser.
[
https://issues.apache.org/jira/browse/HDFS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J reassigned HDFS-2580:
-
Assignee: Harsh J
NameNode#main(...) can make use of GenericOptionsParser.
Key: HDFS-2580
URL: https://issues.apache.org/jira/browse/HDFS-2580
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-2580.patch
DataNode supports passing generic opts when calling via {{hdfs datanode}}.
NameNode can support the same thing as well, but doesn't right now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2580) NameNode#main(...) can make use of GenericOptionsParser.
[
https://issues.apache.org/jira/browse/HDFS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-2580:
--
Attachment: HDFS-2580.patch
NameNode#main(...) can make use of GenericOptionsParser.
Key: HDFS-2580
URL: https://issues.apache.org/jira/browse/HDFS-2580
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Harsh J
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-2580.patch
DataNode supports passing generic opts when calling via {{hdfs datanode}}.
NameNode can support the same thing as well, but doesn't right now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: h2552_2021_0.20s.pdf h2552_2021_0.20s.patch Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf, h2552_2021_0.20s.patch, h2552_2021_0.20s.patch, h2552_2021_0.20s.pdf, h2552_2021_0.20s.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2580) NameNode#main(...) can make use of GenericOptionsParser.
[
https://issues.apache.org/jira/browse/HDFS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154552#comment-13154552
]
Harsh J commented on HDFS-2580:
---
Or perhaps we restructure NN/DN to use the Tool interface?
NameNode#main(...) can make use of GenericOptionsParser.
Key: HDFS-2580
URL: https://issues.apache.org/jira/browse/HDFS-2580
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.23.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 0.24.0
Attachments: HDFS-2580.patch
DataNode supports passing generic opts when calling via {{hdfs datanode}}.
NameNode can support the same thing as well, but doesn't right now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: (was: h2552_2021_0.20s.patch) Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: (was: h2552_2021_0.20s.patch) Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: (was: h2552_2021_0.20s.pdf) Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: (was: h2552_2021_0.20s.pdf) Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: h2552_2021.pdf h2552_2021.patch Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.patch, h2552_2021.pdf, h2552_2021.pdf, h2552_2021_0.20s.patch, h2552_2021_0.20s.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: h2552_2021_0.20s.pdf h2552_2021_0.20s.patch Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.patch, h2552_2021.pdf, h2552_2021.pdf, h2552_2021_0.20s.patch, h2552_2021_0.20s.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: (was: h2552_2021.pdf) Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf, h2552_2021_0.20s.patch, h2552_2021_0.20s.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2552) Add WebHdfs Forrest doc
[ https://issues.apache.org/jira/browse/HDFS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2552: - Attachment: (was: h2552_2021.patch) Add WebHdfs Forrest doc --- Key: HDFS-2552 URL: https://issues.apache.org/jira/browse/HDFS-2552 Project: Hadoop HDFS Issue Type: Task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h2552_2015.patch, h2552_2015b.patch, h2552_2015b.pdf, h2552_2017.patch, h2552_2017.pdf, h2552_2021.patch, h2552_2021.pdf, h2552_2021_0.20s.patch, h2552_2021_0.20s.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2578) Use protoc to generate protobuf artifacts instead of checking it in.
[ https://issues.apache.org/jira/browse/HDFS-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154562#comment-13154562 ] Doug Cutting commented on HDFS-2578: This requires that everyone who builds and every Jenkins box has the right version of protoc installed, a version that matches the protobuf java library in use. I suggest testing this explicitly in the build, by checking 'protoc --version' against the value in Hadoop's pom.xml, failing the build when they don't match. Use protoc to generate protobuf artifacts instead of checking it in. Key: HDFS-2578 URL: https://issues.apache.org/jira/browse/HDFS-2578 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-107) Data-nodes should be formatted when the name-node is formatted.
[ https://issues.apache.org/jira/browse/HDFS-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154563#comment-13154563 ] Harsh J commented on HDFS-107: -- Reformatting a namenode comes with responsibilities right (how often do you format anyway, and why?)? Why can't we just leave it to the ops/users to clean their dirs up themselves, or switch to new dirnames? Adding such a thing, I feel, is potentially dangerous. Data-nodes should be formatted when the name-node is formatted. --- Key: HDFS-107 URL: https://issues.apache.org/jira/browse/HDFS-107 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0 Reporter: Konstantin Shvachko Attachments: HDFS-107-1.patch The upgrade feature HADOOP-702 requires data-nodes to store persistently the namespaceID in their version files and verify during startup that it matches the one stored on the name-node. When the name-node reformats it generates a new namespaceID. Now if the cluster starts with the reformatted name-node, and not reformatted data-nodes the data-nodes will fail with java.io.IOException: Incompatible namespaceIDs ... Data-nodes should be reformatted whenever the name-node is. I see 2 approaches here: 1) In order to reformat the cluster we call start-dfs -format or make a special script format-dfs. This would format the cluster components all together. The question is whether it should start the cluster after formatting? 2) Format the name-node only. When data-nodes connect to the name-node it will tell them to format their storage directories if it sees that the namespace is empty and its cTime=0. The drawback of this approach is that we can loose blocks of a data-node from another cluster if it connects by mistake to the empty name-node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
[ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154568#comment-13154568 ] Tsz Wo (Nicholas), SZE commented on HDFS-2246: -- Hi Harsh, the key is supposed to be dfs.client.read.shortcircuit.*skip*.checksum. It does not sound right to rename it to dfs.client.read.shortcircuit.*skip.checksum.enabled*. Shortcut a local client reads to a Datanodes files directly --- Key: HDFS-2246 URL: https://issues.apache.org/jira/browse/HDFS-2246 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Attachments: 0001-HDFS-347.-Local-reads.patch, HDFS-2246-branch-0.20-security-205.1.patch, HDFS-2246-branch-0.20-security-205.2.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security.3.patch, HDFS-2246-branch-0.20-security.no-softref.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246-trunk.patch, HDFS-2246.20s.1.patch, HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt, HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, TestShortCircuitLocalRead.java, localReadShortcut20-security.2patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2536) Remove unused imports
[ https://issues.apache.org/jira/browse/HDFS-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-2536: -- Resolution: Fixed Status: Resolved (was: Patch Available) I've committed to 23. Thanks Harsh! Remove unused imports - Key: HDFS-2536 URL: https://issues.apache.org/jira/browse/HDFS-2536 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Aaron T. Myers Assignee: Harsh J Priority: Trivial Labels: newbie Attachments: 0.23-HDFS-2536.patch, HDFS-2536.FSImageTransactionalStorageInspector.patch, HDFS-2536.patch Looks like it has 11 unused imports by my count. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2536) Remove unused imports
[ https://issues.apache.org/jira/browse/HDFS-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154572#comment-13154572 ] Hudson commented on HDFS-2536: -- Integrated in Hadoop-Hdfs-0.23-Commit #199 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/199/]) HDFS-2536. Remove unused imports. Contributed by Harsh J eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204723 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeID.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ReplicaInPipeline.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupJournalManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/DfsServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageTransactionalStorageInspector.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileChecksumServlets.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileDataServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/RenewDelegationTokenServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/RemoteEditLog.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DelegationTokenFetcher.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsLoader.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsVisitor.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsVisitorFactory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/TokenizerFactory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/DirectBufferPool.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/CLITestHelperDFS.java *
[jira] [Commented] (HDFS-2520) Protobuf - Add protobuf service for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154575#comment-13154575 ] Hudson commented on HDFS-2520: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1330 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1330/]) HDFS-2520. Add protobuf service for InterDatanodeProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204709 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/InterDatanodeProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/InterDatanodeProtocol.proto Protobuf - Add protobuf service for InterDatanodeProtocol - Key: HDFS-2520 URL: https://issues.apache.org/jira/browse/HDFS-2520 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2520.txt, HDFS-2520.txt This jira adds a protobuf service definition for InterDatanodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2518) Protobuf - Add protobuf service for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154576#comment-13154576 ] Hudson commented on HDFS-2518: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1330 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1330/]) HDFS-2518. Add protobuf service for NamenodeProtocol. Contributed by Suresh Srinivas. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204707 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/NamenodeProtocolProtos.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/proto/NamenodeProtocol.proto Protobuf - Add protobuf service for NamenodeProtocol Key: HDFS-2518 URL: https://issues.apache.org/jira/browse/HDFS-2518 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2518.txt, HDFS-2518.txt, HDFS-2518.txt This jira adds a protobuf service definition for NamenodeProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2536) Remove unused imports
[ https://issues.apache.org/jira/browse/HDFS-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13154577#comment-13154577 ] Hudson commented on HDFS-2536: -- Integrated in Hadoop-Mapreduce-0.23-Commit #213 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/213/]) HDFS-2536. Remove unused imports. Contributed by Harsh J eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1204723 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeID.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ReplicaInPipeline.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupJournalManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/DfsServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageTransactionalStorageInspector.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileChecksumServlets.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileDataServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/RenewDelegationTokenServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/RemoteEditLog.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DelegationTokenFetcher.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsLoader.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsVisitor.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/EditsVisitorFactory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/TokenizerFactory.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/DirectBufferPool.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/CLITestHelperDFS.java *
[jira] [Updated] (HDFS-2410) Further clean up hard-coded configuration keys
[ https://issues.apache.org/jira/browse/HDFS-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2410: -- Attachment: HDFS-2410.txt Further clean up hard-coded configuration keys -- Key: HDFS-2410 URL: https://issues.apache.org/jira/browse/HDFS-2410 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node, test Reporter: Suresh Srinivas Assignee: Suresh Srinivas Priority: Minor Attachments: HDFS-2410.txt, HDFS-2410.txt HDFS code is littered with hardcoded config key names. This jira changes to use DFSConfigKeys constants. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
