[jira] [Commented] (HDFS-4902) DFSClient.getSnapshotDiffReport should use string path rather than o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HDFS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13681941#comment-13681941
]
Jing Zhao commented on HDFS-4902:
-
By the way, +1 for the patch pending Jenkins.
DFSClient.getSnapshotDiffReport should use string path rather than
o.a.h.fs.Path
Key: HDFS-4902
URL: https://issues.apache.org/jira/browse/HDFS-4902
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.1.0-beta
Reporter: Binglin Chang
Assignee: Binglin Chang
Attachments: HDFS-4902.001.patch, HDFS-4902.patch
{code}
org.apache.hadoop.ipc.RemoteException(java.lang.AssertionError): Absolute
path required
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathNames(INode.java:641)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:619)
at
org.apache.hadoop.hdfs.server.namenode.INodeDirectory.getINodesInPath4Write(INodeDirectory.java:362)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getINodesInPath4Write(FSDirectory.java:1648)
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:354)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:6035)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1172)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolTranslatorPB.java:975)
at
org.apache.hadoop.hdfs.DFSClient.getSnapshotDiffReport(DFSClient.java:2158)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getSnapshotDiffReport(DistributedFileSystem.java:990)
{code}
DistributedFileSystem.getSnapshotDiffReport use Path with scheme, so toString
will return path with scheme, e.g. hdfs://:8020/abc/
But FSNamesystem only accept simple path, not whole URI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4902) DFSClient.getSnapshotDiffReport should use string path rather than o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HDFS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682042#comment-13682042
]
Hadoop QA commented on HDFS-4902:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587565/HDFS-4902.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/4511//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4511//console
This message is automatically generated.
DFSClient.getSnapshotDiffReport should use string path rather than
o.a.h.fs.Path
Key: HDFS-4902
URL: https://issues.apache.org/jira/browse/HDFS-4902
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.1.0-beta
Reporter: Binglin Chang
Assignee: Binglin Chang
Attachments: HDFS-4902.001.patch, HDFS-4902.patch
{code}
org.apache.hadoop.ipc.RemoteException(java.lang.AssertionError): Absolute
path required
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathNames(INode.java:641)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:619)
at
org.apache.hadoop.hdfs.server.namenode.INodeDirectory.getINodesInPath4Write(INodeDirectory.java:362)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getINodesInPath4Write(FSDirectory.java:1648)
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:354)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:6035)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1172)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolTranslatorPB.java:975)
at
org.apache.hadoop.hdfs.DFSClient.getSnapshotDiffReport(DFSClient.java:2158)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getSnapshotDiffReport(DistributedFileSystem.java:990)
{code}
DistributedFileSystem.getSnapshotDiffReport use Path with scheme, so toString
will return path with scheme, e.g. hdfs://:8020/abc/
But FSNamesystem only accept simple path, not whole URI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4878) On Remove Block, Block is not Removed from neededReplications queue
[ https://issues.apache.org/jira/browse/HDFS-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682111#comment-13682111 ] Hudson commented on HDFS-4878: -- Integrated in Hadoop-Yarn-trunk #239 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/239/]) Port HDFS-4878 to branch-0.23.9. (Revision 1492449) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1492449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt On Remove Block, Block is not Removed from neededReplications queue --- Key: HDFS-4878 URL: https://issues.apache.org/jira/browse/HDFS-4878 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Tao Luo Assignee: Tao Luo Fix For: 2.1.0-beta, 0.23.9 Attachments: HDFS-4878.branch-0.23.patch, HDFS-4878_branch2.patch, HDFS-4878.patch Remove block removes the specified block from pendingReplications, but not from neededReplications queue. The fix would be to remove from neededReplications as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4878) On Remove Block, Block is not Removed from neededReplications queue
[ https://issues.apache.org/jira/browse/HDFS-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682202#comment-13682202 ] Hudson commented on HDFS-4878: -- Integrated in Hadoop-Hdfs-0.23-Build #637 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/637/]) HDFS-4878. On Remove Block, block is not removed from neededReplications queue. Contributed by Tao Luo. (Revision 1492448) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1492448 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestMetaSave.java On Remove Block, Block is not Removed from neededReplications queue --- Key: HDFS-4878 URL: https://issues.apache.org/jira/browse/HDFS-4878 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Tao Luo Assignee: Tao Luo Fix For: 2.1.0-beta, 0.23.9 Attachments: HDFS-4878.branch-0.23.patch, HDFS-4878_branch2.patch, HDFS-4878.patch Remove block removes the specified block from pendingReplications, but not from neededReplications queue. The fix would be to remove from neededReplications as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4878) On Remove Block, Block is not Removed from neededReplications queue
[ https://issues.apache.org/jira/browse/HDFS-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682215#comment-13682215 ] Hudson commented on HDFS-4878: -- Integrated in Hadoop-Hdfs-trunk #1429 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1429/]) Port HDFS-4878 to branch-0.23.9. (Revision 1492449) Result = FAILURE shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1492449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt On Remove Block, Block is not Removed from neededReplications queue --- Key: HDFS-4878 URL: https://issues.apache.org/jira/browse/HDFS-4878 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Tao Luo Assignee: Tao Luo Fix For: 2.1.0-beta, 0.23.9 Attachments: HDFS-4878.branch-0.23.patch, HDFS-4878_branch2.patch, HDFS-4878.patch Remove block removes the specified block from pendingReplications, but not from neededReplications queue. The fix would be to remove from neededReplications as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4878) On Remove Block, Block is not Removed from neededReplications queue
[ https://issues.apache.org/jira/browse/HDFS-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682264#comment-13682264 ] Hudson commented on HDFS-4878: -- Integrated in Hadoop-Mapreduce-trunk #1456 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1456/]) Port HDFS-4878 to branch-0.23.9. (Revision 1492449) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1492449 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt On Remove Block, Block is not Removed from neededReplications queue --- Key: HDFS-4878 URL: https://issues.apache.org/jira/browse/HDFS-4878 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Tao Luo Assignee: Tao Luo Fix For: 2.1.0-beta, 0.23.9 Attachments: HDFS-4878.branch-0.23.patch, HDFS-4878_branch2.patch, HDFS-4878.patch Remove block removes the specified block from pendingReplications, but not from neededReplications queue. The fix would be to remove from neededReplications as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682288#comment-13682288 ] Daryn Sharp commented on HDFS-4783: --- Sigh, this is technically masking an underlying bug of a double fwd/rev resolve which can return a different hostname and foul the use of CNAMES. Fixing it is probably more complex than it seems, so +0.5. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows Key: HDFS-4783 URL: https://issues.apache.org/jira/browse/HDFS-4783 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 3.0.0, 2.1.0-beta Reporter: Chris Nauroth Assignee: Chris Nauroth Attachments: HDFS-4783.1.patch This test asserts that delegation tokens previously associated to a host with a resolved IP address no longer match for selection when hadoop.security.token.service.use_ip is set false. The test assumes that 127.0.0.1 resolves to host name localhost. On Windows, this is not the case, and instead it resolves to 127.0.0.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-4888: --- Attachment: HDFS-4888.patch Refactoring patch for trunk. Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.8 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682401#comment-13682401 ] Ravi Prakash commented on HDFS-4888: Kousuke, The code was unnecessarily complex IMHO. I was thinking something more along the lines of what this patch does. Opinions? Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.8 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682407#comment-13682407 ] Ravi Prakash commented on HDFS-4888: The change in behavior is: 1. Prints a It was turned on manually when the safemode was initially entered into manually. 2. Prints the The number of live datanodes %d has reached message when in extension mode. Previously only the The reported blocks %d has reached the threshold message was printed. Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.8 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4849) Idempotent create and append operations.
[
https://issues.apache.org/jira/browse/HDFS-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682421#comment-13682421
]
Konstantin Boudnik commented on HDFS-4849:
--
Wow, hefty tests... The patch looks good - seems to be addressing everything as
discussed above.
+1 with two minor improvements to make:
This seems like a formatting change to me
{noformat}
- // Now we can add the name to the filesystem. This file has no
- // blocks associated with it.
- //
- checkFsObjectLimit();
+// Now we can add the name to the filesystem. This file has no
+// blocks associated with it.
+//
+checkFsObjectLimit();
{noformat}
I understand that originally it was misaligned, but it doesn't look like a part
of the fix.
another whitespace change
{noformat}
+
+ /**
+ * Check if create or append call is a retry.e
{noformat}
I've looked into the failing tests - they look irrelevant to the change.
Idempotent create and append operations.
Key: HDFS-4849
URL: https://issues.apache.org/jira/browse/HDFS-4849
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 2.0.4-alpha
Reporter: Konstantin Shvachko
Assignee: Konstantin Shvachko
Attachments: idempotentCreate.patch, idempotentCreate.patch
create, append and delete operations can be made idempotent. This will reduce
chances for a job or other app failures when NN fails over.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4783) TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682431#comment-13682431 ] Chris Nauroth commented on HDFS-4783: - Daryn, is there an issue for that underlying bug, or should we file a new one? I'm planning on committing this, but I'll wait until end of day in case there are any other objections. TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows Key: HDFS-4783 URL: https://issues.apache.org/jira/browse/HDFS-4783 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 3.0.0, 2.1.0-beta Reporter: Chris Nauroth Assignee: Chris Nauroth Attachments: HDFS-4783.1.patch This test asserts that delegation tokens previously associated to a host with a resolved IP address no longer match for selection when hadoop.security.token.service.use_ip is set false. The test assumes that 127.0.0.1 resolves to host name localhost. On Windows, this is not the case, and instead it resolves to 127.0.0.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4866) Protocol buffer support cannot compile under C
[ https://issues.apache.org/jira/browse/HDFS-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682439#comment-13682439 ] Chris Nauroth commented on HDFS-4866: - I've been trying to track down definitive documentation that tells us whether or not renaming a protobuf service RPC method is backwards-compatible at the protocol layer. Assume a client compiled before this change tried to connect to a server compiled after this change. Would this break, and if so, is there any alternative that achieves backwards-compatibility, or do we just need to accept the unfortunate backwards-incompatibility? I haven't found anything definitive in the protobuf documentation to answer my question. Have you found anything that discusses this? Protocol buffer support cannot compile under C -- Key: HDFS-4866 URL: https://issues.apache.org/jira/browse/HDFS-4866 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.1.0-beta Reporter: Ralph Castain Assignee: Arpit Agarwal Priority: Blocker Attachments: HDFS-4866.branch-2.001.patch, HDFS-4866.trunk.001.patch, NamenodeProtocol.pb-c.c, NamenodeProtocol.pb-c.h, pcreate.pl When compiling Hadoop's .proto descriptions for use in C, an error occurs because one of the RPC's in NamenodeProtocol.proto is named register. This name is a reserved word in languages such as C. When using the Java and C++ languages, the name is hidden inside a class and therefore doesn't cause an error. Unfortunately, that is not the case in non-class languages such as C. Note: generating the C translation of the .proto files requires installation of the protobuf-c package from google: http://code.google.com/p/protobuf-c/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4866) Protocol buffer support cannot compile under C
[ https://issues.apache.org/jira/browse/HDFS-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682452#comment-13682452 ] Arpit Agarwal commented on HDFS-4866: - Hi Chris, Thanks for taking a look. I don't know of a way to avoid the backward incompatibility either. Note that this is the inter-namenode protocol so it should not affect older clients. Protocol buffer support cannot compile under C -- Key: HDFS-4866 URL: https://issues.apache.org/jira/browse/HDFS-4866 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.1.0-beta Reporter: Ralph Castain Assignee: Arpit Agarwal Priority: Blocker Attachments: HDFS-4866.branch-2.001.patch, HDFS-4866.trunk.001.patch, NamenodeProtocol.pb-c.c, NamenodeProtocol.pb-c.h, pcreate.pl When compiling Hadoop's .proto descriptions for use in C, an error occurs because one of the RPC's in NamenodeProtocol.proto is named register. This name is a reserved word in languages such as C. When using the Java and C++ languages, the name is hidden inside a class and therefore doesn't cause an error. Unfortunately, that is not the case in non-class languages such as C. Note: generating the C translation of the .proto files requires installation of the protobuf-c package from google: http://code.google.com/p/protobuf-c/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4900) Print user when services are started
[ https://issues.apache.org/jira/browse/HDFS-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682474#comment-13682474 ] Brandon Li commented on HDFS-4900: -- Looks like Namenode does prints both owner's name and group. Datanode (JournalNode) and Secondarynamenode don't start FSNamesystem, and thus don't print the users name/group. Print user when services are started Key: HDFS-4900 URL: https://issues.apache.org/jira/browse/HDFS-4900 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 3.0.0, 1.3.0 Reporter: Brandon Li Priority: Trivial Printing user name during start can help debug access permission related issues, e.g., the namenode storage directory is not accessible by the user who starts the service. The message could look like: / STARTUP_MSG: Starting NameNode by hdfs ... ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4819) Update Snapshot doc for HDFS-4758
[ https://issues.apache.org/jira/browse/HDFS-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682487#comment-13682487 ] Jing Zhao commented on HDFS-4819: - Also merged to branch-2.1.0-beta. Update Snapshot doc for HDFS-4758 - Key: HDFS-4819 URL: https://issues.apache.org/jira/browse/HDFS-4819 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Priority: Minor Fix For: 2.1.0-beta Attachments: h4819_20130611.patch Update Snapshot doc to clarify that nested snapshots are not allowed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4902) DFSClient.getSnapshotDiffReport should use string path rather than o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HDFS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682511#comment-13682511
]
Hudson commented on HDFS-4902:
--
Integrated in Hadoop-trunk-Commit #3913 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3913/])
HDFS-4902. DFSClient#getSnapshotDiffReport should use string path rather
than o.a.h.fs.Path. Contributed by Binglin Chang. (Revision 1492791)
Result = SUCCESS
jing9 : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1492791
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotDiffReport.java
DFSClient.getSnapshotDiffReport should use string path rather than
o.a.h.fs.Path
Key: HDFS-4902
URL: https://issues.apache.org/jira/browse/HDFS-4902
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.1.0-beta
Reporter: Binglin Chang
Assignee: Binglin Chang
Attachments: HDFS-4902.001.patch, HDFS-4902.patch
{code}
org.apache.hadoop.ipc.RemoteException(java.lang.AssertionError): Absolute
path required
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathNames(INode.java:641)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:619)
at
org.apache.hadoop.hdfs.server.namenode.INodeDirectory.getINodesInPath4Write(INodeDirectory.java:362)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getINodesInPath4Write(FSDirectory.java:1648)
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:354)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:6035)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1172)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolTranslatorPB.java:975)
at
org.apache.hadoop.hdfs.DFSClient.getSnapshotDiffReport(DFSClient.java:2158)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getSnapshotDiffReport(DistributedFileSystem.java:990)
{code}
DistributedFileSystem.getSnapshotDiffReport use Path with scheme, so toString
will return path with scheme, e.g. hdfs://:8020/abc/
But FSNamesystem only accept simple path, not whole URI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4902) DFSClient.getSnapshotDiffReport should use string path rather than o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HDFS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-4902:
Resolution: Fixed
Fix Version/s: 2.1.0-beta
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Binglin! I've committed this to trunk, branch-2 and branch-2.1.0-beta.
DFSClient.getSnapshotDiffReport should use string path rather than
o.a.h.fs.Path
Key: HDFS-4902
URL: https://issues.apache.org/jira/browse/HDFS-4902
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.1.0-beta
Reporter: Binglin Chang
Assignee: Binglin Chang
Fix For: 2.1.0-beta
Attachments: HDFS-4902.001.patch, HDFS-4902.patch
{code}
org.apache.hadoop.ipc.RemoteException(java.lang.AssertionError): Absolute
path required
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathNames(INode.java:641)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:619)
at
org.apache.hadoop.hdfs.server.namenode.INodeDirectory.getINodesInPath4Write(INodeDirectory.java:362)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getINodesInPath4Write(FSDirectory.java:1648)
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:354)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:6035)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1172)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolTranslatorPB.java:975)
at
org.apache.hadoop.hdfs.DFSClient.getSnapshotDiffReport(DFSClient.java:2158)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getSnapshotDiffReport(DistributedFileSystem.java:990)
{code}
DistributedFileSystem.getSnapshotDiffReport use Path with scheme, so toString
will return path with scheme, e.g. hdfs://:8020/abc/
But FSNamesystem only accept simple path, not whole URI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4902) DFSClient.getSnapshotDiffReport should use string path rather than o.a.h.fs.Path
[
https://issues.apache.org/jira/browse/HDFS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-4902:
Component/s: snapshots
DFSClient.getSnapshotDiffReport should use string path rather than
o.a.h.fs.Path
Key: HDFS-4902
URL: https://issues.apache.org/jira/browse/HDFS-4902
Project: Hadoop HDFS
Issue Type: Bug
Components: snapshots
Affects Versions: 2.1.0-beta
Reporter: Binglin Chang
Assignee: Binglin Chang
Fix For: 2.1.0-beta
Attachments: HDFS-4902.001.patch, HDFS-4902.patch
{code}
org.apache.hadoop.ipc.RemoteException(java.lang.AssertionError): Absolute
path required
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathNames(INode.java:641)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:619)
at
org.apache.hadoop.hdfs.server.namenode.INodeDirectory.getINodesInPath4Write(INodeDirectory.java:362)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getINodesInPath4Write(FSDirectory.java:1648)
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:354)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReport(FSNamesystem.java:6035)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReport(NameNodeRpcServer.java:1172)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getSnapshotDiffReport(ClientNamenodeProtocolTranslatorPB.java:975)
at
org.apache.hadoop.hdfs.DFSClient.getSnapshotDiffReport(DFSClient.java:2158)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getSnapshotDiffReport(DistributedFileSystem.java:990)
{code}
DistributedFileSystem.getSnapshotDiffReport use Path with scheme, so toString
will return path with scheme, e.g. hdfs://:8020/abc/
But FSNamesystem only accept simple path, not whole URI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4845: Attachment: HDFS-4845.004.patch Remove the additional asserts as they are spuriously triggered in tests. Thanks to Chris for taking a look. FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-4888: --- Status: Patch Available (was: Open) Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.8, 2.0.4-alpha, 3.0.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682597#comment-13682597 ] Ravi Prakash commented on HDFS-4888: The same patch applies to branch-2 Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.9 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-4888: --- Affects Version/s: 0.23.9 Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.9 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-4888: --- Target Version/s: 3.0.0, 2.3.0 (was: 3.0.0, 0.23.9, 2.3.0) Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.9 Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[ https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-4888: --- Affects Version/s: (was: 0.23.8) Refactor and fix FSNamesystem.getTurnOffTip to sanity - Key: HDFS-4888 URL: https://issues.apache.org/jira/browse/HDFS-4888 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.0.4-alpha Reporter: Ravi Prakash Assignee: Ravi Prakash Attachments: HDFS-4888.patch e.g. When resources are low, the command to leave safe mode is not printed. This method is unnecessarily complex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4888) Refactor and fix FSNamesystem.getTurnOffTip to sanity
[
https://issues.apache.org/jira/browse/HDFS-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682662#comment-13682662
]
Hadoop QA commented on HDFS-4888:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587659/HDFS-4888.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/4512//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4512//console
This message is automatically generated.
Refactor and fix FSNamesystem.getTurnOffTip to sanity
-
Key: HDFS-4888
URL: https://issues.apache.org/jira/browse/HDFS-4888
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 3.0.0, 2.0.4-alpha, 0.23.9
Reporter: Ravi Prakash
Assignee: Ravi Prakash
Attachments: HDFS-4888.patch
e.g. When resources are low, the command to leave safe mode is not printed.
This method is unnecessarily complex
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[
https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682680#comment-13682680
]
Hadoop QA commented on HDFS-4845:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587681/HDFS-4845.004.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/4513//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4513//console
This message is automatically generated.
FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
Key: HDFS-4845
URL: https://issues.apache.org/jira/browse/HDFS-4845
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 3.0.0
Reporter: Kihwal Lee
Assignee: Arpit Agarwal
Priority: Critical
Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch,
HDFS-4845.003.patch, HDFS-4845.004.patch
TestEditLogRace fails occasionally because it gets NPE from manipulating
INodeMap while loading edits.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-4903) Print trash configuration and trash emptier state in namenode log
Suresh Srinivas created HDFS-4903: - Summary: Print trash configuration and trash emptier state in namenode log Key: HDFS-4903 URL: https://issues.apache.org/jira/browse/HDFS-4903 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Suresh Srinivas Priority: Minor Namenode should print an info level log that print trash interval and if trash emptier is started or not. Also fix a typo Cannot start tresh in Namenode.java. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682723#comment-13682723 ] Arpit Agarwal commented on HDFS-4845: - I believe no new tests should be necessary as we are fixing an existing test. FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4903) Print trash configuration and trash emptier state in namenode log
[ https://issues.apache.org/jira/browse/HDFS-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4903: Assignee: Arpit Agarwal Print trash configuration and trash emptier state in namenode log - Key: HDFS-4903 URL: https://issues.apache.org/jira/browse/HDFS-4903 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Namenode should print an info level log that print trash interval and if trash emptier is started or not. Also fix a typo Cannot start tresh in Namenode.java. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-4904) Remove JournalService
Suresh Srinivas created HDFS-4904: - Summary: Remove JournalService Key: HDFS-4904 URL: https://issues.apache.org/jira/browse/HDFS-4904 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 2.0.3-alpha Reporter: Suresh Srinivas JournalService class was added in HDFS-3099. Since it was not used in HDFS-3077, which has JournalNodeRpcServer instead, I propose deleting this dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4904) Remove JournalService
[ https://issues.apache.org/jira/browse/HDFS-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4904: Assignee: Arpit Agarwal Remove JournalService - Key: HDFS-4904 URL: https://issues.apache.org/jira/browse/HDFS-4904 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 2.0.3-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal JournalService class was added in HDFS-3099. Since it was not used in HDFS-3077, which has JournalNodeRpcServer instead, I propose deleting this dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2572) Unnecessary double-check in DN#getHostName
[ https://issues.apache.org/jira/browse/HDFS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2572: -- Fix Version/s: (was: 0.23.1) (was: 0.24.0) 3.0.0 This code is only in trunk, updating the fix version accordingly. Unnecessary double-check in DN#getHostName -- Key: HDFS-2572 URL: https://issues.apache.org/jira/browse/HDFS-2572 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 0.24.0 Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 3.0.0 Attachments: HDFS-2572.patch, HDFS-2572.patch We do a double config.get unnecessarily inside DN#getHostName(...). Can be removed by this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682772#comment-13682772 ] Chris Nauroth commented on HDFS-4845: - Would you mind putting the unlock call behind a finally block? Technically, it's not really required. I don't see any checked exceptions that can get in the way right now, but if the code evolves to something more complex, then it could be easy to forget adding the finally block later. Also, using a finally block would be consistent with the other call sites that acquire/release the lock. FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3498) Make Replica Removal Policy pluggable and ReplicaPlacementPolicyDefault extensible for reusing code in subclass
[ https://issues.apache.org/jira/browse/HDFS-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3498: -- Issue Type: Improvement (was: Bug) Make Replica Removal Policy pluggable and ReplicaPlacementPolicyDefault extensible for reusing code in subclass --- Key: HDFS-3498 URL: https://issues.apache.org/jira/browse/HDFS-3498 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 1.0.0, 2.0.0-alpha Reporter: Junping Du Assignee: Junping Du Fix For: 1.2.0, 2.1.0-beta Attachments: Hadoop-8471-BlockPlacementDefault-extensible.patch, HDFS-3498-branch-2.patch, HDFS-3498.patch, HDFS-3498-v2.patch, HDFS-3498-v3.patch, HDFS-3498-v4.patch, HDFS-3498-v5.patch ReplicaPlacementPolicy is already a pluggable component in Hadoop. However, the Replica Removal Policy is still nested in BlockManager that need to be separated out into a ReplicaPlacementPolicy then can be override later. Also it looks like hadoop unit test lack the testing on replica removal policy, so we add it here. On the other hand, as a implementation of ReplicaPlacementPolicy, ReplicaPlacementDefault still show lots of generic for other topology cases like: virtualization, and we want to make code in ReplicaPlacementPolicyDefault can be reused as much as possible so a few of its methods were changed from private to protected. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2655) BlockReaderLocal#skip performs unnecessary IO
[ https://issues.apache.org/jira/browse/HDFS-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2655: -- Fix Version/s: 0.23.2 BlockReaderLocal#skip performs unnecessary IO - Key: HDFS-2655 URL: https://issues.apache.org/jira/browse/HDFS-2655 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 0.24.0, 0.23.1 Reporter: Eli Collins Assignee: Brandon Li Fix For: 0.23.2 Attachments: HDFS-2655.txt, HDFS-2655.txt Per HDFS-2654 BlockReaderLocal#skip performs the skip by reading the data so we stay in sync with checksums. This could be implemented more efficiently in the future to skip to the beginning of the appropriate checksum chunk and then only read to the middle of that chunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils
[ https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3009: -- Fix Version/s: (was: 3.0.0) 2.1.0-beta I have merged this change to 2.1.0-beta. DFSClient islocaladdress() can use similar routine in netutils -- Key: HDFS-3009 URL: https://issues.apache.org/jira/browse/HDFS-3009 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 0.23.0, 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Trivial Fix For: 2.1.0-beta Attachments: HDFS-3009.patch isLocalAddress() in dfsclient can use similar function in netutils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3792) Fix two findbugs introduced by HDFS-3695
[ https://issues.apache.org/jira/browse/HDFS-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3792: -- Fix Version/s: (was: 3.0.0) 2.1.0-beta I merged this change to 2.1.0-beta. Fix two findbugs introduced by HDFS-3695 Key: HDFS-3792 URL: https://issues.apache.org/jira/browse/HDFS-3792 Project: Hadoop HDFS Issue Type: Bug Components: build, namenode Affects Versions: 3.0.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Trivial Fix For: 2.1.0-beta Attachments: hdfs-3792.txt Accidentally introduced two trivial findbugs warnings in HDFS-3695. This JIRA is to fix them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2572) Unnecessary double-check in DN#getHostName
[ https://issues.apache.org/jira/browse/HDFS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2572: -- Fix Version/s: (was: 3.0.0) 2.1.0-beta I merged this change to 2.1.0-beta (though the merge resulted in no code change). Unnecessary double-check in DN#getHostName -- Key: HDFS-2572 URL: https://issues.apache.org/jira/browse/HDFS-2572 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 0.24.0 Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 2.1.0-beta Attachments: HDFS-2572.patch, HDFS-2572.patch We do a double config.get unnecessarily inside DN#getHostName(...). Can be removed by this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2857) Cleanup BlockInfo class
[ https://issues.apache.org/jira/browse/HDFS-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2857: -- Fix Version/s: (was: 3.0.0) 2.1.0-beta I merged this change to 2.1.0-beta. Cleanup BlockInfo class --- Key: HDFS-2857 URL: https://issues.apache.org/jira/browse/HDFS-2857 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 2.1.0-beta Attachments: HDFS-2857.23.txt, HDFS-2857.txt Following are some of the cleanup required: # Remove unnecessary methods # Add interface annotation # Make some of the method private -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682893#comment-13682893
]
Suresh Srinivas commented on HDFS-3934:
---
Colin, please mark the Fixed Version(s) field as 3.0.0 on committing a patch to
trunk. Please do merge this to branch-2.1-beta.
duplicative dfs_hosts entries handled wrong
---
Key: HDFS-3934
URL: https://issues.apache.org/jira/browse/HDFS-3934
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.0.1-alpha
Reporter: Andy Isaacson
Assignee: Colin Patrick McCabe
Priority: Minor
Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch,
HDFS-3934.016.patch, HDFS-3934.017.patch
A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
after the NN restarts because {{getDatanodeListForReport}} does not handle
such a pseudo-duplicate correctly:
# the Remove any nodes we know about from the map loop no longer has the
knowledge to remove the spurious entries
# the The remaining nodes are ones that are referenced by the hosts files
loop does not do hostname lookups, so does not know that the IP and hostname
refer to the same host.
Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
the JSP output: The *Node* column shows :50010 as the nodename, with HTML
markup {{a
href=http://:50075/browseDirectory.jsp?namenodeInfoPort=50070amp;dir=%2Famp;nnaddr=172.29.97.196:8020;
title=172.29.97.216:50010:50010/a}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4866) Protocol buffer support cannot compile under C
[ https://issues.apache.org/jira/browse/HDFS-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682913#comment-13682913 ] Chris Nauroth commented on HDFS-4866: - Recapping some offline discussion, the backwards-incompatibility is unavoidable. The RPC method name goes on the wire. Unfortunately, we have no choice but to make a backwards-incompatible change to enable a C client. At least the scope is limited to namenode protocol and not something more widely deployed like the client protocol. +1 for the patch. I plan to commit this and flag it as backwards-incompatible, but I'll wait one more day in case the other watchers on this issue want to offer additional feedback. Thinking ahead to future development using protobuf, we may want to declare a set of officially supported language bindings, take the union of all those languages' reserved words, and declare them off-limits in the proto specs. This could go into the code review checklist as a reminder. Protocol buffer support cannot compile under C -- Key: HDFS-4866 URL: https://issues.apache.org/jira/browse/HDFS-4866 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.0.0, 2.1.0-beta Reporter: Ralph Castain Assignee: Arpit Agarwal Priority: Blocker Attachments: HDFS-4866.branch-2.001.patch, HDFS-4866.trunk.001.patch, NamenodeProtocol.pb-c.c, NamenodeProtocol.pb-c.h, pcreate.pl When compiling Hadoop's .proto descriptions for use in C, an error occurs because one of the RPC's in NamenodeProtocol.proto is named register. This name is a reserved word in languages such as C. When using the Java and C++ languages, the name is hidden inside a class and therefore doesn't cause an error. Unfortunately, that is not the case in non-class languages such as C. Note: generating the C translation of the .proto files requires installation of the protobuf-c package from google: http://code.google.com/p/protobuf-c/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4845: Attachment: HDFS-4845.005.patch Good idea. Fixed. FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3163) TestHDFSCLI.testAll fails if the user name is not all lowercase
[
https://issues.apache.org/jira/browse/HDFS-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-3163:
--
Target Version/s: (was: 3.0.0)
Fix Version/s: (was: 3.0.0)
2.1.0-beta
I have merged this change to 2.1.0-beta.
TestHDFSCLI.testAll fails if the user name is not all lowercase
---
Key: HDFS-3163
URL: https://issues.apache.org/jira/browse/HDFS-3163
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.0.0-alpha
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
Fix For: 1.2.0, 2.1.0-beta
Attachments: HDFS-3163.branch-1.patch, HDFS-3163.branch-1-win.patch,
HDFS-3163.patch
In the test resource file testHDFSConf.xml, the test comparators expect user
name to be all lowercase.
If the user issuing the test has an uppercase in the username (e.g., Brandon
instead of brandon), many RegexpComarator tests will fail. The following is
one example:
{noformat}
comparator
typeRegexpComparator/type
expected-output^-rw-r--r--( )*1( )*[a-z]*( )*supergroup( )*0(
)*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}(
)*/file1/expected-output
/comparator
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-4905) Add appendToFile command to hdfs dfs
Arpit Agarwal created HDFS-4905: --- Summary: Add appendToFile command to hdfs dfs Key: HDFS-4905 URL: https://issues.apache.org/jira/browse/HDFS-4905 Project: Hadoop HDFS Issue Type: Bug Components: tools Affects Versions: 3.0.0 Reporter: Arpit Agarwal Priority: Minor A hdfs dfs -appendToFile... option would be quite useful for quick testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[
https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683006#comment-13683006
]
Hadoop QA commented on HDFS-4845:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587733/HDFS-4845.005.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/4514//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4514//console
This message is automatically generated.
FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
Key: HDFS-4845
URL: https://issues.apache.org/jira/browse/HDFS-4845
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 3.0.0
Reporter: Kihwal Lee
Assignee: Arpit Agarwal
Priority: Critical
Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch,
HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch
TestEditLogRace fails occasionally because it gets NPE from manipulating
INodeMap while loading edits.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-4845: - Hadoop Flags: Reviewed +1 patch looks good. FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-4845: - Resolution: Fixed Fix Version/s: 2.1.0-beta Status: Resolved (was: Patch Available) I have committed this. Thanks, Arpit! FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Fix For: 2.1.0-beta Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683042#comment-13683042 ] Hudson commented on HDFS-4845: -- Integrated in Hadoop-trunk-Commit #3918 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3918/]) HDFS-4845. FSNamesystem.deleteInternal should acquire write-lock before changing the inode map. Contributed by Arpit Agarwal (Revision 1492941) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1492941 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Fix For: 2.1.0-beta Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683045#comment-13683045 ] Chris Nauroth commented on HDFS-4845: - Arpit, thank you for addressing my feedback! (+1 from me too.) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Fix For: 2.1.0-beta Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4845) FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace
[ https://issues.apache.org/jira/browse/HDFS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683055#comment-13683055 ] Arpit Agarwal commented on HDFS-4845: - Thanks Chris and Nicholas! FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace Key: HDFS-4845 URL: https://issues.apache.org/jira/browse/HDFS-4845 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Kihwal Lee Assignee: Arpit Agarwal Priority: Critical Fix For: 2.1.0-beta Attachments: HDFS-4845.001.patch, HDFS-4845.002.patch, HDFS-4845.003.patch, HDFS-4845.004.patch, HDFS-4845.005.patch TestEditLogRace fails occasionally because it gets NPE from manipulating INodeMap while loading edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4533) start-dfs.sh ignored additional parameters besides -upgrade
[ https://issues.apache.org/jira/browse/HDFS-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683060#comment-13683060 ] Fengdong Yu commented on HDFS-4533: --- Hi Suresh, I Just noticed this issue was fixed, sorry for response later. I will file icla soon. Thanks. start-dfs.sh ignored additional parameters besides -upgrade --- Key: HDFS-4533 URL: https://issues.apache.org/jira/browse/HDFS-4533 Project: Hadoop HDFS Issue Type: Bug Components: datanode, namenode Affects Versions: 2.0.3-alpha Reporter: Fengdong Yu Labels: patch Fix For: 2.1.0-beta Attachments: HDFS-4533.patch start-dfs.sh only takes -upgrade option and ignored others. So If run the following command, it will ignore the clusterId option. start-dfs.sh -upgrade -clusterId 1234 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4905) Add appendToFile command to hdfs dfs
[ https://issues.apache.org/jira/browse/HDFS-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4905: Assignee: Arpit Agarwal Status: Patch Available (was: Open) Add appendToFile command to hdfs dfs -- Key: HDFS-4905 URL: https://issues.apache.org/jira/browse/HDFS-4905 Project: Hadoop HDFS Issue Type: Bug Components: tools Affects Versions: 3.0.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Priority: Minor Attachments: HDFS-4905.patch A hdfs dfs -appendToFile... option would be quite useful for quick testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4905) Add appendToFile command to hdfs dfs
[ https://issues.apache.org/jira/browse/HDFS-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4905: Attachment: HDFS-4905.patch Add appendToFile command to hdfs dfs -- Key: HDFS-4905 URL: https://issues.apache.org/jira/browse/HDFS-4905 Project: Hadoop HDFS Issue Type: Bug Components: tools Affects Versions: 3.0.0 Reporter: Arpit Agarwal Priority: Minor Attachments: HDFS-4905.patch A hdfs dfs -appendToFile... option would be quite useful for quick testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4903) Print trash configuration and trash emptier state in namenode log
[
https://issues.apache.org/jira/browse/HDFS-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683071#comment-13683071
]
Arpit Agarwal commented on HDFS-4903:
-
Not sure about this reference to journalservice in
hadoop-hdfs-project/hadoop-hdfs/pom.xml. It was introduced by HDFS-3077.
{code:xml}
execution
idjournal/id
phasegenerate-sources/phase
goals
goalcompile/goal
/goals
configuration
compilefalse/compile
workingDirectory${project.build.directory}/generated-sources/java/workingDirectory
webFragmentFile${project.build.directory}/journal-jsp-servlet-definitions.xml/webFragmentFile
packageNameorg.apache.hadoop.hdfs.server.journalservice/packageName
sources
directory${basedir}/src/main/webapps/journal/directory
includes
include*.jsp/include
/includes
/sources
{code}
Print trash configuration and trash emptier state in namenode log
-
Key: HDFS-4903
URL: https://issues.apache.org/jira/browse/HDFS-4903
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Reporter: Suresh Srinivas
Assignee: Arpit Agarwal
Priority: Minor
Namenode should print an info level log that print trash interval and if
trash emptier is started or not.
Also fix a typo Cannot start tresh in Namenode.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4903) Print trash configuration and trash emptier state in namenode log
[ https://issues.apache.org/jira/browse/HDFS-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683072#comment-13683072 ] Arpit Agarwal commented on HDFS-4903: - Please ignore comment, wrong Jira. Print trash configuration and trash emptier state in namenode log - Key: HDFS-4903 URL: https://issues.apache.org/jira/browse/HDFS-4903 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Namenode should print an info level log that print trash interval and if trash emptier is started or not. Also fix a typo Cannot start tresh in Namenode.java. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4904) Remove JournalService
[
https://issues.apache.org/jira/browse/HDFS-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683073#comment-13683073
]
Arpit Agarwal commented on HDFS-4904:
-
Not sure about this reference to journalservice in
hadoop-hdfs-project/hadoop-hdfs/pom.xml. It was introduced by HDFS-3077.
{code:xml}
execution
idjournal/id
phasegenerate-sources/phase
goals
goalcompile/goal
/goals
configuration
compilefalse/compile
workingDirectory${project.build.directory}/generated-sources/java/workingDirectory
webFragmentFile${project.build.directory}/journal-jsp-servlet-definitions.xml/webFragmentFile
packageNameorg.apache.hadoop.hdfs.server.journalservice/packageName
sources
directory${basedir}/src/main/webapps/journal/directory
includes
include*.jsp/include
/includes
/sources
{code}
Remove JournalService
-
Key: HDFS-4904
URL: https://issues.apache.org/jira/browse/HDFS-4904
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.0.3-alpha
Reporter: Suresh Srinivas
Assignee: Arpit Agarwal
JournalService class was added in HDFS-3099. Since it was not used in
HDFS-3077, which has JournalNodeRpcServer instead, I propose deleting this
dead code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4904) Remove JournalService
[ https://issues.apache.org/jira/browse/HDFS-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4904: Status: Patch Available (was: Open) Remove JournalService - Key: HDFS-4904 URL: https://issues.apache.org/jira/browse/HDFS-4904 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 2.0.3-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Attachments: HDFS-4904.patch JournalService class was added in HDFS-3099. Since it was not used in HDFS-3077, which has JournalNodeRpcServer instead, I propose deleting this dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4904) Remove JournalService
[ https://issues.apache.org/jira/browse/HDFS-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-4904: Attachment: HDFS-4904.patch Remove JournalService - Key: HDFS-4904 URL: https://issues.apache.org/jira/browse/HDFS-4904 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 2.0.3-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Attachments: HDFS-4904.patch JournalService class was added in HDFS-3099. Since it was not used in HDFS-3077, which has JournalNodeRpcServer instead, I propose deleting this dead code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4659) Support setting execution bit for regular files
[ https://issues.apache.org/jira/browse/HDFS-4659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-4659: -- Fix Version/s: 2.1.0-beta I merged this change to 2.1.0-beta. Support setting execution bit for regular files --- Key: HDFS-4659 URL: https://issues.apache.org/jira/browse/HDFS-4659 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Brandon Li Assignee: Brandon Li Fix For: 2.1.0-beta Attachments: HDFS-4659.5.patch, HDFS-4659.patch, HDFS-4659.patch, HDFS-4659.patch, HDFS-4659.patch By default regular files are created with mode rw-r--r--, which is similar as that on many UNIX platforms. However, setting execution bit for regular files are not supported by HDFS. It's the client's choice to set file access mode. HDFS would be easier to use if it can support it, especially when HDFS is accessed by network file system protocols. This JIRA is to track the change to support execution bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
