[jira] [Updated] (HDFS-2354) Generalize the HAServiceProtocol interface
[ https://issues.apache.org/jira/browse/HDFS-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Joseph updated HDFS-2354: Attachment: Namenode HA using Backup Namenode as Hot Standby.pdf The solution document which is already uploaded in the original contribution issue HDFS-2124 Generalize the HAServiceProtocol interface -- Key: HDFS-2354 URL: https://issues.apache.org/jira/browse/HDFS-2354 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Justin Joseph Assignee: Justin Joseph Attachments: HAService_fw_Class_Diagram.JPG, HDFS-2354.1.patch, HDFS-2354.patch, Namenode HA using Backup Namenode as Hot Standby.pdf This JIRA intends to revisit the patches committed for HADOOP-7455 and HDFS-1974 to provide more generic interfaces which allows alternative HA implementations to co-exist complying with HAServiceProtocol. Some of the considerations are 1) Support life cycle methods (start*() and stop() APIs) in HAServiceProtocol 2) Support custom states in HAServiceProtocol 3) As per the patch submitted for HDFS-1974, Namenode implements HAService interface. This needs to be reconsidered. I will elaborate on these points, in the form of comments below. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2411) with webhdfs enabled in secure mode the auth to local mappings are not being respected.
[ https://issues.apache.org/jira/browse/HDFS-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135771#comment-13135771 ] Hadoop QA commented on HDFS-2411: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500627/HDFS-2411-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.TestBalancerBandwidth +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1457//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1457//console This message is automatically generated. with webhdfs enabled in secure mode the auth to local mappings are not being respected. --- Key: HDFS-2411 URL: https://issues.apache.org/jira/browse/HDFS-2411 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0 Attachments: HDFS-2411-branch-0.20-security.patch, HDFS-2411-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2492) BlockManager cross-rack replication checks only work for ScriptBasedMapping
[
https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135781#comment-13135781
]
Steve Loughran commented on HDFS-2492:
--
I'm not sure it's quite so serious, as for the normal script based mapper it
works; it looks for the script file and assumes the presence of the file means
cross-rack.
It's only for people who have written their own DNS mapper that the problem
will arise, and the problem isn't topology mapping, its that the replication
logic that tries to ensure that replicas are spread across multiple racks.
The risk then is: if you have a custom DNS mapper, all your blocks may end up
on the same rack, making you less resilient to rack failures. Your data is
still spread across machines, and the actual placement code itself will work to
place the blocks in multiple racks. The only issue is that at a switch or rack
failure is slightly more likely to take all replicas of a block offline until
the switch comes back up.
It's easy to fix though, I will try and do something today.
Before then: who is running a custom DNS Mapper? Do you expect it to work
unchanged on 0.23+, or is adding another method to an interface something you
can handle?
BlockManager cross-rack replication checks only work for ScriptBasedMapping
---
Key: HDFS-2492
URL: https://issues.apache.org/jira/browse/HDFS-2492
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.23.0, 0.24.0
Reporter: Steve Loughran
Priority: Blocker
The BlockManager cross-rack replication checks only works if script files are
used for replication, not if alternate plugins provide the topology
information.
This is because the BlockManager sets its rack checking flag if there is a
filename key
{code}
shouldCheckForEnoughRacks =
conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY) != null;
{code}
yet this filename key is only used if the topology mapper defined by
{code}
DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
{code}
is an instance of {{ScriptBasedMapping}}
If any other mapper is used, the system may be multi rack, but the Block
Manager will not be aware of this fact unless the filename key is set to
something non-null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1971) HA: Send block report from datanode to both active and standby namenodes
[ https://issues.apache.org/jira/browse/HDFS-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HDFS-1971: --- Attachment: dualBr3.patch Updated patch. Still a draft but good enough to read and get feedback. * Block report additions and deletions are give to both BPService thread (for active and standby) * BRs as per option 2: are created for each thread separately (note BR is created from in-memory BMap). Optimization 1 or 3 is under consideration. HA: Send block report from datanode to both active and standby namenodes Key: HDFS-1971 URL: https://issues.apache.org/jira/browse/HDFS-1971 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Sanjay Radia Attachments: DualBlockReports.pdf, daulBr1.patch, dualBr2.patch, dualBr3.patch To enable hot standby namenode, the standby node must have current information for - namenode state (image + edits) and block location information. This jira addresses keeping the block location information current in the standby node. To do this, the proposed solution is to send block reports from the datanodes to both the active and the standby namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2452) OutOfMemoryError in DataXceiverServer takes down the DataNode
[ https://issues.apache.org/jira/browse/HDFS-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135898#comment-13135898 ] Uma Maheswara Rao G commented on HDFS-2452: --- Hi Konstantin, I just compared the code with trunk and 0.22 branch. I don't see any difference, except getInputStream stub and there is no sleeps as well!. I think, i he may be pointing about source code 30sec sleep or may be about testcase timeout. That will be the failure case. If CountDownLatch does not decrement the count then, latch.await() may wait forever. So , to avaoid this, we gave 3ms as testcase timeout. This will be the failure case. If that is the case, we need to correct the actual testcase problem first. Why it is not throwing the OutOfMemory Error from aspectj mocks and did he provided any trace? If you get any trace please provide, i will take a look. one more point is, some one started looking on AspectJ on trunk? if yes, can you share the options to run the aspectJ on trunk if you have idea? Thanks, Uma OutOfMemoryError in DataXceiverServer takes down the DataNode - Key: HDFS-2452 URL: https://issues.apache.org/jira/browse/HDFS-2452 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 0.22.0, 0.23.0, 0.24.0 Reporter: Konstantin Shvachko Assignee: Uma Maheswara Rao G Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HDFS-2452-22Branch.2.patch, HDFS-2452-22branch.1.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch_with-around_.patch, HDFS-2452-22branch_with-around_.patch, HDFS-2452-Trunk-src-fix.patch, HDFS-2452-Trunk.patch, HDFS-2452.patch, HDFS-2452.patch OutOfMemoryError brings down DataNode, when DataXceiverServer tries to spawn a new data transfer thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2504) NPE when running hdfs dfs -copyToLocal, -get etc
NPE when running hdfs dfs -copyToLocal, -get etc Key: HDFS-2504 URL: https://issues.apache.org/jira/browse/HDFS-2504 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0, 0.24.0 Reporter: John George Assignee: John George Priority: Blocker NPE when running hdfs dfs -copyToLocal if the destination directory does not exist. The behavior in branch-0.20-security is to create the directory and copy/get the contents from source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135906#comment-13135906 ] Uma Maheswara Rao G commented on HDFS-2494: --- Sorry Nicholas, i was in Travel. So, i could not post the patch yesterday. I will post the patch for security, in couple of minutes. Thanks Uma [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2504) NPE when running hdfs dfs -copyToLocal, -get etc
[ https://issues.apache.org/jira/browse/HDFS-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135905#comment-13135905 ] John George commented on HDFS-2504: --- $ hadoop fs -copyToLocal /tmp/tryout /tmp/thisisit.2 copyToLocal: Fatal internal error java.lang.NullPointerException at org.apache.hadoop.fs.shell.PathData.getPathDataForChild(PathData.java:182) at org.apache.hadoop.fs.shell.CommandWithDestination.processPaths(CommandWithDestination.java:115) at org.apache.hadoop.fs.shell.Command.recursePath(Command.java:329) at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:302) at org.apache.hadoop.fs.shell.CommandWithDestination.processPaths(CommandWithDestination.java:116) at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:272) at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:255) at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:239) at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:105) at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:185) at org.apache.hadoop.fs.shell.Command.run(Command.java:149) at org.apache.hadoop.fs.FsShell.run(FsShell.java:254) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83) at org.apache.hadoop.fs.FsShell.main(FsShell.java:296) NPE when running hdfs dfs -copyToLocal, -get etc Key: HDFS-2504 URL: https://issues.apache.org/jira/browse/HDFS-2504 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0, 0.24.0 Reporter: John George Assignee: John George Priority: Blocker Attachments: HDFS-2504.patch NPE when running hdfs dfs -copyToLocal if the destination directory does not exist. The behavior in branch-0.20-security is to create the directory and copy/get the contents from source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2504) NPE when running hdfs dfs -copyToLocal, -get etc
[ https://issues.apache.org/jira/browse/HDFS-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John George updated HDFS-2504: -- Attachment: HDFS-2504.patch NPE when running hdfs dfs -copyToLocal, -get etc Key: HDFS-2504 URL: https://issues.apache.org/jira/browse/HDFS-2504 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0, 0.24.0 Reporter: John George Assignee: John George Priority: Blocker Attachments: HDFS-2504.patch NPE when running hdfs dfs -copyToLocal if the destination directory does not exist. The behavior in branch-0.20-security is to create the directory and copy/get the contents from source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2504) NPE when running hdfs dfs -copyToLocal, -get etc
[ https://issues.apache.org/jira/browse/HDFS-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John George updated HDFS-2504: -- Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) NPE when running hdfs dfs -copyToLocal, -get etc Key: HDFS-2504 URL: https://issues.apache.org/jira/browse/HDFS-2504 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0, 0.24.0 Reporter: John George Assignee: John George Priority: Blocker Attachments: HDFS-2504.patch NPE when running hdfs dfs -copyToLocal if the destination directory does not exist. The behavior in branch-0.20-security is to create the directory and copy/get the contents from source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2504) NPE when running hdfs dfs -copyToLocal, -get etc
[ https://issues.apache.org/jira/browse/HDFS-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135915#comment-13135915 ] Hadoop QA commented on HDFS-2504: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500874/HDFS-2504.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1459//console This message is automatically generated. NPE when running hdfs dfs -copyToLocal, -get etc Key: HDFS-2504 URL: https://issues.apache.org/jira/browse/HDFS-2504 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0, 0.24.0 Reporter: John George Assignee: John George Priority: Blocker Attachments: HDFS-2504.patch NPE when running hdfs dfs -copyToLocal if the destination directory does not exist. The behavior in branch-0.20-security is to create the directory and copy/get the contents from source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2501) add version prefix and root methods to webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135924#comment-13135924 ] Hudson commented on HDFS-2501: -- Integrated in Hadoop-Hdfs-trunk #844 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/844/]) HDFS-2501. Add version prefix and root methods to webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189028 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java add version prefix and root methods to webhdfs -- Key: HDFS-2501 URL: https://issues.apache.org/jira/browse/HDFS-2501 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.1, 0.20.206.0, 0.24.0 Attachments: h2501_20111025.patch, h2501_20111025_0.20s.patch, h2501_20111025b.patch, h2501_20111025b2.patch, h2501_20111025b_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2501) add version prefix and root methods to webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135940#comment-13135940 ] Hudson commented on HDFS-2501: -- Integrated in Hadoop-Mapreduce-trunk #872 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/872/]) HDFS-2501. Add version prefix and root methods to webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189028 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java add version prefix and root methods to webhdfs -- Key: HDFS-2501 URL: https://issues.apache.org/jira/browse/HDFS-2501 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.1, 0.20.206.0, 0.24.0 Attachments: h2501_20111025.patch, h2501_20111025_0.20s.patch, h2501_20111025b.patch, h2501_20111025b2.patch, h2501_20111025b_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2504) NPE when running hdfs dfs -copyToLocal, -get etc
[ https://issues.apache.org/jira/browse/HDFS-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John George updated HDFS-2504: -- Resolution: Duplicate Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Resolved (was: Patch Available) created a JIRA (HADOOP-7771) in the right bucket (common) and resolving this as duplicate of HADOOP-7771 NPE when running hdfs dfs -copyToLocal, -get etc Key: HDFS-2504 URL: https://issues.apache.org/jira/browse/HDFS-2504 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0, 0.24.0 Reporter: John George Assignee: John George Priority: Blocker Attachments: HDFS-2504.patch NPE when running hdfs dfs -copyToLocal if the destination directory does not exist. The behavior in branch-0.20-security is to create the directory and copy/get the contents from source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2354) Generalize the HAServiceProtocol interface
[ https://issues.apache.org/jira/browse/HDFS-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136006#comment-13136006 ] Justin Joseph commented on HDFS-2354: - For better understanding of Backup Node based HA with Zookeeper as the distributed coordinator, please refer the [Namenode HA using Backup Namenode as Hot Standby.pdf|https://issues.apache.org/jira/secure/attachment/12500829/Namenode+HA+using+Backup+Namenode+as+Hot+Standby.pdf] I have attached in this JIRA. Generalize the HAServiceProtocol interface -- Key: HDFS-2354 URL: https://issues.apache.org/jira/browse/HDFS-2354 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Justin Joseph Assignee: Justin Joseph Attachments: HAService_fw_Class_Diagram.JPG, HDFS-2354.1.patch, HDFS-2354.patch, Namenode HA using Backup Namenode as Hot Standby.pdf This JIRA intends to revisit the patches committed for HADOOP-7455 and HDFS-1974 to provide more generic interfaces which allows alternative HA implementations to co-exist complying with HAServiceProtocol. Some of the considerations are 1) Support life cycle methods (start*() and stop() APIs) in HAServiceProtocol 2) Support custom states in HAServiceProtocol 3) As per the patch submitted for HDFS-1974, Namenode implements HAService interface. This needs to be reconsidered. I will elaborate on these points, in the form of comments below. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2354) Generalize the HAServiceProtocol interface
[ https://issues.apache.org/jira/browse/HDFS-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136011#comment-13136011 ] Justin Joseph commented on HDFS-2354: - bq. Can you explain hot it is against the goal of HDFS-1623? HDFS-1623 aims to build a framework for HA, where there is room for multiple approaches for HA (Shared Storage / Backup Node Based, Zookeeper / Linux HA, IP failover / Intelligent Clients so on). In the patch submitted for HDFS-1974, Namenode (NamenodeRpcServer) implements the HAServiceProtocol interface provides the implementation based on Shared Storage approach. Now when I want to provide another implementation based on Backup Node, I am confused how fit it into the HA framework. This is the point I want to make. Generalize the HAServiceProtocol interface -- Key: HDFS-2354 URL: https://issues.apache.org/jira/browse/HDFS-2354 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Justin Joseph Assignee: Justin Joseph Attachments: HAService_fw_Class_Diagram.JPG, HDFS-2354.1.patch, HDFS-2354.patch, Namenode HA using Backup Namenode as Hot Standby.pdf This JIRA intends to revisit the patches committed for HADOOP-7455 and HDFS-1974 to provide more generic interfaces which allows alternative HA implementations to co-exist complying with HAServiceProtocol. Some of the considerations are 1) Support life cycle methods (start*() and stop() APIs) in HAServiceProtocol 2) Support custom states in HAServiceProtocol 3) As per the patch submitted for HDFS-1974, Namenode implements HAService interface. This needs to be reconsidered. I will elaborate on these points, in the form of comments below. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported to access data via hdfs://
[ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136030#comment-13136030 ] Daryn Sharp commented on HDFS-2450: --- Full test suite passes. [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. Only complete hostname is supported to access data via hdfs:// -- Key: HDFS-2450 URL: https://issues.apache.org/jira/browse/HDFS-2450 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Rajit Saha Assignee: Daryn Sharp Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch, IP vs. Hostname.pdf If my complete hostname is host1.abc.xyz.com, only complete hostname must be used to access data via hdfs:// I am running following in .20.205 Client to get data from .20.205 NN (host1) $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com Usage: java FsShell [-copyFromLocal localsrc ... dst] $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/ copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com Usage: java FsShell [-copyFromLocal localsrc ... dst] $hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/ copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com Usage: java FsShell [-copyFromLocal localsrc ... dst] Only following is supported $hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2487) HA: Administrative CLI to control HA daemons
[ https://issues.apache.org/jira/browse/HDFS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136031#comment-13136031 ] Suresh Srinivas commented on HDFS-2487: --- dfsadmin command be added to change NN HA states based on adding a client that communicates using HAServiceProtocol. Any other things you are planning to do as a part of this jira? HA: Administrative CLI to control HA daemons Key: HDFS-2487 URL: https://issues.apache.org/jira/browse/HDFS-2487 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Affects Versions: HA branch (HDFS-1623) Reporter: Aaron T. Myers Assignee: Aaron T. Myers We'll need to have some way of controlling the HA nodes while they're live, probably by adding some more commands to dfsadmin. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2494: -- Attachment: HDFS-2494-20.205Security.patch [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136044#comment-13136044 ] Hadoop QA commented on HDFS-2494: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500889/HDFS-2494-20.205Security.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1460//console This message is automatically generated. [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136051#comment-13136051 ] Uma Maheswara Rao G commented on HDFS-2494: --- Patch is for 20.205 security branch! [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2505) Add a test to verify getFileChecksum works with ViewFS
Add a test to verify getFileChecksum works with ViewFS -- Key: HDFS-2505 URL: https://issues.apache.org/jira/browse/HDFS-2505 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Priority: Critical Fix For: 0.23.0 Please refer to HADOOP-7770. getFileChecksum was failing on files such as /tmp/someFile, but working fine for /someDir/someFile. The fix is in HADOOP-7770 but we should have a test in HDFS which checks this functionality (this test will fail until HADOOP-7770 is checked in) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2505) Add a test to verify getFileChecksum works with ViewFS
[ https://issues.apache.org/jira/browse/HDFS-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-2505: --- Attachment: HDFS-2505.patch This patch contains the test. It should fail until HADOOP-7770 goes in Add a test to verify getFileChecksum works with ViewFS -- Key: HDFS-2505 URL: https://issues.apache.org/jira/browse/HDFS-2505 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Priority: Critical Labels: test Fix For: 0.23.0 Attachments: HDFS-2505.patch Please refer to HADOOP-7770. getFileChecksum was failing on files such as /tmp/someFile, but working fine for /someDir/someFile. The fix is in HADOOP-7770 but we should have a test in HDFS which checks this functionality (this test will fail until HADOOP-7770 is checked in) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2505) Add a test to verify getFileChecksum works with ViewFS
[ https://issues.apache.org/jira/browse/HDFS-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-2505: --- Status: Patch Available (was: Open) Add a test to verify getFileChecksum works with ViewFS -- Key: HDFS-2505 URL: https://issues.apache.org/jira/browse/HDFS-2505 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Priority: Critical Labels: test Fix For: 0.23.0 Attachments: HDFS-2505.patch Please refer to HADOOP-7770. getFileChecksum was failing on files such as /tmp/someFile, but working fine for /someDir/someFile. The fix is in HADOOP-7770 but we should have a test in HDFS which checks this functionality (this test will fail until HADOOP-7770 is checked in) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2496) Separate datatypes for DatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2496: -- Attachment: HDFS-2496.txt New patch addresses the comments. bq. move the utility methods in DatanodeCommandHelper to DatanodeCommandWritable and then remove DatanodeCommandHelper. The reason why I did not add it to DatanodeCommandWritable is to avoid it having awareness of it's subclasses. As the wire type separation work continues, I might move DatanodeCommandHelper to a common utility. Separate datatypes for DatanodeProtocol --- Key: HDFS-2496 URL: https://issues.apache.org/jira/browse/HDFS-2496 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2496.txt, HDFS-2496.txt, HDFS-2496.txt This jira separates for DatanodeProtocol the wire types from the types used by the client and server, similar to HDFS-2181. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2181) Separate HDFS Client wire protocol data types
[ https://issues.apache.org/jira/browse/HDFS-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2181: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-2506 Separate HDFS Client wire protocol data types - Key: HDFS-2181 URL: https://issues.apache.org/jira/browse/HDFS-2181 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Sanjay Radia Fix For: 0.24.0 Attachments: separateDataType1.patch, separateDataType10.patch, separateDataType11.patch, separateDataType12.patch, separateDataType13.patch, separateDataType14.patch, separateDataType16.patch, separateDataType19.patch, separateDataType2.patch, separateDataType20.patch, separateDataType6.patch, separateDataType7.patch, separateDataType8.patch, separateDataType9.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2459) Separate datatypes for Journal protocol
[ https://issues.apache.org/jira/browse/HDFS-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2459: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-2506 Separate datatypes for Journal protocol --- Key: HDFS-2459 URL: https://issues.apache.org/jira/browse/HDFS-2459 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2459.txt, HDFS-2459.txt, HDFS-2459.txt This jira separates for JournalProtocol the wire types from the types used by the client and server, similar to HDFS-2181. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2506) Umbrella jira for tracking separation of wire protocol datatypes from the implementation types
Umbrella jira for tracking separation of wire protocol datatypes from the implementation types -- Key: HDFS-2506 URL: https://issues.apache.org/jira/browse/HDFS-2506 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Umbrella jira for tracking separation of wire protocol datatypes from the implementation types. See subtasks for more details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2488) Separate datatypes for InterDatanodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2488: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-2506 Separate datatypes for InterDatanodeProtocol Key: HDFS-2488 URL: https://issues.apache.org/jira/browse/HDFS-2488 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2488.txt, HDFS-2488.txt, HDFS-2488.txt This jira separates for InterDatanodeProtocol the wire types from the types used by the client and server, similar to HDFS-2181. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2480) Separate datatypes for NamenodeProtocol
[ https://issues.apache.org/jira/browse/HDFS-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2480: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-2506 Separate datatypes for NamenodeProtocol --- Key: HDFS-2480 URL: https://issues.apache.org/jira/browse/HDFS-2480 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2480.txt, HDFS-2480.txt, HDFS-2480.txt This jira separates for NamenodeProtocol the wire types from the types used by the client and server, similar to HDFS-2181. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2489) Move commands Finalize and Register out of DatanodeCommand class.
[ https://issues.apache.org/jira/browse/HDFS-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2489: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-2506 Move commands Finalize and Register out of DatanodeCommand class. - Key: HDFS-2489 URL: https://issues.apache.org/jira/browse/HDFS-2489 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.23.0, 0.24.0 Environment: There are other subclasses available in separate files. These commands should be moved to separate files as well. Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2489.patch, HDFS-2489.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2499) Fix RPC client creation bug from HDFS-2459
[ https://issues.apache.org/jira/browse/HDFS-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2499: -- Status: Patch Available (was: Open) Fix RPC client creation bug from HDFS-2459 -- Key: HDFS-2499 URL: https://issues.apache.org/jira/browse/HDFS-2499 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2499.txt, HDFS-2499.txt HDFS-2459 incorrectly implemented the RPC getProxy for the JournalProtocol client side. It sets retry policies and other policies that are not necessary -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1971) HA: Send block report from datanode to both active and standby namenodes
[ https://issues.apache.org/jira/browse/HDFS-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136137#comment-13136137 ] Todd Lipcon commented on HDFS-1971: --- Hey Sanjay. I like the architecture of this patch - makes sense to separate into the different Actor objects. I have a number of small nits/improvements to suggest, but I can wait until you have a more final patch. Or, if you like, I can do an iteration on top of the patch you've posted here and re-post by end of day today with suggested changes. HA: Send block report from datanode to both active and standby namenodes Key: HDFS-1971 URL: https://issues.apache.org/jira/browse/HDFS-1971 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Sanjay Radia Attachments: DualBlockReports.pdf, daulBr1.patch, dualBr2.patch, dualBr3.patch To enable hot standby namenode, the standby node must have current information for - namenode state (image + edits) and block location information. This jira addresses keeping the block location information current in the standby node. To do this, the proposed solution is to send block reports from the datanodes to both the active and the standby namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1971) HA: Send block report from datanode to both active and standby namenodes
[
https://issues.apache.org/jira/browse/HDFS-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136147#comment-13136147
]
Todd Lipcon commented on HDFS-1971:
---
Only non-nitty thing I would suggest: rather than having just a
bpServiceToActive and bpServiceToStandby, could we consider having:
{code}
ListBPServiceActor bpServiceActors;
BPServiceActor bpServiceToActive; // reference to one of the above NNs
{code}
then we can change a lot of the code to loop over bpServiceActors rather than
having two separate calls to active and standby.
Although we don't plan to support multiple-standby in today's implementation, I
don't think the above introduces any extra complexity, and in fact will make
the code cleaner, while also paving the way for a multiple-standby
implementation in the future.
HA: Send block report from datanode to both active and standby namenodes
Key: HDFS-1971
URL: https://issues.apache.org/jira/browse/HDFS-1971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: data-node, name-node
Reporter: Suresh Srinivas
Assignee: Sanjay Radia
Attachments: DualBlockReports.pdf, daulBr1.patch, dualBr2.patch,
dualBr3.patch
To enable hot standby namenode, the standby node must have current
information for - namenode state (image + edits) and block location
information. This jira addresses keeping the block location information
current in the standby node. To do this, the proposed solution is to send
block reports from the datanodes to both the active and the standby namenode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2479) HDFS Client Data Types in Protocol Buffers
[ https://issues.apache.org/jira/browse/HDFS-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HDFS-2479: --- Status: Open (was: Patch Available) HDFS Client Data Types in Protocol Buffers -- Key: HDFS-2479 URL: https://issues.apache.org/jira/browse/HDFS-2479 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hdfs.proto, hdfs.proto, hdfs.proto, pbClientTypes1.patch, pbClientTypes2.patch, pbClientTypes3.patch, pbClientTypes4.patch, pbClientTypes5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2479) HDFS Client Data Types in Protocol Buffers
[ https://issues.apache.org/jira/browse/HDFS-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HDFS-2479: --- Attachment: pbClientTypes5.patch Update patch that addresses Suresh's feedback. HDFS Client Data Types in Protocol Buffers -- Key: HDFS-2479 URL: https://issues.apache.org/jira/browse/HDFS-2479 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hdfs.proto, hdfs.proto, hdfs.proto, pbClientTypes1.patch, pbClientTypes2.patch, pbClientTypes3.patch, pbClientTypes4.patch, pbClientTypes5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2479) HDFS Client Data Types in Protocol Buffers
[ https://issues.apache.org/jira/browse/HDFS-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HDFS-2479: --- Status: Patch Available (was: Open) HDFS Client Data Types in Protocol Buffers -- Key: HDFS-2479 URL: https://issues.apache.org/jira/browse/HDFS-2479 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hdfs.proto, hdfs.proto, hdfs.proto, pbClientTypes1.patch, pbClientTypes2.patch, pbClientTypes3.patch, pbClientTypes4.patch, pbClientTypes5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2505) Add a test to verify getFileChecksum works with ViewFS
[ https://issues.apache.org/jira/browse/HDFS-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136162#comment-13136162 ] Hadoop QA commented on HDFS-2505: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500896/HDFS-2505.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.TestBalancerBandwidth org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1461//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1461//console This message is automatically generated. Add a test to verify getFileChecksum works with ViewFS -- Key: HDFS-2505 URL: https://issues.apache.org/jira/browse/HDFS-2505 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Priority: Critical Labels: test Fix For: 0.23.0 Attachments: HDFS-2505.patch Please refer to HADOOP-7770. getFileChecksum was failing on files such as /tmp/someFile, but working fine for /someDir/someFile. The fix is in HADOOP-7770 but we should have a test in HDFS which checks this functionality (this test will fail until HADOOP-7770 is checked in) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2452) OutOfMemoryError in DataXceiverServer takes down the DataNode
[ https://issues.apache.org/jira/browse/HDFS-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136164#comment-13136164 ] Konstantin Boudnik commented on HDFS-2452: -- bq. If you get any trace please provide, i will take a look. I don't think there will be traces - aspectj support is totally broken in trunk and it hasn't been fixed yet. Hopefully, I will have some spare time to work on it. OutOfMemoryError in DataXceiverServer takes down the DataNode - Key: HDFS-2452 URL: https://issues.apache.org/jira/browse/HDFS-2452 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 0.22.0, 0.23.0, 0.24.0 Reporter: Konstantin Shvachko Assignee: Uma Maheswara Rao G Fix For: 0.22.0, 0.23.0, 0.24.0 Attachments: HDFS-2452-22Branch.2.patch, HDFS-2452-22branch.1.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch.patch, HDFS-2452-22branch_with-around_.patch, HDFS-2452-22branch_with-around_.patch, HDFS-2452-Trunk-src-fix.patch, HDFS-2452-Trunk.patch, HDFS-2452.patch, HDFS-2452.patch OutOfMemoryError brings down DataNode, when DataXceiverServer tries to spawn a new data transfer thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1900) Use the block size key defined by common
[ https://issues.apache.org/jira/browse/HDFS-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136166#comment-13136166 ] Eli Collins commented on HDFS-1900: --- I agree with Konst, we should remove the overlap between FsConfig and DFSCOnfig. Use the block size key defined by common - Key: HDFS-1900 URL: https://issues.apache.org/jira/browse/HDFS-1900 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.21.1 Reporter: Eli Collins Assignee: Abel Perez Labels: newbie Fix For: 0.22.0 Attachments: HDFS-1900.txt HADOOP-4952 added a dfs.block.size key to common configuration, defined in o.a.h.fs.FsConfig. This conflicts with the original HDFS block size key of the same name, which is now deprecated in favor of dfs.blocksize. It doesn't make sense to have two different keys for the block size (ie they can disagree). Why doesn't HDFS just use the key defined in common? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2411) with webhdfs enabled in secure mode the auth to local mappings are not being respected.
[ https://issues.apache.org/jira/browse/HDFS-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136181#comment-13136181 ] Jitendra Nath Pandey commented on HDFS-2411: Note that HADOOP-7766 patch is also required to fix this issue completely. with webhdfs enabled in secure mode the auth to local mappings are not being respected. --- Key: HDFS-2411 URL: https://issues.apache.org/jira/browse/HDFS-2411 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0 Attachments: HDFS-2411-branch-0.20-security.patch, HDFS-2411-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2507) HA: Allow saveNamespace operations to be canceled
HA: Allow saveNamespace operations to be canceled - Key: HDFS-2507 URL: https://issues.apache.org/jira/browse/HDFS-2507 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: HA branch (HDFS-1623), 0.23.0 Reporter: Todd Lipcon For HDFS-2291 (checkpointing in HA setup) we need to allow the StandbyNode to handle a failover while it may be in the middle of saving a checkpoint. This implies that checkpoints need to be abortable. This JIRA is to implement this functionality. Even though this is only needed for HA, I think we should do the infrastructure in trunk to simplify future merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2411) with webhdfs enabled in secure mode the auth to local mappings are not being respected.
[ https://issues.apache.org/jira/browse/HDFS-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136192#comment-13136192 ] Hudson commented on HDFS-2411: -- Integrated in Hadoop-Common-trunk-Commit #1159 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1159/]) HDFS-2411. The auth to local mappings are not respected with webhdfs and security enabled. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189355 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java with webhdfs enabled in secure mode the auth to local mappings are not being respected. --- Key: HDFS-2411 URL: https://issues.apache.org/jira/browse/HDFS-2411 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0 Attachments: HDFS-2411-branch-0.20-security.patch, HDFS-2411-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136197#comment-13136197 ] Hudson commented on HDFS-2494: -- Integrated in Hadoop-Hdfs-trunk-Commit #1237 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1237/]) HDFS-2494. Close the streams and DFSClient in DatanodeWebHdfsMethods. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189360 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.20.205.1, 0.20.206.0, 0.24.0 Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2411) with webhdfs enabled in secure mode the auth to local mappings are not being respected.
[ https://issues.apache.org/jira/browse/HDFS-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136196#comment-13136196 ] Hudson commented on HDFS-2411: -- Integrated in Hadoop-Hdfs-trunk-Commit #1237 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1237/]) HDFS-2411. The auth to local mappings are not respected with webhdfs and security enabled. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189355 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java with webhdfs enabled in secure mode the auth to local mappings are not being respected. --- Key: HDFS-2411 URL: https://issues.apache.org/jira/browse/HDFS-2411 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0 Attachments: HDFS-2411-branch-0.20-security.patch, HDFS-2411-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2494: - Resolution: Fixed Fix Version/s: 0.24.0 0.20.206.0 0.20.205.1 Target Version/s: 0.20.205.1, 0.24.0 (was: 0.24.0, 0.20.205.1) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I have committed this. Thanks a lot, Uma! [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.20.205.1, 0.20.206.0, 0.24.0 Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2507) HA: Allow saveNamespace operations to be canceled
[ https://issues.apache.org/jira/browse/HDFS-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2507: -- Assignee: Todd Lipcon Target Version/s: HA branch (HDFS-1623), 0.24.0 (was: 0.24.0, HA branch (HDFS-1623)) Status: Patch Available (was: Open) HA: Allow saveNamespace operations to be canceled - Key: HDFS-2507 URL: https://issues.apache.org/jira/browse/HDFS-2507 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: HA branch (HDFS-1623), 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2507.txt For HDFS-2291 (checkpointing in HA setup) we need to allow the StandbyNode to handle a failover while it may be in the middle of saving a checkpoint. This implies that checkpoints need to be abortable. This JIRA is to implement this functionality. Even though this is only needed for HA, I think we should do the infrastructure in trunk to simplify future merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2505) Add a test to verify getFileChecksum works with ViewFS
[ https://issues.apache.org/jira/browse/HDFS-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136199#comment-13136199 ] Ravi Prakash commented on HDFS-2505: This patch couldn't possibly fail any of those tests because it doesn't have any src/main code. Only src/test code. Add a test to verify getFileChecksum works with ViewFS -- Key: HDFS-2505 URL: https://issues.apache.org/jira/browse/HDFS-2505 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Priority: Critical Labels: test Fix For: 0.23.0 Attachments: HDFS-2505.patch Please refer to HADOOP-7770. getFileChecksum was failing on files such as /tmp/someFile, but working fine for /someDir/someFile. The fix is in HADOOP-7770 but we should have a test in HDFS which checks this functionality (this test will fail until HADOOP-7770 is checked in) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2507) HA: Allow saveNamespace operations to be canceled
[ https://issues.apache.org/jira/browse/HDFS-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2507: -- Attachment: hdfs-2507.txt HA: Allow saveNamespace operations to be canceled - Key: HDFS-2507 URL: https://issues.apache.org/jira/browse/HDFS-2507 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: HA branch (HDFS-1623), 0.23.0 Reporter: Todd Lipcon Attachments: hdfs-2507.txt For HDFS-2291 (checkpointing in HA setup) we need to allow the StandbyNode to handle a failover while it may be in the middle of saving a checkpoint. This implies that checkpoints need to be abortable. This JIRA is to implement this functionality. Even though this is only needed for HA, I think we should do the infrastructure in trunk to simplify future merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136203#comment-13136203 ] Hudson commented on HDFS-2494: -- Integrated in Hadoop-Common-trunk-Commit #1160 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1160/]) HDFS-2494. Close the streams and DFSClient in DatanodeWebHdfsMethods. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189360 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.20.205.1, 0.20.206.0, 0.24.0 Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported to access data via hdfs://
[
https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136207#comment-13136207
]
Suresh Srinivas commented on HDFS-2450:
---
Comments:
# FileSystem#checkPath
#* Unrelated this patch - can you get thatScheme before making {{if
(uri.getScheme() == null}} check. This avoids getting scheme twice from the URI.
#* Earlier the check was based on thisAuthority using the port that the file
system had. Now it with getCanonicalUri(), it uses default port. Is this
correct? All checks seem to be using defaultPort.
#* Why remove?
{noformat}
if (thisAuthority == thatAuthority || // authorities match
(thisAuthority != null
thisAuthority.equalsIgnoreCase(thatAuthority)))
return;
try { // or the default fs's uri
defaultUri = get(getConf()).getUri();
} catch (IOException e) {
throw new RuntimeException(e);
}
{noformat}
#* The new method does not work for some cases like the previous one. If
thisAuthority and thatAuthority is null, the new mehod throw
IllegalArgumentException. This is due to removing an if condition after scheme
check.
#* If thisAuthority and thatAuthority are not null, you end up parsing the URI
for thatAuthority twice. This is due to removing an if condition after scheme
check.
# DistributedFileSystem#makeQualified remove makes me concerned. I have not
gone through all the cases to make sure there is no problem with that change.
Can you please describe tests you run?
Only complete hostname is supported to access data via hdfs://
--
Key: HDFS-2450
URL: https://issues.apache.org/jira/browse/HDFS-2450
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Rajit Saha
Assignee: Daryn Sharp
Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch,
IP vs. Hostname.pdf
If my complete hostname is host1.abc.xyz.com, only complete hostname must be
used to access data via hdfs://
I am running following in .20.205 Client to get data from .20.205 NN (host1)
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal localsrc ... dst]
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected:
hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal localsrc ... dst]
$hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected:
hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal localsrc ... dst]
Only following is supported
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2411) with webhdfs enabled in secure mode the auth to local mappings are not being respected.
[ https://issues.apache.org/jira/browse/HDFS-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136214#comment-13136214 ] Hudson commented on HDFS-2411: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1173 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1173/]) HDFS-2411. The auth to local mappings are not respected with webhdfs and security enabled. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189355 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java with webhdfs enabled in secure mode the auth to local mappings are not being respected. --- Key: HDFS-2411 URL: https://issues.apache.org/jira/browse/HDFS-2411 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0 Attachments: HDFS-2411-branch-0.20-security.patch, HDFS-2411-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2284) Write Http access to HDFS
[ https://issues.apache.org/jira/browse/HDFS-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136218#comment-13136218 ] Hudson commented on HDFS-2284: -- Integrated in Hadoop-Common-0.23-Commit #55 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/55/]) svn merge -c 1167662 from trunk for HDFS-2284. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189375 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BooleanParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DstPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumSetParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/ExceptionHandler.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GroupParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/IntegerParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/LongParam.java *
[jira] [Commented] (HDFS-2499) Fix RPC client creation bug from HDFS-2459
[ https://issues.apache.org/jira/browse/HDFS-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136227#comment-13136227 ] Hadoop QA commented on HDFS-2499: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500618/HDFS-2499.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestFileCreationClient org.apache.hadoop.hdfs.TestSetrepIncreasing +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1462//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1462//console This message is automatically generated. Fix RPC client creation bug from HDFS-2459 -- Key: HDFS-2499 URL: https://issues.apache.org/jira/browse/HDFS-2499 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2499.txt, HDFS-2499.txt HDFS-2459 incorrectly implemented the RPC getProxy for the JournalProtocol client side. It sets retry policies and other policies that are not necessary -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2284) Write Http access to HDFS
[ https://issues.apache.org/jira/browse/HDFS-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136229#comment-13136229 ] Hudson commented on HDFS-2284: -- Integrated in Hadoop-Hdfs-0.23-Commit #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/56/]) svn merge -c 1166009 from trunk for HDFS-2284. svn merge -c 1167662 from trunk for HDFS-2284. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189384 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FSMainOperationsBaseTest.java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189375 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BooleanParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DstPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumSetParam.java *
[jira] [Commented] (HDFS-2284) Write Http access to HDFS
[ https://issues.apache.org/jira/browse/HDFS-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136232#comment-13136232 ] Hudson commented on HDFS-2284: -- Integrated in Hadoop-Common-0.23-Commit #56 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/56/]) svn merge -c 1166009 from trunk for HDFS-2284. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189384 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FSMainOperationsBaseTest.java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java Write Http access to HDFS - Key: HDFS-2284 URL: https://issues.apache.org/jira/browse/HDFS-2284 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.24.0 Attachments: h2284_20110902b.patch, h2284_20110904_0.20s.patch, h2284_20110904b_0.20s.patch, h2284_20110905_0.20s.patch, h2284_20110906_0.20s.patch, h2284_20110906b.patch, h2284_20110906b_0.20s.patch, h2284_20110906c.patch, h2284_20110906c_0.20s.patch, h2284_20110906d.patch, h2284_20110907.patch, h2284_20110907_0.20s.patch, h2284_20110909.patch, h2284_20110909.patch, h2284_20110909_0.20s.patch HFTP allows on read access to HDFS via HTTP. Add write HTTP access to HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2284) Write Http access to HDFS
[ https://issues.apache.org/jira/browse/HDFS-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136235#comment-13136235 ] Hudson commented on HDFS-2284: -- Integrated in Hadoop-Mapreduce-0.23-Commit #55 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/55/]) svn merge -c 1167662 from trunk for HDFS-2284. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189375 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BooleanParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DstPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumSetParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/ExceptionHandler.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GroupParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/IntegerParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/LongParam.java *
[jira] [Commented] (HDFS-2494) [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.
[ https://issues.apache.org/jira/browse/HDFS-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136249#comment-13136249 ] Hudson commented on HDFS-2494: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1174 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1174/]) HDFS-2494. Close the streams and DFSClient in DatanodeWebHdfsMethods. Contributed by Uma Maheswara Rao G szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189360 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Key: HDFS-2494 URL: https://issues.apache.org/jira/browse/HDFS-2494 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Fix For: 0.20.205.1, 0.20.206.0, 0.24.0 Attachments: HDFS-2494-20.205Security.patch, HDFS-2494.patch, HDFS-2494.patch, h2494_20111024.patch, h2494_20111024b.patch As part of the reliable test, Scenario: Initially check the socket count. ---there are aroud 42 sockets are there. open the file with DataNode http address using op=OPEN request parameter about 500 times in loop. Wait for some time and check the socket count. --- There are thousands of ESTABLISHED sockets are growing. ~2052 Here is the netstat result: C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 C:\Users\umanetstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l 2042 This count is not coming down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2436) FSNamesystem.setTimes(..) expects the path is a file.
[
https://issues.apache.org/jira/browse/HDFS-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136251#comment-13136251
]
Arpit Gupta commented on HDFS-2436:
---
Also the delegation token rest api's throw a 500 stating a npe in the code. We
should also see if we can better handle the case where the user tries to get
the delegation token when security is off. 500 is not the ideal response as the
user has no idea what they did wrong.
FSNamesystem.setTimes(..) expects the path is a file.
-
Key: HDFS-2436
URL: https://issues.apache.org/jira/browse/HDFS-2436
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Arpit Gupta
FSNamesystem.setTimes(..) does not work if the path is a directory.
Arpit found this bug when testing webhdfs:
{quote}
settimes api is working when called on a file, but when called on a dir it
returns a 404. I should be able to set time on both a file and a directory.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2317) Read access to HDFS using HTTP REST
[ https://issues.apache.org/jira/browse/HDFS-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2317: - Fix Version/s: 0.23.0 Merged to 0.23. Read access to HDFS using HTTP REST --- Key: HDFS-2317 URL: https://issues.apache.org/jira/browse/HDFS-2317 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2317_20110912.patch, h2317_20110912b.patch, h2317_20110912c.patch, h2317_20110912c_0.20s.patch, h2317_20110912e.patch, h2317_20110912e_0.20s.patch, h2317_20110913.patch, h2317_20110913_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2284) Write Http access to HDFS
[ https://issues.apache.org/jira/browse/HDFS-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2284: - Fix Version/s: 0.23.0 Merged to 0.23. Write Http access to HDFS - Key: HDFS-2284 URL: https://issues.apache.org/jira/browse/HDFS-2284 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.23.0, 0.24.0 Attachments: h2284_20110902b.patch, h2284_20110904_0.20s.patch, h2284_20110904b_0.20s.patch, h2284_20110905_0.20s.patch, h2284_20110906_0.20s.patch, h2284_20110906b.patch, h2284_20110906b_0.20s.patch, h2284_20110906c.patch, h2284_20110906c_0.20s.patch, h2284_20110906d.patch, h2284_20110907.patch, h2284_20110907_0.20s.patch, h2284_20110909.patch, h2284_20110909.patch, h2284_20110909_0.20s.patch HFTP allows on read access to HDFS via HTTP. Add write HTTP access to HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2317) Read access to HDFS using HTTP REST
[ https://issues.apache.org/jira/browse/HDFS-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136269#comment-13136269 ] Hudson commented on HDFS-2317: -- Integrated in Hadoop-Common-0.23-Commit #57 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/57/]) svn merge -c 1170085 from trunk for HDFS-2317. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189405 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/ByteRangeInputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UriFsPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestFSMainOperationsWebHdfs.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Read access to HDFS using HTTP REST --- Key: HDFS-2317 URL: https://issues.apache.org/jira/browse/HDFS-2317 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2317_20110912.patch, h2317_20110912b.patch, h2317_20110912c.patch, h2317_20110912c_0.20s.patch, h2317_20110912e.patch, h2317_20110912e_0.20s.patch, h2317_20110913.patch, h2317_20110913_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2317) Read access to HDFS using HTTP REST
[ https://issues.apache.org/jira/browse/HDFS-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136271#comment-13136271 ] Hudson commented on HDFS-2317: -- Integrated in Hadoop-Hdfs-0.23-Commit #57 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/57/]) svn merge -c 1170085 from trunk for HDFS-2317. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189405 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/ByteRangeInputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UriFsPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestFSMainOperationsWebHdfs.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Read access to HDFS using HTTP REST --- Key: HDFS-2317 URL: https://issues.apache.org/jira/browse/HDFS-2317 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2317_20110912.patch, h2317_20110912b.patch, h2317_20110912c.patch, h2317_20110912c_0.20s.patch, h2317_20110912e.patch, h2317_20110912e_0.20s.patch, h2317_20110913.patch, h2317_20110913_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2333) HDFS-2284 introduced 2 findbugs warnings on trunk
[ https://issues.apache.org/jira/browse/HDFS-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2333: - Fix Version/s: 0.23.0 Merged to 0.23. HDFS-2284 introduced 2 findbugs warnings on trunk - Key: HDFS-2333 URL: https://issues.apache.org/jira/browse/HDFS-2333 Project: Hadoop HDFS Issue Type: Bug Reporter: Ivan Kelly Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2333.diff, h2333_20110914.patch, h2333_20110914b.patch, h2333_20110915.patch, h2333_20110915_0.20s.patch When HDFS-2284 was submitted it made DFSOutputStream public which triggered two SC_START_IN_CTOR findbug warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2331) Hdfs compilation fails
[
https://issues.apache.org/jira/browse/HDFS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2331:
-
Fix Version/s: 0.23.0
Merged to 0.23.
Hdfs compilation fails
--
Key: HDFS-2331
URL: https://issues.apache.org/jira/browse/HDFS-2331
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.20.205.0, 0.24.0
Reporter: Abhijit Suresh Shingate
Assignee: Abhijit Suresh Shingate
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: HDFS-2331-1.patch, HDFS-2331.patch
Original Estimate: 1h
Remaining Estimate: 1h
I am trying to perform complete build from trunk folder but the compilation
fails.
*Commandline:*
mvn clean install
*Error Message:*
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.
3.2:compile (default-compile) on project hadoop-hdfs: Compilation failure
[ERROR] \Hadoop\SVN\trunk\hadoop-hdfs-project\hadoop-hdfs\src\main\java\org
\apache\hadoop\hdfs\web\WebHdfsFileSystem.java:[209,21] type parameters of
TT
cannot be determined; no unique maximal instance exists for type variable T
with
upper bounds T,java.lang.Object
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
rea
d the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExc
eption
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
This is because of known reason
[http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954]
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the code snippet with issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return jsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
Fix to the issue
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the fix to the issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return WebHdfsFileSystem.TjsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2338) Configuration option to enable/disable webhdfs.
[ https://issues.apache.org/jira/browse/HDFS-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2338: - Fix Version/s: 0.23.0 Merged to 0.23. Configuration option to enable/disable webhdfs. --- Key: HDFS-2338 URL: https://issues.apache.org/jira/browse/HDFS-2338 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2338.1.patch, HDFS-2338.20s.1.patch, HDFS-2338.20s.2.patch We should add a configuration option to enable/disable webhdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2331) Hdfs compilation fails
[
https://issues.apache.org/jira/browse/HDFS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136287#comment-13136287
]
Hudson commented on HDFS-2331:
--
Integrated in Hadoop-Hdfs-0.23-Commit #58 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/58/])
svn merge -c 1170996 from trunk for HDFS-2331.
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189410
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
Hdfs compilation fails
--
Key: HDFS-2331
URL: https://issues.apache.org/jira/browse/HDFS-2331
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.20.205.0, 0.24.0
Reporter: Abhijit Suresh Shingate
Assignee: Abhijit Suresh Shingate
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: HDFS-2331-1.patch, HDFS-2331.patch
Original Estimate: 1h
Remaining Estimate: 1h
I am trying to perform complete build from trunk folder but the compilation
fails.
*Commandline:*
mvn clean install
*Error Message:*
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.
3.2:compile (default-compile) on project hadoop-hdfs: Compilation failure
[ERROR] \Hadoop\SVN\trunk\hadoop-hdfs-project\hadoop-hdfs\src\main\java\org
\apache\hadoop\hdfs\web\WebHdfsFileSystem.java:[209,21] type parameters of
TT
cannot be determined; no unique maximal instance exists for type variable T
with
upper bounds T,java.lang.Object
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
rea
d the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExc
eption
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
This is because of known reason
[http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954]
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the code snippet with issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return jsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
Fix to the issue
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the fix to the issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return WebHdfsFileSystem.TjsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2338) Configuration option to enable/disable webhdfs.
[ https://issues.apache.org/jira/browse/HDFS-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136285#comment-13136285 ] Hudson commented on HDFS-2338: -- Integrated in Hadoop-Hdfs-0.23-Commit #58 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/58/]) svn merge -c 1171379 from trunk for HDFS-2338. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189415 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java Configuration option to enable/disable webhdfs. --- Key: HDFS-2338 URL: https://issues.apache.org/jira/browse/HDFS-2338 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2338.1.patch, HDFS-2338.20s.1.patch, HDFS-2338.20s.2.patch We should add a configuration option to enable/disable webhdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2333) HDFS-2284 introduced 2 findbugs warnings on trunk
[ https://issues.apache.org/jira/browse/HDFS-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136283#comment-13136283 ] Hudson commented on HDFS-2333: -- Integrated in Hadoop-Common-0.23-Commit #58 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/58/]) svn merge -c 1171136 from trunk for HDFS-2333. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189411 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java HDFS-2284 introduced 2 findbugs warnings on trunk - Key: HDFS-2333 URL: https://issues.apache.org/jira/browse/HDFS-2333 Project: Hadoop HDFS Issue Type: Bug Reporter: Ivan Kelly Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2333.diff, h2333_20110914.patch, h2333_20110914b.patch, h2333_20110915.patch, h2333_20110915_0.20s.patch When HDFS-2284 was submitted it made DFSOutputStream public which triggered two SC_START_IN_CTOR findbug warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2333) HDFS-2284 introduced 2 findbugs warnings on trunk
[ https://issues.apache.org/jira/browse/HDFS-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136286#comment-13136286 ] Hudson commented on HDFS-2333: -- Integrated in Hadoop-Hdfs-0.23-Commit #58 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/58/]) svn merge -c 1171136 from trunk for HDFS-2333. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189411 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java HDFS-2284 introduced 2 findbugs warnings on trunk - Key: HDFS-2333 URL: https://issues.apache.org/jira/browse/HDFS-2333 Project: Hadoop HDFS Issue Type: Bug Reporter: Ivan Kelly Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2333.diff, h2333_20110914.patch, h2333_20110914b.patch, h2333_20110915.patch, h2333_20110915_0.20s.patch When HDFS-2284 was submitted it made DFSOutputStream public which triggered two SC_START_IN_CTOR findbug warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2331) Hdfs compilation fails
[
https://issues.apache.org/jira/browse/HDFS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136284#comment-13136284
]
Hudson commented on HDFS-2331:
--
Integrated in Hadoop-Common-0.23-Commit #58 (See
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/58/])
svn merge -c 1170996 from trunk for HDFS-2331.
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189410
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
Hdfs compilation fails
--
Key: HDFS-2331
URL: https://issues.apache.org/jira/browse/HDFS-2331
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.20.205.0, 0.24.0
Reporter: Abhijit Suresh Shingate
Assignee: Abhijit Suresh Shingate
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: HDFS-2331-1.patch, HDFS-2331.patch
Original Estimate: 1h
Remaining Estimate: 1h
I am trying to perform complete build from trunk folder but the compilation
fails.
*Commandline:*
mvn clean install
*Error Message:*
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.
3.2:compile (default-compile) on project hadoop-hdfs: Compilation failure
[ERROR] \Hadoop\SVN\trunk\hadoop-hdfs-project\hadoop-hdfs\src\main\java\org
\apache\hadoop\hdfs\web\WebHdfsFileSystem.java:[209,21] type parameters of
TT
cannot be determined; no unique maximal instance exists for type variable T
with
upper bounds T,java.lang.Object
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
rea
d the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExc
eption
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
This is because of known reason
[http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954]
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the code snippet with issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return jsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
Fix to the issue
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the fix to the issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return WebHdfsFileSystem.TjsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2318) Provide authentication to webhdfs using SPNEGO
[ https://issues.apache.org/jira/browse/HDFS-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2318: - Fix Version/s: 0.23.0 Merged to 0.23. Provide authentication to webhdfs using SPNEGO -- Key: HDFS-2318 URL: https://issues.apache.org/jira/browse/HDFS-2318 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2318_20110914.patch, h2318_20110914c.patch, h2318_20110914c_0.20s.patch, h2318_20110915.patch, h2318_20110915_0.20s.patch, h2318_20110915b.patch, h2318_20110915b_0.20s.patch, h2318_20110916.patch, h2318_20110916_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2507) HA: Allow saveNamespace operations to be canceled
[ https://issues.apache.org/jira/browse/HDFS-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136290#comment-13136290 ] Hadoop QA commented on HDFS-2507: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500928/hdfs-2507.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.TestBalancerBandwidth +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1465//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1465//console This message is automatically generated. HA: Allow saveNamespace operations to be canceled - Key: HDFS-2507 URL: https://issues.apache.org/jira/browse/HDFS-2507 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: HA branch (HDFS-1623), 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2507.txt For HDFS-2291 (checkpointing in HA setup) we need to allow the StandbyNode to handle a failover while it may be in the middle of saving a checkpoint. This implies that checkpoints need to be abortable. This JIRA is to implement this functionality. Even though this is only needed for HA, I think we should do the infrastructure in trunk to simplify future merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2338) Configuration option to enable/disable webhdfs.
[ https://issues.apache.org/jira/browse/HDFS-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136294#comment-13136294 ] Hudson commented on HDFS-2338: -- Integrated in Hadoop-Common-0.23-Commit #59 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/59/]) svn merge -c 1171379 from trunk for HDFS-2338. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189415 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java Configuration option to enable/disable webhdfs. --- Key: HDFS-2338 URL: https://issues.apache.org/jira/browse/HDFS-2338 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2338.1.patch, HDFS-2338.20s.1.patch, HDFS-2338.20s.2.patch We should add a configuration option to enable/disable webhdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2318) Provide authentication to webhdfs using SPNEGO
[ https://issues.apache.org/jira/browse/HDFS-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136295#comment-13136295 ] Hudson commented on HDFS-2318: -- Integrated in Hadoop-Common-0.23-Commit #59 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/59/]) svn merge -c 1171611 from trunk for HDFS-2318. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189418 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetDelegationTokenServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilter.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/KerberosUgiAuthenticator.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserProvider.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestFSMainOperationsWebHdfs.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Provide authentication to webhdfs using SPNEGO -- Key: HDFS-2318 URL: https://issues.apache.org/jira/browse/HDFS-2318 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2318_20110914.patch, h2318_20110914c.patch, h2318_20110914c_0.20s.patch, h2318_20110915.patch, h2318_20110915_0.20s.patch, h2318_20110915b.patch, h2318_20110915b_0.20s.patch, h2318_20110916.patch, h2318_20110916_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2318) Provide authentication to webhdfs using SPNEGO
[ https://issues.apache.org/jira/browse/HDFS-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136292#comment-13136292 ] Hudson commented on HDFS-2318: -- Integrated in Hadoop-Hdfs-0.23-Commit #59 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/59/]) svn merge -c 1171611 from trunk for HDFS-2318. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189418 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetDelegationTokenServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilter.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/KerberosUgiAuthenticator.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserProvider.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestFSMainOperationsWebHdfs.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Provide authentication to webhdfs using SPNEGO -- Key: HDFS-2318 URL: https://issues.apache.org/jira/browse/HDFS-2318 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2318_20110914.patch, h2318_20110914c.patch, h2318_20110914c_0.20s.patch, h2318_20110915.patch, h2318_20110915_0.20s.patch, h2318_20110915b.patch, h2318_20110915b_0.20s.patch, h2318_20110916.patch, h2318_20110916_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2284) Write Http access to HDFS
[ https://issues.apache.org/jira/browse/HDFS-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136296#comment-13136296 ] Hudson commented on HDFS-2284: -- Integrated in Hadoop-Mapreduce-0.23-Commit #56 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/56/]) svn merge -c 1166009 from trunk for HDFS-2284. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189384 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-common-project * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FSMainOperationsBaseTest.java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java Write Http access to HDFS - Key: HDFS-2284 URL: https://issues.apache.org/jira/browse/HDFS-2284 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Sanjay Radia Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.23.0, 0.24.0 Attachments: h2284_20110902b.patch, h2284_20110904_0.20s.patch, h2284_20110904b_0.20s.patch, h2284_20110905_0.20s.patch, h2284_20110906_0.20s.patch, h2284_20110906b.patch, h2284_20110906b_0.20s.patch, h2284_20110906c.patch, h2284_20110906c_0.20s.patch, h2284_20110906d.patch, h2284_20110907.patch, h2284_20110907_0.20s.patch, h2284_20110909.patch, h2284_20110909.patch, h2284_20110909_0.20s.patch HFTP allows on read access to HDFS via HTTP. Add write HTTP access to HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2340) Support getFileBlockLocations and getDelegationToken in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2340: - Fix Version/s: 0.23.0 Merged to 0.23. Support getFileBlockLocations and getDelegationToken in webhdfs --- Key: HDFS-2340 URL: https://issues.apache.org/jira/browse/HDFS-2340 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2340_20110918b.patch, h2340_20110919.patch, h2340_20110919_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2487) HA: Administrative CLI to control HA daemons
[ https://issues.apache.org/jira/browse/HDFS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reassigned HDFS-2487: - Assignee: Todd Lipcon (was: Aaron T. Myers) HA: Administrative CLI to control HA daemons Key: HDFS-2487 URL: https://issues.apache.org/jira/browse/HDFS-2487 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs client Affects Versions: HA branch (HDFS-1623) Reporter: Aaron T. Myers Assignee: Todd Lipcon We'll need to have some way of controlling the HA nodes while they're live, probably by adding some more commands to dfsadmin. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2356) webhdfs: support case insensitive query parameter names
[ https://issues.apache.org/jira/browse/HDFS-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2356: - Fix Version/s: 0.23.0 Merged to 0.23. webhdfs: support case insensitive query parameter names --- Key: HDFS-2356 URL: https://issues.apache.org/jira/browse/HDFS-2356 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2356_20110922.patch, h2356_20110922_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2450) Only complete hostname is supported to access data via hdfs://
[
https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136305#comment-13136305
]
Daryn Sharp commented on HDFS-2450:
---
# can you get thatScheme before making if ...
#* Gladly! That was one of the aforementioned things I was itching to do,
but removed before posting the patch to minimize the change.
# All checks seem to be using defaultPort
#* No, the default port is only used if there is not a port in the uri
# Why remove?
#* Default uri is sufficient because {{FileSystem.get(defaultUri,
conf).getUri()}} equals {{defaultUri}}. Also note about instantiating the
default fs:
#*# It's relatively expensive, and horrifically bad if the fs cache is disabled
to create a new fs
#*# If the default fs is down, the thread blocks while rpc retries occur
#*# The thrown {{RuntimeException}} is likely blow up the client, as opposed to
the {{IllegalArgumentException}} the caller expects.
# If thisAuthority and thatAuthority is null, the new mehod throw
IllegalArgumentException
#* No, the == on the authorities will catch this case.
# If thisAuthority and thatAuthority are not null, you end up parsing the URI
for thatAuthority twice.
#* Yes, once before canonicalization, once after canonicalization.
# DistributedFileSystem#makeQualified remove makes me concerned
#* The only reason it existed is because {{NameNode.getUri()}} was stripping
off the default port. This alone would cause {{p = new
Path(hdfs://host:default-port/file); fs = p.getFileSystem(conf)}} to be
incompatible because the path has a port, but the fs stripped it off. So an
override of {{checkPath}} and {{makeQualified}} were added to strip the default
port from paths too. All of it is completely unnecessary, especially with
canonicalization.
Tests:
# full test suite
# manual testing with FsShell ls:
#* scheme://short
#* scheme://short:port
#* scheme://short.partial
#* scheme://short.partial:port
#* scheme://fqdn
#* scheme://fqdn:port
#* scheme://ip
#* scheme://ip:port
# adding more unit tests to automate stressing {{checkPath}} via {{DFS}}, both
positive and negative cases
Only complete hostname is supported to access data via hdfs://
--
Key: HDFS-2450
URL: https://issues.apache.org/jira/browse/HDFS-2450
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Rajit Saha
Assignee: Daryn Sharp
Attachments: HDFS-2450-1.patch, HDFS-2450-2.patch, HDFS-2450.patch,
IP vs. Hostname.pdf
If my complete hostname is host1.abc.xyz.com, only complete hostname must be
used to access data via hdfs://
I am running following in .20.205 Client to get data from .20.205 NN (host1)
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1/tmp
copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal localsrc ... dst]
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc/tmp/
copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected:
hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal localsrc ... dst]
$hadoop dfs -copyFromLocal /etc/passwd hftp://host1.abc.xyz/tmp/
copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected:
hdfs://host1.abc.xyz.com
Usage: java FsShell [-copyFromLocal localsrc ... dst]
Only following is supported
$hadoop dfs -copyFromLocal /etc/passwd hdfs://host1.abc.xyz.com/tmp/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2366) webhdfs throws a npe when ugi is null from getDelegationToken
[ https://issues.apache.org/jira/browse/HDFS-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2366: - Fix Version/s: 0.23.0 Merged to 0.23. webhdfs throws a npe when ugi is null from getDelegationToken - Key: HDFS-2366 URL: https://issues.apache.org/jira/browse/HDFS-2366 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2366_20110926.patch, h2366_20110926_0.20s.patch, h2366_20110926_0.20s_b.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2366) webhdfs throws a npe when ugi is null from getDelegationToken
[ https://issues.apache.org/jira/browse/HDFS-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136311#comment-13136311 ] Hudson commented on HDFS-2366: -- Integrated in Hadoop-Common-0.23-Commit #60 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/60/]) svn merge -c 1176178 from trunk for HDFS-2366. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189425 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java webhdfs throws a npe when ugi is null from getDelegationToken - Key: HDFS-2366 URL: https://issues.apache.org/jira/browse/HDFS-2366 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2366_20110926.patch, h2366_20110926_0.20s.patch, h2366_20110926_0.20s_b.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2340) Support getFileBlockLocations and getDelegationToken in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136310#comment-13136310 ] Hudson commented on HDFS-2340: -- Integrated in Hadoop-Common-0.23-Commit #60 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/60/]) svn merge -c 1173468 from trunk for HDFS-2340. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189421 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OverwriteParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/RenewerParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Support getFileBlockLocations and getDelegationToken in webhdfs --- Key: HDFS-2340 URL: https://issues.apache.org/jira/browse/HDFS-2340 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2340_20110918b.patch, h2340_20110919.patch, h2340_20110919_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2340) Support getFileBlockLocations and getDelegationToken in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136313#comment-13136313 ] Hudson commented on HDFS-2340: -- Integrated in Hadoop-Hdfs-0.23-Commit #60 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/60/]) svn merge -c 1173468 from trunk for HDFS-2340. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189421 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OverwriteParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/RenewerParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Support getFileBlockLocations and getDelegationToken in webhdfs --- Key: HDFS-2340 URL: https://issues.apache.org/jira/browse/HDFS-2340 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2340_20110918b.patch, h2340_20110919.patch, h2340_20110919_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2356) webhdfs: support case insensitive query parameter names
[ https://issues.apache.org/jira/browse/HDFS-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136314#comment-13136314 ] Hudson commented on HDFS-2356: -- Integrated in Hadoop-Hdfs-0.23-Commit #60 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/60/]) svn merge -c 1175113 from trunk for HDFS-2356. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189424 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/ParamFilter.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DstPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/ModificationTimeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/PostOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/PutOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/RenameOptionSetParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java webhdfs: support case insensitive query parameter names --- Key: HDFS-2356 URL: https://issues.apache.org/jira/browse/HDFS-2356 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2356_20110922.patch, h2356_20110922_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2487) HA: Administrative CLI to control HA daemons
[
https://issues.apache.org/jira/browse/HDFS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-2487:
--
Attachment: hdfs-2487.txt
Here's a patch which adds an HAAdmin command line tool.
Its usage output:
{code}
Usage: java HAAdmin
[-transitionToActive host:port]
[-transitionToStandby host:port]
[-checkHealth host:port]
[-help command]
{code}
Notes:
- I didn't add it to the bin/hadoop wrapper script since this is mostly going
to be used as a testing tool for now. Actual failovers will be done through the
failover controller.
- Similarly, I didn't try to integrate this with configuration for NN addresses
or anything - this is a patch for common rather than HDFS. I'll move the JIRA
to common.
HA: Administrative CLI to control HA daemons
Key: HDFS-2487
URL: https://issues.apache.org/jira/browse/HDFS-2487
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs client
Affects Versions: HA branch (HDFS-1623)
Reporter: Aaron T. Myers
Assignee: Todd Lipcon
Attachments: hdfs-2487.txt
We'll need to have some way of controlling the HA nodes while they're live,
probably by adding some more commands to dfsadmin.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2487) HA: Administrative CLI to control HA daemons
[ https://issues.apache.org/jira/browse/HDFS-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-2487: -- Issue Type: New Feature (was: Sub-task) Parent: (was: HDFS-1623) HA: Administrative CLI to control HA daemons Key: HDFS-2487 URL: https://issues.apache.org/jira/browse/HDFS-2487 Project: Hadoop HDFS Issue Type: New Feature Components: util Affects Versions: HA Branch (HDFS-1623) Reporter: Aaron T. Myers Assignee: Todd Lipcon Attachments: hdfs-2487.txt We'll need to have some way of controlling the HA nodes while they're live, probably by adding some more commands to dfsadmin. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1869) mkdirs should use the supplied permission for all of the created directories
[ https://issues.apache.org/jira/browse/HDFS-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136321#comment-13136321 ] Hadoop QA commented on HDFS-1869: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500466/HDFS-1869-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1467//console This message is automatically generated. mkdirs should use the supplied permission for all of the created directories Key: HDFS-1869 URL: https://issues.apache.org/jira/browse/HDFS-1869 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Fix For: 0.23.0 Attachments: HDFS-1869-2.patch, HDFS-1869-3.patch, HDFS-1869.patch Mkdirs only uses the supplied FsPermission for the last directory of the path. Paths 0..N-1 will all inherit the parent dir's permissions -even if- inheritPermission is false. This is a regression from somewhere around 0.20.9 and does not follow posix semantics. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2507) HA: Allow saveNamespace operations to be canceled
[ https://issues.apache.org/jira/browse/HDFS-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136324#comment-13136324 ] Todd Lipcon commented on HDFS-2507: --- The failing tests are failing because TestDfsOverAvroRpc failed right before it, leaving a locked namespace directory in place. This is not a new issue due to this patch. This should be ready for review. HA: Allow saveNamespace operations to be canceled - Key: HDFS-2507 URL: https://issues.apache.org/jira/browse/HDFS-2507 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: HA branch (HDFS-1623), 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-2507.txt For HDFS-2291 (checkpointing in HA setup) we need to allow the StandbyNode to handle a failover while it may be in the middle of saving a checkpoint. This implies that checkpoints need to be abortable. This JIRA is to implement this functionality. Even though this is only needed for HA, I think we should do the infrastructure in trunk to simplify future merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2366) webhdfs throws a npe when ugi is null from getDelegationToken
[ https://issues.apache.org/jira/browse/HDFS-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136329#comment-13136329 ] Hudson commented on HDFS-2366: -- Integrated in Hadoop-Hdfs-0.23-Commit #61 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/61/]) svn merge -c 1176178 from trunk for HDFS-2366. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189425 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java webhdfs throws a npe when ugi is null from getDelegationToken - Key: HDFS-2366 URL: https://issues.apache.org/jira/browse/HDFS-2366 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2366_20110926.patch, h2366_20110926_0.20s.patch, h2366_20110926_0.20s_b.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2368) defaults created for web keytab and principal, these properties should not have defaults
[
https://issues.apache.org/jira/browse/HDFS-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2368:
-
Fix Version/s: 0.23.0
Merged to 0.23.
defaults created for web keytab and principal, these properties should not
have defaults
Key: HDFS-2368
URL: https://issues.apache.org/jira/browse/HDFS-2368
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Arpit Gupta
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: h2368_20110927.patch, h2368_20110927_0.20s.patch
the following defaults are set in hdfs-defaults.xml
property
namedfs.web.authentication.kerberos.principal/name
valueHTTP/${dfs.web.hostname}@${kerberos.realm}/value
description
The HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint.
The HTTP Kerberos principal MUST start with 'HTTP/' per Kerberos
HTTP SPENGO specification.
/description
/property
property
namedfs.web.authentication.kerberos.keytab/name
value${user.home}/dfs.web.keytab/value
description
The Kerberos keytab file with the credentials for the
HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint.
/description
/property
These properties should not have defaults
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2317) Read access to HDFS using HTTP REST
[ https://issues.apache.org/jira/browse/HDFS-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136336#comment-13136336 ] Hudson commented on HDFS-2317: -- Integrated in Hadoop-Mapreduce-0.23-Commit #57 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/57/]) svn merge -c 1170085 from trunk for HDFS-2317. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189405 Files : * /hadoop/common/branches/branch-0.23 * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/ByteRangeInputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UriFsPathParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestFSMainOperationsWebHdfs.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Read access to HDFS using HTTP REST --- Key: HDFS-2317 URL: https://issues.apache.org/jira/browse/HDFS-2317 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2317_20110912.patch, h2317_20110912b.patch, h2317_20110912c.patch, h2317_20110912c_0.20s.patch, h2317_20110912e.patch, h2317_20110912e_0.20s.patch, h2317_20110913.patch, h2317_20110913_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2338) Configuration option to enable/disable webhdfs.
[ https://issues.apache.org/jira/browse/HDFS-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136337#comment-13136337 ] Hudson commented on HDFS-2338: -- Integrated in Hadoop-Mapreduce-0.23-Commit #57 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/57/]) svn merge -c 1171379 from trunk for HDFS-2338. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189415 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java Configuration option to enable/disable webhdfs. --- Key: HDFS-2338 URL: https://issues.apache.org/jira/browse/HDFS-2338 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2338.1.patch, HDFS-2338.20s.1.patch, HDFS-2338.20s.2.patch We should add a configuration option to enable/disable webhdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2333) HDFS-2284 introduced 2 findbugs warnings on trunk
[ https://issues.apache.org/jira/browse/HDFS-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136339#comment-13136339 ] Hudson commented on HDFS-2333: -- Integrated in Hadoop-Mapreduce-0.23-Commit #57 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/57/]) svn merge -c 1171136 from trunk for HDFS-2333. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189411 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java HDFS-2284 introduced 2 findbugs warnings on trunk - Key: HDFS-2333 URL: https://issues.apache.org/jira/browse/HDFS-2333 Project: Hadoop HDFS Issue Type: Bug Reporter: Ivan Kelly Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: HDFS-2333.diff, h2333_20110914.patch, h2333_20110914b.patch, h2333_20110915.patch, h2333_20110915_0.20s.patch When HDFS-2284 was submitted it made DFSOutputStream public which triggered two SC_START_IN_CTOR findbug warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2318) Provide authentication to webhdfs using SPNEGO
[ https://issues.apache.org/jira/browse/HDFS-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136338#comment-13136338 ] Hudson commented on HDFS-2318: -- Integrated in Hadoop-Mapreduce-0.23-Commit #57 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/57/]) svn merge -c 1171611 from trunk for HDFS-2318. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189418 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetDelegationTokenServlet.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilter.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/KerberosUgiAuthenticator.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BlockSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserProvider.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestFSMainOperationsWebHdfs.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java Provide authentication to webhdfs using SPNEGO -- Key: HDFS-2318 URL: https://issues.apache.org/jira/browse/HDFS-2318 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0 Attachments: h2318_20110914.patch, h2318_20110914c.patch, h2318_20110914c_0.20s.patch, h2318_20110915.patch, h2318_20110915_0.20s.patch, h2318_20110915b.patch, h2318_20110915b_0.20s.patch, h2318_20110916.patch, h2318_20110916_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2331) Hdfs compilation fails
[
https://issues.apache.org/jira/browse/HDFS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136340#comment-13136340
]
Hudson commented on HDFS-2331:
--
Integrated in Hadoop-Mapreduce-0.23-Commit #57 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/57/])
svn merge -c 1170996 from trunk for HDFS-2331.
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189410
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
Hdfs compilation fails
--
Key: HDFS-2331
URL: https://issues.apache.org/jira/browse/HDFS-2331
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.20.205.0, 0.24.0
Reporter: Abhijit Suresh Shingate
Assignee: Abhijit Suresh Shingate
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: HDFS-2331-1.patch, HDFS-2331.patch
Original Estimate: 1h
Remaining Estimate: 1h
I am trying to perform complete build from trunk folder but the compilation
fails.
*Commandline:*
mvn clean install
*Error Message:*
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.
3.2:compile (default-compile) on project hadoop-hdfs: Compilation failure
[ERROR] \Hadoop\SVN\trunk\hadoop-hdfs-project\hadoop-hdfs\src\main\java\org
\apache\hadoop\hdfs\web\WebHdfsFileSystem.java:[209,21] type parameters of
TT
cannot be determined; no unique maximal instance exists for type variable T
with
upper bounds T,java.lang.Object
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
rea
d the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExc
eption
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
This is because of known reason
[http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954]
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the code snippet with issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return jsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
Fix to the issue
{code:title=WebHdfsFileSystem.java|borderStyle=solid}
Following is the fix to the issue.
private T T run(final HttpOpParam.Op op, final Path fspath,
final Param?,?... parameters) throws IOException {
final HttpURLConnection conn = httpConnect(op, fspath, parameters);
validateResponse(op, conn);
try {
return WebHdfsFileSystem.TjsonParse(conn.getInputStream());
} finally {
conn.disconnect();
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2368) defaults created for web keytab and principal, these properties should not have defaults
[
https://issues.apache.org/jira/browse/HDFS-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136343#comment-13136343
]
Hudson commented on HDFS-2368:
--
Integrated in Hadoop-Hdfs-0.23-Commit #62 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/62/])
svn merge -c 1176719 from trunk for HDFS-2368.
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189435
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
defaults created for web keytab and principal, these properties should not
have defaults
Key: HDFS-2368
URL: https://issues.apache.org/jira/browse/HDFS-2368
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Arpit Gupta
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: h2368_20110927.patch, h2368_20110927_0.20s.patch
the following defaults are set in hdfs-defaults.xml
property
namedfs.web.authentication.kerberos.principal/name
valueHTTP/${dfs.web.hostname}@${kerberos.realm}/value
description
The HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint.
The HTTP Kerberos principal MUST start with 'HTTP/' per Kerberos
HTTP SPENGO specification.
/description
/property
property
namedfs.web.authentication.kerberos.keytab/name
value${user.home}/dfs.web.keytab/value
description
The Kerberos keytab file with the credentials for the
HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint.
/description
/property
These properties should not have defaults
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2368) defaults created for web keytab and principal, these properties should not have defaults
[
https://issues.apache.org/jira/browse/HDFS-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136341#comment-13136341
]
Hudson commented on HDFS-2368:
--
Integrated in Hadoop-Common-0.23-Commit #61 (See
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/61/])
svn merge -c 1176719 from trunk for HDFS-2368.
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1189435
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
defaults created for web keytab and principal, these properties should not
have defaults
Key: HDFS-2368
URL: https://issues.apache.org/jira/browse/HDFS-2368
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.205.0
Reporter: Arpit Gupta
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.20.205.0, 0.20.206.0, 0.23.0, 0.24.0
Attachments: h2368_20110927.patch, h2368_20110927_0.20s.patch
the following defaults are set in hdfs-defaults.xml
property
namedfs.web.authentication.kerberos.principal/name
valueHTTP/${dfs.web.hostname}@${kerberos.realm}/value
description
The HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint.
The HTTP Kerberos principal MUST start with 'HTTP/' per Kerberos
HTTP SPENGO specification.
/description
/property
property
namedfs.web.authentication.kerberos.keytab/name
value${user.home}/dfs.web.keytab/value
description
The Kerberos keytab file with the credentials for the
HTTP Kerberos principal used by Hadoop-Auth in the HTTP endpoint.
/description
/property
These properties should not have defaults
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2371) Refactor BlockSender.java for better readability
[ https://issues.apache.org/jira/browse/HDFS-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136347#comment-13136347 ] Tsz Wo (Nicholas), SZE commented on HDFS-2371: -- How about merging this to 0.23? Refactor BlockSender.java for better readability Key: HDFS-2371 URL: https://issues.apache.org/jira/browse/HDFS-2371 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2371.txt, HDFS-2371.txt, HDFS-2371.txt BlockSender.java is hard to read and understand. I propose refactoring it for better readability -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
