date:20120224

[
https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215721#comment-13215721
]

Hadoop QA commented on HDFS-2492:
-

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12515926/HDFS-2492-blockmanager.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1900//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1900//console

This message is automatically generated.

BlockManager cross-rack replication checks only work for ScriptBasedMapping
---

Key: HDFS-2492
URL: https://issues.apache.org/jira/browse/HDFS-2492
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.23.0, 0.24.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Priority: Minor
Fix For: 0.24.0, 0.23.1

Attachments: HDFS-2492-blockmanager.patch,
HDFS-2492-blockmanager.patch, HDFS-2492-blockmanager.patch,
HDFS-2492-blockmanager.patch, HDFS-2492-blockmanager.patch

The BlockManager cross-rack replication checks only works if script files are
used for replication, not if alternate plugins provide the topology
information.
This is because the BlockManager sets its rack checking flag if there is a
filename key
{code}
shouldCheckForEnoughRacks =
conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY) != null;
{code}
yet this filename key is only used if the topology mapper defined by
{code}
DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
{code}
is an instance of {{ScriptBasedMapping}}
If any other mapper is used, the system may be multi rack, but the Block
Manager will not be aware of this fact unless the filename key is set to
something non-null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3003) DFSUtil is a better palce for method getHostPortString() than NameNode

[
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215733#comment-13215733
]

Hadoop QA commented on HDFS-3003:
-

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515859/HDFS-3003.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 21 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1901//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1901//console

This message is automatically generated.

DFSUtil is a better palce for method getHostPortString() than NameNode
--

Key: HDFS-3003
URL: https://issues.apache.org/jira/browse/HDFS-3003
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch

The method getHostPortString() is not directly related with NameNode, and
it's more of a utility method.

[jira] [Commented] (HDFS-3003) DFSUtil is a better palce for method getHostPortString() than NameNode

2012-02-24 Thread Aaron T. Myers (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215735#comment-13215735
 ] 

Aaron T. Myers commented on HDFS-3003:
--

+1, the patch looks good to me. I'll commit this momentarily.

 DFSUtil is a better palce for method getHostPortString() than NameNode
 --

 Key: HDFS-3003
 URL: https://issues.apache.org/jira/browse/HDFS-3003
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch


 The method getHostPortString() is not directly related with NameNode, and 
 it's more of a utility method.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()


 [ 
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron T. Myers updated HDFS-3003:
-

Summary: Remove getHostPortString() from NameNode, replace it with 
NetUtils.getHostPortString()  (was: DFSUtil is a better palce for method 
getHostPortString() than NameNode)

 Remove getHostPortString() from NameNode, replace it with 
 NetUtils.getHostPortString()
 --

 Key: HDFS-3003
 URL: https://issues.apache.org/jira/browse/HDFS-3003
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch


 The method getHostPortString() is not directly related with NameNode, and 
 it's more of a utility method.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()


 [ 
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron T. Myers updated HDFS-3003:
-

  Resolution: Fixed
   Fix Version/s: 0.24.0
Target Version/s: 0.24.0
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

I've just committed this patch to trunk. Thanks a lot, Brandon.

 Remove getHostPortString() from NameNode, replace it with 
 NetUtils.getHostPortString()
 --

 Key: HDFS-3003
 URL: https://issues.apache.org/jira/browse/HDFS-3003
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch


 The method getHostPortString() is not directly related with NameNode, and 
 it's more of a utility method.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()


[ 
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215747#comment-13215747
 ] 

Hudson commented on HDFS-3003:
--

Integrated in Hadoop-Hdfs-trunk-Commit #1845 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1845/])
HDFS-3003. Remove getHostPortString() from NameNode, replace it with 
NetUtils.getHostPortString(). Contributed by Brandon Li. (Revision 1293338)

 Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293338
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileChecksumServlets.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileDataServlet.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeJsp.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStreamFile.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestTransferFsImage.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java


 Remove getHostPortString() from NameNode, replace it with 
 NetUtils.getHostPortString()
 --

 Key: HDFS-3003
 URL: https://issues.apache.org/jira/browse/HDFS-3003
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch


 The method getHostPortString() is not directly related with NameNode, and 
 it's more of a utility method.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()


[ 
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215749#comment-13215749
 ] 

Hudson commented on HDFS-3003:
--

Integrated in Hadoop-Common-trunk-Commit #1771 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1771/])
HDFS-3003. Remove getHostPortString() from NameNode, replace it with 
NetUtils.getHostPortString(). Contributed by Brandon Li. (Revision 1293338)

 Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293338
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileChecksumServlets.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileDataServlet.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeJsp.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStreamFile.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestTransferFsImage.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java


 Remove getHostPortString() from NameNode, replace it with 
 NetUtils.getHostPortString()
 --

 Key: HDFS-3003
 URL: https://issues.apache.org/jira/browse/HDFS-3003
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch


 The method getHostPortString() is not directly related with NameNode, and 
 it's more of a utility method.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-3009) DFSclient islocaladdress() can use similar routine in netutils

2012-02-24 Thread Hari Mankude (Created) (JIRA)

DFSclient islocaladdress() can use similar routine in netutils
--

 Key: HDFS-3009
 URL: https://issues.apache.org/jira/browse/HDFS-3009
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Trivial


isLocalAddress() in dfsclient can use similar function in netutils

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-2978) The NameNode should expose name dir statuses via JMX


 [ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron T. Myers updated HDFS-2978:
-

Attachment: HDFS-2978.patch

Here's a patch for trunk which addresses the issue.

 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3008) Negative caching of local addrs doesn't work

2012-02-24 Thread Eli Collins (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eli Collins updated HDFS-3008:
--

Target Version/s: 1.1.0, 0.23.2  (was: 0.23.2, 1.1.0)
  Status: Patch Available  (was: Open)

 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Attachments: hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work

[
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215764#comment-13215764
]

Hadoop QA commented on HDFS-3008:
-

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515900/hdfs-3008.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

-1 javac. The patch appears to cause tar ant target to fail.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

-1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed the unit tests build

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1902//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1902//console

This message is automatically generated.

Negative caching of local addrs doesn't work

Key: HDFS-3008
URL: https://issues.apache.org/jira/browse/HDFS-3008
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
Attachments: hdfs-3008.txt

HDFS-2653 added negative caching of local addrs, however it still goes
through the fall through path every time if the address is non-local.

[jira] [Commented] (HDFS-2492) BlockManager cross-rack replication checks only work for ScriptBasedMapping

2012-02-24 Thread Steve Loughran (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215772#comment-13215772
 ] 

Steve Loughran commented on HDFS-2492:
--

no tests because the tests for this are in hadoop common; the existing test 
suite are the regression test, and now that TestDistributedUpgrade is no longer 
(spuriously) failing, I plan to commit this unless anyone vetos it

 BlockManager cross-rack replication checks only work for ScriptBasedMapping
 ---

 Key: HDFS-2492
 URL: https://issues.apache.org/jira/browse/HDFS-2492
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.0, 0.24.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Priority: Minor
 Fix For: 0.24.0, 0.23.1

 Attachments: HDFS-2492-blockmanager.patch, 
 HDFS-2492-blockmanager.patch, HDFS-2492-blockmanager.patch, 
 HDFS-2492-blockmanager.patch, HDFS-2492-blockmanager.patch


 The BlockManager cross-rack replication checks only works if script files are 
 used for replication, not if alternate plugins provide the topology 
 information.
 This is because the BlockManager sets its rack checking flag if there is a 
 filename key
 {code}
 shouldCheckForEnoughRacks = 
 conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY) != null;
 {code}
 yet this filename key is only used if the topology mapper defined by 
 {code}
 DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
 {code}
 is an instance of {{ScriptBasedMapping}}
 If any other mapper is used, the system may be multi rack, but the Block 
 Manager will not be aware of this fact unless the filename key is set to 
 something non-null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()


[ 
https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215776#comment-13215776
 ] 

Hudson commented on HDFS-3003:
--

Integrated in Hadoop-Mapreduce-trunk-Commit #1782 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1782/])
HDFS-3003. Remove getHostPortString() from NameNode, replace it with 
NetUtils.getHostPortString(). Contributed by Brandon Li. (Revision 1293338)

 Result = ABORTED
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293338
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileChecksumServlets.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileDataServlet.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeJsp.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestBackupNode.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStreamFile.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestTransferFsImage.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java


 Remove getHostPortString() from NameNode, replace it with 
 NetUtils.getHostPortString()
 --

 Key: HDFS-3003
 URL: https://issues.apache.org/jira/browse/HDFS-3003
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 0.24.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch


 The method getHostPortString() is not directly related with NameNode, and 
 it's more of a utility method.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-3010) Get performance on HA branch to match trunk

2012-02-24 Thread Todd Lipcon (Created) (JIRA)

Get performance on HA branch to match trunk
---

 Key: HDFS-3010
 URL: https://issues.apache.org/jira/browse/HDFS-3010
 Project: Hadoop HDFS
  Issue Type: Sub-task
Reporter: Todd Lipcon
Assignee: Todd Lipcon




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3010) Get performance on HA branch to match trunk

2012-02-24 Thread Todd Lipcon (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HDFS-3010:
--

  Component/s: name-node
   ha
  Description: As described in [this 
comment|https://issues.apache.org/jira/browse/HDFS-1623?focusedCommentId=13215309page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13215309]
 the performance of the HA branch for writes is significantly reduced compared 
to trunk. We need to dig a bit and optimize whatever it is that's hurting us in 
order to get back to the same performance numbers.
 Priority: Critical  (was: Major)
 Target Version/s: HA branch (HDFS-1623)
Affects Version/s: HA branch (HDFS-1623)

 Get performance on HA branch to match trunk
 ---

 Key: HDFS-3010
 URL: https://issues.apache.org/jira/browse/HDFS-3010
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical

 As described in [this 
 comment|https://issues.apache.org/jira/browse/HDFS-1623?focusedCommentId=13215309page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13215309]
  the performance of the HA branch for writes is significantly reduced 
 compared to trunk. We need to dig a bit and optimize whatever it is that's 
 hurting us in order to get back to the same performance numbers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-2978) The NameNode should expose name dir statuses via JMX


 [ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron T. Myers updated HDFS-2978:
-

Status: Patch Available  (was: Open)

 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3010) Get performance on HA branch to match trunk


[ 
https://issues.apache.org/jira/browse/HDFS-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215806#comment-13215806
 ] 

Todd Lipcon commented on HDFS-3010:
---

There are three possible components to the perf issue, I think:

1) DN now sends RBW replicas to both NNs as soon as a block starts to be 
created. This adds 3 RPCs to each block creation (though they don't write to 
the edit logs)
2) When blocks are allocated, we now log the full block list of that file. This 
creates a much bigger edit log, so of course takes more time.
3) When HA is enabled, these new edit log entries are fsynced, which makes it 
even slower.

I'm hoping to set up a cluster to test each of these in isolation by commenting 
out the related code from the HA branch and measuring a write benchmark. Once 
we identify which is the worst issue we can tackle it.

 Get performance on HA branch to match trunk
 ---

 Key: HDFS-3010
 URL: https://issues.apache.org/jira/browse/HDFS-3010
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical

 As described in [this 
 comment|https://issues.apache.org/jira/browse/HDFS-1623?focusedCommentId=13215309page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13215309]
  the performance of the HA branch for writes is significantly reduced 
 compared to trunk. We need to dig a bit and optimize whatever it is that's 
 hurting us in order to get back to the same performance numbers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3009) DFSclient islocaladdress() can use similar routine in netutils

[
https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215807#comment-13215807
]

Hadoop QA commented on HDFS-3009:
-

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515948/HDFS-3009.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1903//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1903//console

This message is automatically generated.

DFSclient islocaladdress() can use similar routine in netutils
--

Key: HDFS-3009
URL: https://issues.apache.org/jira/browse/HDFS-3009
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.0, 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Trivial
Attachments: HDFS-3009.patch

isLocalAddress() in dfsclient can use similar function in netutils

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

[
https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215812#comment-13215812
]

Todd Lipcon commented on HDFS-2904:
---

bq. I think we should not couple the idea of logical uri with the failover
proxy configuration. For example, someone might want to turnoff ha and not
provide failover proxy configurations, but the logical URIs should continue to
work.

This seems similar to your JIRA HDFS-2839, right? I don't disagree with you
that it would be nice to generalize the concept of logical URIs further, but as
it stands today, it is not general - this patch just works with the existing
design, rather than seeking to change it.

bq. In the cloneDelegationTokenForLogicalUri, should we clone one more token
for the standby as well for ha configurations?

Not sure I follow what you mean here. Before the
ConfiguredFailoverProxyProvider creates a new proxy, it calls this method to
clone the logical-service DT to the physical-address DT it's about to connect
to. So this works with any number of standby nodes.

bq. I think it is a good idea to clone a token for a logical service, to
multiple tokens with underlying service addresses. We should have it
irrespective of the failover proxy configuration i.e. whenever we have a
logical uri, we map it to actual service addresses and clone token for each.
Currently the only thing that can map to actual service addresses is the
proxy provider. This is similar to your #1 above. In the case of a ZK-based
failover, for example, there is no list of actual service addresses to
consult, unless it had explicit registration/deregistration steps.

HA: Client support for getting delegation tokens to an HA cluster
-

Key: HDFS-2904
URL: https://issues.apache.org/jira/browse/HDFS-2904
Project: Hadoop HDFS
Issue Type: Sub-task
Components: ha, hdfs client, name-node, security
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh

Currently we have server-side support for delegation tokens in HA, and some
tests to verify it, but the client throws NPEs when trying to fetch a DT.
This is because the cluster doesn't have a single hostname, but instead a
logical nameservice name.

[jira] [Commented] (HDFS-3009) DFSclient islocaladdress() can use similar routine in netutils

2012-02-24 Thread Suresh Srinivas (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215832#comment-13215832
 ] 

Suresh Srinivas commented on HDFS-3009:
---

+1. The code in method replacing the code is identical. No need for adding unit 
tests, since it is already covered in TestNetUtils.

 DFSclient islocaladdress() can use similar routine in netutils
 --

 Key: HDFS-3009
 URL: https://issues.apache.org/jira/browse/HDFS-3009
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.0, 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Trivial
 Attachments: HDFS-3009.patch


 isLocalAddress() in dfsclient can use similar function in netutils

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils

2012-02-24 Thread Suresh Srinivas (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Srinivas updated HDFS-3009:
--

   Resolution: Fixed
Fix Version/s: 0.24.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed the patch. Thank you Hari.

 DFSClient islocaladdress() can use similar routine in netutils
 --

 Key: HDFS-3009
 URL: https://issues.apache.org/jira/browse/HDFS-3009
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.0, 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3009.patch


 isLocalAddress() in dfsclient can use similar function in netutils

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils


[ 
https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215844#comment-13215844
 ] 

Hudson commented on HDFS-3009:
--

Integrated in Hadoop-Hdfs-trunk-Commit #1846 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1846/])
HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using 
NetUtils. Contributed by Hari Mankude. (Revision 1293390)

 Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293390
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java


 DFSClient islocaladdress() can use similar routine in netutils
 --

 Key: HDFS-3009
 URL: https://issues.apache.org/jira/browse/HDFS-3009
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.0, 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3009.patch


 isLocalAddress() in dfsclient can use similar function in netutils

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils


[ 
https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215846#comment-13215846
 ] 

Hudson commented on HDFS-3009:
--

Integrated in Hadoop-Common-trunk-Commit #1772 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1772/])
HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using 
NetUtils. Contributed by Hari Mankude. (Revision 1293390)

 Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293390
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java


 DFSClient islocaladdress() can use similar routine in netutils
 --

 Key: HDFS-3009
 URL: https://issues.apache.org/jira/browse/HDFS-3009
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.0, 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Trivial
 Fix For: 0.24.0

 Attachments: HDFS-3009.patch


 isLocalAddress() in dfsclient can use similar function in netutils

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215856#comment-13215856
]

Hadoop QA commented on HDFS-2978:
-

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515947/HDFS-2978.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1904//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1904//console

This message is automatically generated.

The NameNode should expose name dir statuses via JMX

Key: HDFS-2978
URL: https://issues.apache.org/jira/browse/HDFS-2978
Project: Hadoop HDFS
Issue Type: New Feature
Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Attachments: HDFS-2978.patch

We currently display this info on the NN web UI, so users who wish to monitor
this must either do it manually or parse HTML. We should publish this
information via JMX.

[jira] [Updated] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


 [ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE updated HDFS-3006:
-

Attachment: h3006_20120224.patch

h3006_20120224.patch: also fixed HttpFs.

 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3008) Negative caching of local addrs doesn't work

2012-02-24 Thread Eli Collins (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eli Collins updated HDFS-3008:
--

Attachment: hdfs-3008.txt

Thanks Suresh. Had to make a minor update to the patch, put parens around the 
ternary expression, which apparently is required by some compilers (worked in 
eclipse).

 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE updated HDFS-3006:
-

Attachment: h3006_20120224b.patch

The content-type is still JSON even if it is not set in the code.  It is using 
the preivous content-type.

h3006_20120224b.patch: setting it to APPLICATION_OCTET_STREAM.

 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HDFS-3001) dfsadmin -refreshServiceAcl fails Kerb authentication with valid Kerb ticket, other subcommands succeed

2012-02-24 Thread John George (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John George reassigned HDFS-3001:
-

Assignee: John George

 dfsadmin -refreshServiceAcl fails Kerb authentication with valid Kerb ticket, 
 other subcommands succeed
 ---

 Key: HDFS-3001
 URL: https://issues.apache.org/jira/browse/HDFS-3001
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1
Reporter: patrick white
Assignee: John George

 With a valid hdfs kerberos ticket, the dfsadmin subcommand 
 '-refreshServiceAcl' still fails on Kerb authentication. Please see the 
 comment for more details.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3008) Negative caching of local addrs doesn't work

2012-02-24 Thread Eli Collins (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eli Collins updated HDFS-3008:
--

  Resolution: Fixed
   Fix Version/s: 0.23.2
  1.1.0
Target Version/s:   (was: 0.23.2, 1.1.0)
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Thanks for the review Suresh. I've committed this and merged to branch-23 and 
branch-1.  

 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215925#comment-13215925
 ] 

Hudson commented on HDFS-3008:
--

Integrated in Hadoop-Hdfs-trunk-Commit #1847 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1847/])
HDFS-3008. Negative caching of local addrs doesn't work. Contributed by Eli 
Collins (Revision 1293419)

 Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293419
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java


 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215923#comment-13215923
 ] 

Tsz Wo (Nicholas), SZE commented on HDFS-3008:
--

Hi Eli,

Thanks for fixing this.

For merging to other branches, I think it is better to merge the specific 
project.  In this case, we should merge only hadoop-hdfs-project/hadoop-hdfs 
since all the changes are under it.  Otherwise, it will generate unnecessary 
merge info like below.  What do you think?

{noformat}
Author: eli
Date: Fri Feb 24 21:16:57 2012
New Revision: 1293421

URL: http://svn.apache.org/viewvc?rev=1293421view=rev
Log:
HDFS-3008. svn merge -c 1293419 from trunk

Modified:
hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/  (props 
changed)
hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt  
(props changed)
hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin/  (props 
changed)
...
{noformat}

 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215927#comment-13215927
 ] 

Hudson commented on HDFS-3008:
--

Integrated in Hadoop-Common-0.23-Commit #588 (See 
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/588/])
HDFS-3008. svn merge -c 1293419 from trunk (Revision 1293421)

 Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293421
Files : 
* /hadoop/common/branches/branch-0.23
* /hadoop/common/branches/branch-0.23/hadoop-common-project
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
* /hadoop/common/branches/branch-0.23/hadoop-project
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site


 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215928#comment-13215928
 ] 

Hudson commented on HDFS-3008:
--

Integrated in Hadoop-Hdfs-0.23-Commit #575 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/575/])
HDFS-3008. svn merge -c 1293419 from trunk (Revision 1293421)

 Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293421
Files : 
* /hadoop/common/branches/branch-0.23
* /hadoop/common/branches/branch-0.23/hadoop-common-project
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
* /hadoop/common/branches/branch-0.23/hadoop-project
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site


 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215932#comment-13215932
 ] 

Hudson commented on HDFS-3008:
--

Integrated in Hadoop-Common-trunk-Commit #1773 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1773/])
HDFS-3008. Negative caching of local addrs doesn't work. Contributed by Eli 
Collins (Revision 1293419)

 Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293419
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java


 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-3011) move webapps/ into the JAR

2012-02-24 Thread Alejandro Abdelnur (Created) (JIRA)

move webapps/ into the JAR
--

 Key: HDFS-3011
 URL: https://issues.apache.org/jira/browse/HDFS-3011
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: build, data-node, name-node
Affects Versions: 0.24.0, 0.23.2
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
 Fix For: 0.23.2


Currently the webapps dir is in the filesystem and added to the classpath.

As effectively it is picked up from the classpath, we could move into the 
component JAR itself.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Moved] (HDFS-3012) Exception while renewing delegation token

2012-02-24 Thread Robert Joseph Evans (Moved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Joseph Evans moved MAPREDUCE-3819 to HDFS-3012:
--

  Component/s: (was: mrv2)
 Target Version/s: 0.23.2  (was: 0.23.2)
Affects Version/s: (was: 0.23.1)
   0.23.1
  Key: HDFS-3012  (was: MAPREDUCE-3819)
  Project: Hadoop HDFS  (was: Hadoop Map/Reduce)

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3012) Exception while renewing delegation token

2012-02-24 Thread Robert Joseph Evans (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Joseph Evans updated HDFS-3012:
--

Attachment: HDFS-3012.txt

This patch was manually tested on a secure cluster and it does fix the issue.  
Unit tests would be very hard to write as it has to deal with configuration and 
the classpath to make it work.

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
 Attachments: HDFS-3012.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3012) Exception while renewing delegation token

2012-02-24 Thread Robert Joseph Evans (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Joseph Evans updated HDFS-3012:
--

Status: Patch Available  (was: Open)

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
 Attachments: HDFS-3012.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3011) move webapps/ into the JAR


[ 
https://issues.apache.org/jira/browse/HDFS-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215969#comment-13215969
 ] 

Todd Lipcon commented on HDFS-3011:
---

While you're at it, another thing I've noticed is that, when I run unit tests 
from eclipse, I have to manually do cp -a target/webapps target/classes/ or 
else the tests fail. I'm guessing this is related?

 move webapps/ into the JAR
 --

 Key: HDFS-3011
 URL: https://issues.apache.org/jira/browse/HDFS-3011
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: build, data-node, name-node
Affects Versions: 0.24.0, 0.23.2
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
 Fix For: 0.23.2


 Currently the webapps dir is in the filesystem and added to the classpath.
 As effectively it is picked up from the classpath, we could move into the 
 component JAR itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215984#comment-13215984
 ] 

Hudson commented on HDFS-3008:
--

Integrated in Hadoop-Mapreduce-trunk-Commit #1784 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1784/])
HDFS-3008. Negative caching of local addrs doesn't work. Contributed by Eli 
Collins (Revision 1293419)

 Result = ABORTED
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293419
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java


 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3008) Negative caching of local addrs doesn't work

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Updated) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215983#comment-13215983
 ] 

Hudson commented on HDFS-3008:
--

Integrated in Hadoop-Mapreduce-0.23-Commit #590 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/590/])
HDFS-3008. svn merge -c 1293419 from trunk (Revision 1293421)

 Result = ABORTED
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293421
Files : 
* /hadoop/common/branches/branch-0.23
* /hadoop/common/branches/branch-0.23/hadoop-common-project
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/bin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
* /hadoop/common/branches/branch-0.23/hadoop-project
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site


 Negative caching of local addrs doesn't work
 

 Key: HDFS-3008
 URL: https://issues.apache.org/jira/browse/HDFS-3008
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: hdfs client
Affects Versions: 0.23.1, 1.1.0
Reporter: Eli Collins
Assignee: Eli Collins
 Fix For: 1.1.0, 0.23.2

 Attachments: hdfs-3008.txt, hdfs-3008.txt


 HDFS-2653 added negative caching of local addrs, however it still goes 
 through the fall through path every time if the address is non-local. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:

[jira] [Updated] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


 [ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE updated HDFS-3006:
-

Attachment: h3006_20120224c.patch

h3006_20120224c.patch:
- changes the content type in more places;
- also changes @Produces annotation;
- adds a unit test;
- reverts the change in HttpFs since the simple fix won't work and I am not 
able to spend much time on it.

 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3012) Exception while renewing delegation token

2012-02-24 Thread Robert Joseph Evans (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216046#comment-13216046
]

Hadoop QA commented on HDFS-3012:
-

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515974/HDFS-3012.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1907//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1907//console

This message is automatically generated.

Exception while renewing delegation token
-

Key: HDFS-3012
URL: https://issues.apache.org/jira/browse/HDFS-3012
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
Attachments: HDFS-3012.txt

[jira] [Commented] (HDFS-3012) Exception while renewing delegation token


[ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216068#comment-13216068
 ] 

Robert Joseph Evans commented on HDFS-3012:
---

Just fair warning that I think the patch is kind of ugly.  I don't like calling 
an empty method to fix the issue, but I really don't want to change how these 
resources are loaded, because it would probably result in a much larger patch.

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
 Attachments: HDFS-3012.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3012) Exception while renewing delegation token

2012-02-24 Thread Robert Joseph Evans (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216088#comment-13216088
 ] 

Robert Joseph Evans commented on HDFS-3012:
---

@Mahadev,

The Renewer is already static.  The issue is that the Renewer code has no 
direct interaction with HdfsConfiguration.  Just an include is not enough to 
ensure that java will load the class and run the static block.  In the past, 
and in most other locations the HDFS Client would create an instance of 
HDFSConfiguration to access any configuration values.  This would force it to 
load, but because the RM does not interact with HDFS at all, except to renew 
and cancel tokens that interaction with HdfsConfiguration must be on the 
execution path of the Renewer.

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
 Attachments: HDFS-3012.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3012) Exception while renewing delegation token

2012-02-24 Thread Robert Joseph Evans (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216091#comment-13216091
 ] 

Robert Joseph Evans commented on HDFS-3012:
---

Sorry I misread your comment.  The Renewer has to be static or it cannot be 
loaded through the ServiceLoader. 

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
 Attachments: HDFS-3012.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type

2012-02-24 Thread Jitendra Nath Pandey (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216090#comment-13216090
]

Hadoop QA commented on HDFS-3006:
-

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515975/h3006_20120224c.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1908//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1908//console

This message is automatically generated.

Webhdfs SETOWNER call returns incorrect content-type
--

Key: HDFS-3006
URL: https://issues.apache.org/jira/browse/HDFS-3006
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
Attachments: h3006_20120223.patch, h3006_20120224.patch,
h3006_20120224b.patch, h3006_20120224c.patch

The SETOWNER call returns an empty body. But the header has Content-Type:
application/json, which is a contradiction (empty string is not valid json).
This appears to happen for SETTIMES and SETPERMISSION as well.

[jira] [Commented] (HDFS-3012) Exception while renewing delegation token


[ 
https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216096#comment-13216096
 ] 

Jitendra Nath Pandey commented on HDFS-3012:


HdfsConfiguration#init is already there which is noop.
I will recommend just adding a static initializer to Renewer that calls 
HdfsConfiguration#init.

 Exception while renewing delegation token
 -

 Key: HDFS-3012
 URL: https://issues.apache.org/jira/browse/HDFS-3012
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Assignee: Robert Joseph Evans
Priority: Critical
 Attachments: HDFS-3012.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Suresh Srinivas (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216101#comment-13216101
 ] 

Suresh Srinivas commented on HDFS-2978:
---

Minor comment -  In NameNodeMXBean.java can you please elaborate on what name 
dirs mean  in the javadoc.
+1 for the patch.


 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-24 Thread Jitendra Nath Pandey (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216126#comment-13216126
]

Jitendra Nath Pandey commented on HDFS-2904:

bq. This seems similar to your JIRA HDFS-2839, right? I don't disagree with you
that it would be nice to generalize the concept of logical URIs further, but as
it stands today, it is not general - this patch just works with the existing
design, rather than seeking to change it.

I am concerned that if someone wants to downgrade their cluster from ha to
non-ha and remove proxy provider configuration, they will have to also change
the fs.defaultFS of the cluster from logical uri to actual hostname based uri.
That essentially means it becomes a different cluster because file paths for
this cluster will stop working.

bq. Not sure I follow what you mean here. Before the
ConfiguredFailoverProxyProvider creates a new proxy, it calls this method to
clone the logical-service DT to the physical-address DT it's about to connect
to. So this works with any number of standby nodes.

I see, that makes sense. I understood it differently.

bq. Currently the only thing that can map to actual service addresses is the
proxy provider.
Exactly, IMO it shouldn't be the proxy provider, rather a different entity that
proxy provider uses.

I agree these issues should be discussed in a different jira.
+1 for the patch as it makes delegation tokens work for now.

HA: Client support for getting delegation tokens to an HA cluster
-

[jira] [Updated] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE updated HDFS-3006:
-

Attachment: h3006_20120224c_branch1.patch

h3006_20120224c_branch1.patch: for branch-1.

 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-24 Thread Colin Patrick McCabe (Resolved) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216134#comment-13216134
 ] 

Todd Lipcon commented on HDFS-2904:
---

Cool, I agree with your points above. Let's continue the discussion on 
HDFS-2839. Thanks for the +1, I'll commit this momentarily.

 HA: Client support for getting delegation tokens to an HA cluster
 -

 Key: HDFS-2904
 URL: https://issues.apache.org/jira/browse/HDFS-2904
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, hdfs client, name-node, security
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh


 Currently we have server-side support for delegation tokens in HA, and some 
 tests to verify it, but the client throws NPEs when trying to fetch a DT. 
 This is because the cluster doesn't have a single hostname, but instead a 
 logical nameservice name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-2966) TestNameNodeMetrics tests can fail under load

2012-02-24 Thread Suresh Srinivas (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Srinivas updated HDFS-2966:
--

Attachment: HDFS-2966.patch

Updated patch post HDFS-3002 changes.

 TestNameNodeMetrics tests can fail under load
 -

 Key: HDFS-2966
 URL: https://issues.apache.org/jira/browse/HDFS-2966
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: test
Affects Versions: 0.24.0
 Environment: OS/X running intellij IDEA, firefox, winxp in a 
 virtualbox.
Reporter: Steve Loughran
Assignee: Steve Loughran
Priority: Minor
 Fix For: 0.24.0, 0.23.2

 Attachments: HDFS-2966.patch, HDFS-2966.patch, HDFS-2966.patch


 I've managed to recreate HDFS-540 and HDFS-2434 by the simple technique of 
 running the HDFS tests on a desktop with out enough memory for all the 
 programs trying to run. Things got swapped out and the tests failed as the DN 
 heartbeats didn't come in on time.
 the tests both rely on {{waitForDeletion()}} to block the tests until the 
 delete operation has completed, but all it does is sleep for the same number 
 of seconds as there are datanodes. This is too brittle -it may work on a 
 lightly-loaded system, but not on a system under heavy load where it is 
 taking longer to replicate than expect.
 Immediate fix: double, triple, the sleep time?
 Better fix: have the thread block until all the DN heartbeats have finished.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type

2012-02-24 Thread Suresh Srinivas (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216142#comment-13216142
 ] 

Suresh Srinivas commented on HDFS-3006:
---

Minor comment: One line in DatanodeWebHdfsMethods.java and a couple in 
NamenodeWebHdfsMethods.java  80 chars.

+1 for the patch.


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster

2012-02-24 Thread Todd Lipcon (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon resolved HDFS-2904.
---

   Resolution: Fixed
Fix Version/s: HA branch (HDFS-1623)
 Hadoop Flags: Reviewed

Committed to branch, thanks for all of you who reviewed.

 HA: Client support for getting delegation tokens to an HA cluster
 -

 Key: HDFS-2904
 URL: https://issues.apache.org/jira/browse/HDFS-2904
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, hdfs client, name-node, security
Affects Versions: HA branch (HDFS-1623)
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Critical
 Fix For: HA branch (HDFS-1623)

 Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh


 Currently we have server-side support for delegation tokens in HA, and some 
 tests to verify it, but the client throws NPEs when trying to fetch a DT. 
 This is because the cluster doesn't have a single hostname, but instead a 
 logical nameservice name.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing

2012-02-24 Thread Colin Patrick McCabe (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Patrick McCabe updated HDFS-2872:
---

Attachment: HDFS-2872-branch-1.patch

Yeah, you're right.  The bug is against branch-1, so the patch should be too 
(in this case.)

I re-did this against branch-1

 Add sanity checks during edits loading that generation stamps are 
 non-decreasing
 

 Key: HDFS-2872
 URL: https://issues.apache.org/jira/browse/HDFS-2872
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 1.0.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2872-branch-1.patch, HDFS-2872.patch


 In 0.23 and later versions, we have a txid per edit, and the loading process 
 verifies that there are no gaps. Lacking this in 1.0, we can use generation 
 stamps as a proxy - the OP_SET_GENERATION_STAMP opcode should never result in 
 a decreased genstamp. If it does, that would indicate that the edits are 
 corrupt, or older edits are being applied to a newer checkpoint, for example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-2990) Move OfflineImageViewer, OfflineEditsViewer into the same package as the NameNode


 [ 
https://issues.apache.org/jira/browse/HDFS-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Patrick McCabe resolved HDFS-2990.


Resolution: Won't Fix

It looks like we'll keep oev and oiv in separate namespaces, and use interface 
annotations where needed.

 Move OfflineImageViewer, OfflineEditsViewer into the same package as the 
 NameNode
 -

 Key: HDFS-2990
 URL: https://issues.apache.org/jira/browse/HDFS-2990
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2990.patch


 Since OfflineImageviewer and OfflineEditsVeiwer are in different namespaces 
 than the NameNode, we can't ruuse a lot of the code from the NameNode without 
 making things public that we probably don't want to make public.  Let's move 
 them into the NameNode namespace to avoid this problem.  These tools will 
 always be tightly tied to the NameNode anyway (they are parsing the same 
 on-disk structures, after all), so that is where they belong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2990) Move OfflineImageViewer, OfflineEditsViewer into the same package as the NameNode

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216158#comment-13216158
 ] 

Tsz Wo (Nicholas), SZE commented on HDFS-2990:
--

Please minimize the changes to public and try to separate the interfaces and 
implementations.  I believe you could clean up the code as well.  ;)

 Move OfflineImageViewer, OfflineEditsViewer into the same package as the 
 NameNode
 -

 Key: HDFS-2990
 URL: https://issues.apache.org/jira/browse/HDFS-2990
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2990.patch


 Since OfflineImageviewer and OfflineEditsVeiwer are in different namespaces 
 than the NameNode, we can't ruuse a lot of the code from the NameNode without 
 making things public that we probably don't want to make public.  Let's move 
 them into the NameNode namespace to avoid this problem.  These tools will 
 always be tightly tied to the NameNode anyway (they are parsing the same 
 on-disk structures, after all), so that is where they belong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216162#comment-13216162
 ] 

Hudson commented on HDFS-3006:
--

Integrated in Hadoop-Hdfs-trunk-Commit #1851 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1851/])
HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type 
to application/octet-stream instead of application/json. (Revision 1293487)

 Result = SUCCESS
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293487
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216161#comment-13216161
 ] 

Hudson commented on HDFS-3002:
--

Integrated in Hadoop-Hdfs-trunk-Commit #1851 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1851/])
HDFS-3002. TestNameNodeMetrics need not wait for metrics update. 
Contributed by Suresh Srinivas. (Revision 1293482)

 Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293482
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java


 TestNameNodeMetrics need not wait for metrics update with new metrics 
 framework
 ---

 Key: HDFS-3002
 URL: https://issues.apache.org/jira/browse/HDFS-3002
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: test
Affects Versions: 0.23.0, 0.24.0
Reporter: Suresh Srinivas
Assignee: Suresh Srinivas
Priority: Trivial
 Attachments: HDFS-3002.patch


 With older metrics framework, the namenode metrics was updated by replication 
 thread. This required test having to wait for replication interval. This is 
 no longer necessary with metrics2 framework. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Hitesh Shah (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216160#comment-13216160
 ] 

Hitesh Shah commented on HDFS-2978:
---

@Aaron, Minor question on the use of emitting the data as a json-encoded string 
- any reason why this is being done for all the information exposed through the 
mbeans instead of using defined structures which are recognized and handled by 
the JMX interface in any case? 

Also, ignoring the above comment, would it be a good idea to have this 
available in 1.x too? 

 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216163#comment-13216163
 ] 

Tsz Wo (Nicholas), SZE commented on HDFS-3006:
--

Hi Suresh,

Thanks for reviewing it.

I did not change the long line since it is minor style issue.  I think the 80 
character line limit is an out-dated style since the screen resolution nowadays 
is usually large and wide-screen are very common.  I will start a discussion 
about it.

 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HDFS-3013) NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure

2012-02-24 Thread Mingjie Lai (Created) (JIRA)

NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure
-

 Key: HDFS-3013
 URL: https://issues.apache.org/jira/browse/HDFS-3013
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: ha, name-node
Affects Versions: 0.23.0, HA branch (HDFS-1623)
Reporter: Mingjie Lai


One problem I observed at HA branch is that namenode -format doesn't pick up 
the right configured dfs.namenode.name.dir.NameServiceId. In this case, 
``namenode -format'' will format the default directory instead of the 
configured one, while namenode will use the correct one. 

The root cause is that NameNode.initializeGenericKeys() is only invoked at the 
NameNode constructor. So it doesn't get called by static methods like format 
and finalize. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE updated HDFS-3006:
-

   Resolution: Fixed
Fix Version/s: 1.0.2
   0.23.2
   1.1.0
   0.24.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

I have committed this to trunk, 0.23, branch-1 and branch-1.0.

 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2

 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework


[ 
https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216168#comment-13216168
 ] 

Hudson commented on HDFS-3002:
--

Integrated in Hadoop-Common-trunk-Commit #1777 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1777/])
HDFS-3002. TestNameNodeMetrics need not wait for metrics update. 
Contributed by Suresh Srinivas. (Revision 1293482)

 Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293482
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java


 TestNameNodeMetrics need not wait for metrics update with new metrics 
 framework
 ---

 Key: HDFS-3002
 URL: https://issues.apache.org/jira/browse/HDFS-3002
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: test
Affects Versions: 0.23.0, 0.24.0
Reporter: Suresh Srinivas
Assignee: Suresh Srinivas
Priority: Trivial
 Attachments: HDFS-3002.patch


 With older metrics framework, the namenode metrics was updated by replication 
 thread. This required test having to wait for replication interval. This is 
 no longer necessary with metrics2 framework. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216169#comment-13216169
 ] 

Hudson commented on HDFS-3006:
--

Integrated in Hadoop-Common-trunk-Commit #1777 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1777/])
HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type 
to application/octet-stream instead of application/json. (Revision 1293487)

 Result = SUCCESS
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293487
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2

 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing

2012-02-24 Thread Colin Patrick McCabe (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Patrick McCabe updated HDFS-2872:
---

Attachment: HDFS-2872-branch-1.patch

the file I meant to append...

 Add sanity checks during edits loading that generation stamps are 
 non-decreasing
 

 Key: HDFS-2872
 URL: https://issues.apache.org/jira/browse/HDFS-2872
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 1.0.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2872-branch-1.patch, HDFS-2872.patch


 In 0.23 and later versions, we have a txid per edit, and the loading process 
 verifies that there are no gaps. Lacking this in 1.0, we can use generation 
 stamps as a proxy - the OP_SET_GENERATION_STAMP opcode should never result in 
 a decreased genstamp. If it does, that would indicate that the edits are 
 corrupt, or older edits are being applied to a newer checkpoint, for example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing

2012-02-24 Thread Colin Patrick McCabe (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Patrick McCabe updated HDFS-2872:
---

Attachment: (was: HDFS-2872-branch-1.patch)

 Add sanity checks during edits loading that generation stamps are 
 non-decreasing
 

 Key: HDFS-2872
 URL: https://issues.apache.org/jira/browse/HDFS-2872
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 1.0.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2872-branch-1.patch, HDFS-2872.patch


 In 0.23 and later versions, we have a txid per edit, and the loading process 
 verifies that there are no gaps. Lacking this in 1.0, we can use generation 
 stamps as a proxy - the OP_SET_GENERATION_STAMP opcode should never result in 
 a decreased genstamp. If it does, that would indicate that the edits are 
 corrupt, or older edits are being applied to a newer checkpoint, for example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework


[ 
https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216177#comment-13216177
 ] 

Hudson commented on HDFS-3002:
--

Integrated in Hadoop-Mapreduce-trunk-Commit #1787 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1787/])
HDFS-3002. TestNameNodeMetrics need not wait for metrics update. 
Contributed by Suresh Srinivas. (Revision 1293482)

 Result = ABORTED
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293482
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java


 TestNameNodeMetrics need not wait for metrics update with new metrics 
 framework
 ---

 Key: HDFS-3002
 URL: https://issues.apache.org/jira/browse/HDFS-3002
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: test
Affects Versions: 0.23.0, 0.24.0
Reporter: Suresh Srinivas
Assignee: Suresh Srinivas
Priority: Trivial
 Attachments: HDFS-3002.patch


 With older metrics framework, the namenode metrics was updated by replication 
 thread. This required test having to wait for replication interval. This is 
 no longer necessary with metrics2 framework. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3013) NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure


[ 
https://issues.apache.org/jira/browse/HDFS-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216185#comment-13216185
 ] 

Todd Lipcon commented on HDFS-3013:
---

Nice find, Mingjie. Want to take a crack at a patch? If not, I'll take this one.

 NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure
 -

 Key: HDFS-3013
 URL: https://issues.apache.org/jira/browse/HDFS-3013
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: ha, name-node
Affects Versions: HA branch (HDFS-1623), 0.23.0
Reporter: Mingjie Lai

 One problem I observed at HA branch is that namenode -format doesn't pick up 
 the right configured dfs.namenode.name.dir.NameServiceId. In this case, 
 ``namenode -format'' will format the default directory instead of the 
 configured one, while namenode will use the correct one. 
 The root cause is that NameNode.initializeGenericKeys() is only invoked at 
 the NameNode constructor. So it doesn't get called by static methods like 
 format and finalize. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216184#comment-13216184
 ] 

Hudson commented on HDFS-3006:
--

Integrated in Hadoop-Hdfs-0.23-Commit #579 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/579/])
svn merge -c 1293487 from trunk for HDFS-3006. (Revision 1293489)

 Result = SUCCESS
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293489
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2

 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216190#comment-13216190
 ] 

Todd Lipcon commented on HDFS-2978:
---

I agree with Hitesh - JMX is supposed to support arbitrarily nested map 
structures of java primitives, isn't it? I don't know it well, but I think we 
do this in other places.

 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing


[ 
https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216187#comment-13216187
 ] 

Tsz Wo (Nicholas), SZE commented on HDFS-2872:
--

This is a good check.  Should it be more strict, i.e. check whether cur == prv 
+ 1?

 Add sanity checks during edits loading that generation stamps are 
 non-decreasing
 

 Key: HDFS-2872
 URL: https://issues.apache.org/jira/browse/HDFS-2872
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 1.0.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2872-branch-1.patch, HDFS-2872.patch


 In 0.23 and later versions, we have a txid per edit, and the loading process 
 verifies that there are no gaps. Lacking this in 1.0, we can use generation 
 stamps as a proxy - the OP_SET_GENERATION_STAMP opcode should never result in 
 a decreased genstamp. If it does, that would indicate that the edits are 
 corrupt, or older edits are being applied to a newer checkpoint, for example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type

2012-02-24 Thread Colin Patrick McCabe (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216193#comment-13216193
 ] 

Hudson commented on HDFS-3006:
--

Integrated in Hadoop-Common-0.23-Commit #591 (See 
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/591/])
svn merge -c 1293487 from trunk for HDFS-3006. (Revision 1293489)

 Result = SUCCESS
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293489
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2

 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing


[ 
https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216203#comment-13216203
 ] 

Colin Patrick McCabe commented on HDFS-2872:


generation stamps don't increase on every EditLog operation.  

However, normally they should either increase by 0 or 1 on each new operation.  
In theory, we could forbid the generation stamp from ever increasing by more 
than one, but that would probably make recovery of a corrupted EditLog file 
more difficult than it is now.

 Add sanity checks during edits loading that generation stamps are 
 non-decreasing
 

 Key: HDFS-2872
 URL: https://issues.apache.org/jira/browse/HDFS-2872
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: name-node
Affects Versions: 1.0.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
 Attachments: HDFS-2872-branch-1.patch, HDFS-2872.patch


 In 0.23 and later versions, we have a txid per edit, and the loading process 
 verifies that there are no gaps. Lacking this in 1.0, we can use generation 
 stamps as a proxy - the OP_SET_GENERATION_STAMP opcode should never result in 
 a decreased genstamp. If it does, that would indicate that the edits are 
 corrupt, or older edits are being applied to a newer checkpoint, for example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2872) Add sanity checks during edits loading that generation stamps are non-decreasing

2012-02-24 Thread Tsz Wo (Nicholas), SZE (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HDFS-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216213#comment-13216213
]

Tsz Wo (Nicholas), SZE commented on HDFS-2872:
--

generation stamps don't increase on every EditLog operation.

Sorry that I am not being clear. I mean for each OP_SET_GENSTAMP, check
whether the cur == prv + 1, where cur is the value in the current
OP_SET_GENSTAMP and prv is the value in the previous OP_SET_GENSTAMP. For
other OPs, no check is needed.

Add sanity checks during edits loading that generation stamps are
non-decreasing

Key: HDFS-2872
URL: https://issues.apache.org/jira/browse/HDFS-2872
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 1.0.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
Attachments: HDFS-2872-branch-1.patch, HDFS-2872.patch

In 0.23 and later versions, we have a txid per edit, and the loading process
verifies that there are no gaps. Lacking this in 1.0, we can use generation
stamps as a proxy - the OP_SET_GENERATION_STAMP opcode should never result in
a decreased genstamp. If it does, that would indicate that the edits are
corrupt, or older edits are being applied to a newer checkpoint, for example.

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216218#comment-13216218
 ] 

Hudson commented on HDFS-3006:
--

Integrated in Hadoop-Mapreduce-trunk-Commit #1788 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1788/])
HDFS-3006. In WebHDFS, when the return body is empty, set the Content-Type 
to application/octet-stream instead of application/json. (Revision 1293487)

 Result = ABORTED
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293487
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* 
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2

 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-3006) Webhdfs SETOWNER call returns incorrect content-type


[ 
https://issues.apache.org/jira/browse/HDFS-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216219#comment-13216219
 ] 

Hudson commented on HDFS-3006:
--

Integrated in Hadoop-Mapreduce-0.23-Commit #594 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/594/])
svn merge -c 1293487 from trunk for HDFS-3006. (Revision 1293489)

 Result = ABORTED
szetszwo : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1293489
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
* 
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java


 Webhdfs SETOWNER call returns incorrect content-type
 --

 Key: HDFS-3006
 URL: https://issues.apache.org/jira/browse/HDFS-3006
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: name-node
Affects Versions: 0.23.2
Reporter: bc Wong
Assignee: Tsz Wo (Nicholas), SZE
 Fix For: 0.24.0, 1.1.0, 0.23.2, 1.0.2

 Attachments: h3006_20120223.patch, h3006_20120224.patch, 
 h3006_20120224b.patch, h3006_20120224c.patch, h3006_20120224c_branch1.patch


 The SETOWNER call returns an empty body. But the header has Content-Type: 
 application/json, which is a contradiction (empty string is not valid json). 
 This appears to happen for SETTIMES and SETPERMISSION as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Suresh Srinivas (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216221#comment-13216221
 ] 

Suresh Srinivas commented on HDFS-2978:
---

Exposing Strings and primitives is straight forward in JMX. But not so for more 
complicated data structures to non java programs. These interfaces were 
intended as a replacement of screen scraping of Namenode web UI, to be used in 
scripts etc. using local JMX connection. For such usages, the information is 
exposed java independent way.

 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2966) TestNameNodeMetrics tests can fail under load

[
https://issues.apache.org/jira/browse/HDFS-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216223#comment-13216223
]

Hadoop QA commented on HDFS-2966:
-

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516000/HDFS-2966.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/1910//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1910//console

This message is automatically generated.

TestNameNodeMetrics tests can fail under load
-

Key: HDFS-2966
URL: https://issues.apache.org/jira/browse/HDFS-2966
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 0.24.0
Environment: OS/X running intellij IDEA, firefox, winxp in a
virtualbox.
Reporter: Steve Loughran
Assignee: Steve Loughran
Priority: Minor
Fix For: 0.24.0, 0.23.2

Attachments: HDFS-2966.patch, HDFS-2966.patch, HDFS-2966.patch

I've managed to recreate HDFS-540 and HDFS-2434 by the simple technique of
running the HDFS tests on a desktop with out enough memory for all the
programs trying to run. Things got swapped out and the tests failed as the DN
heartbeats didn't come in on time.
the tests both rely on {{waitForDeletion()}} to block the tests until the
delete operation has completed, but all it does is sleep for the same number
of seconds as there are datanodes. This is too brittle -it may work on a
lightly-loaded system, but not on a system under heavy load where it is
taking longer to replicate than expect.
Immediate fix: double, triple, the sleep time?
Better fix: have the thread block until all the DN heartbeats have finished.

[jira] [Commented] (HDFS-2966) TestNameNodeMetrics tests can fail under load

2012-02-24 Thread Aaron T. Myers (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HDFS-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216230#comment-13216230
]

Aaron T. Myers commented on HDFS-2966:
--

bq. Rather than rename the method, I made the metric scope a parameter, so it
can block for other metrics too.

I still find the name a tad misleading, since the method is still a little
DN-specific. e.g. it retries based on the number of DNs, and sleeps a multiple
of DFS_REPLICATION_INTERVAL. But, I don't feel super strongly about this point.
Take it or leave it.

+1, the patch looks good to me, assuming you don't want to address the above
feedback.

TestNameNodeMetrics tests can fail under load
-

Attachments: HDFS-2966.patch, HDFS-2966.patch, HDFS-2966.patch

[jira] [Commented] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework

2012-02-24 Thread Aaron T. Myers (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216239#comment-13216239
 ] 

Aaron T. Myers commented on HDFS-3002:
--

Hey Suresh, I see that you committed this to trunk. I was about to resolve it 
for you, but then I noticed that you had the affects version set to 0.23.0, 
but with no target version set.

Do you intend to commit this to 0.23 as well? Seems to me like we should.

 TestNameNodeMetrics need not wait for metrics update with new metrics 
 framework
 ---

 Key: HDFS-3002
 URL: https://issues.apache.org/jira/browse/HDFS-3002
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: test
Affects Versions: 0.23.0, 0.24.0
Reporter: Suresh Srinivas
Assignee: Suresh Srinivas
Priority: Trivial
 Attachments: HDFS-3002.patch


 With older metrics framework, the namenode metrics was updated by replication 
 thread. This required test having to wait for replication interval. This is 
 no longer necessary with metrics2 framework. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HDFS-2978) The NameNode should expose name dir statuses via JMX

2012-02-24 Thread Aaron T. Myers (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216242#comment-13216242
 ] 

Aaron T. Myers commented on HDFS-2978:
--

bq. Also, ignoring the above comment, would it be a good idea to have this 
available in 1.x too?

Sure, good thinking. I'd be happy to provide such a patch.

Todd/Hitesh - does Suresh's reply re: why JSON for JMX data address your 
concerns?

 The NameNode should expose name dir statuses via JMX
 

 Key: HDFS-2978
 URL: https://issues.apache.org/jira/browse/HDFS-2978
 Project: Hadoop HDFS
  Issue Type: New Feature
  Components: name-node
Affects Versions: 0.23.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
 Attachments: HDFS-2978.patch


 We currently display this info on the NN web UI, so users who wish to monitor 
 this must either do it manually or parse HTML. We should publish this 
 information via JMX.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3013) HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure


 [ 
https://issues.apache.org/jira/browse/HDFS-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron T. Myers updated HDFS-3013:
-

Issue Type: Sub-task  (was: Bug)
Parent: HDFS-1623

 HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId 
 configure
 -

 Key: HDFS-3013
 URL: https://issues.apache.org/jira/browse/HDFS-3013
 Project: Hadoop HDFS
  Issue Type: Sub-task
  Components: ha, name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Mingjie Lai

 One problem I observed at HA branch is that namenode -format doesn't pick up 
 the right configured dfs.namenode.name.dir.NameServiceId. In this case, 
 ``namenode -format'' will format the default directory instead of the 
 configured one, while namenode will use the correct one. 
 The root cause is that NameNode.initializeGenericKeys() is only invoked at 
 the NameNode constructor. So it doesn't get called by static methods like 
 format and finalize. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HDFS-3013) HA: NameNode format doesn't pick up dfs.namenode.name.dir.NameServiceId configure