[jira] [Commented] (MAPREDUCE-3664) HDFS Federation Documentation has incorrect configuration example

2012-01-12 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185155#comment-13185155
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-3664:
-

Committed. Thanks to Brandon Li!

 HDFS Federation Documentation has incorrect configuration example
 -

 Key: MAPREDUCE-3664
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3664
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.23.0, 0.24.0
Reporter: praveen sripati
Priority: Minor
 Attachments: HDFS-2778.txt, HDFS-2778.txt


 HDFS Federation documentation example (1) has the following
 property
 namedfs.namenode.rpc-address.ns1/name
 valuehdfs://nn-host1:rpc-port/value
 /property
 dfs.namenode.rpc-address.* should be set to hostname:port, hdfs:// should not 
 be there.
 (1) - 
 http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/Federation.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-3236) Distcp with hdfs:// passed with error in JT log while copying from .20.204 to .20.205 ( with useIp=false)

2011-10-20 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131973#comment-13131973
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-3236:
-

 It is expected that JT can't renew tokens from a different cluster over 
hdfs:// (i.e. using rpc) if rpc ports are closed for outside cluster access. It 
is usually expected that http ports are open so it should work over hftp. It 
seems to me its not a bug.

Daryn,
  Lets address the issue you pointed out in a separate jira.

 Distcp with hdfs:// passed with error in JT log while copying from .20.204  
 to .20.205 ( with useIp=false)
 --

 Key: MAPREDUCE-3236
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3236
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: jobtracker, security
Affects Versions: 0.20.205.0
Reporter: Rajit Saha
Assignee: Daryn Sharp
 Attachments: HDFS-2447-1.patch, HDFS-2447.patch


 I tried to copy file from .20.204 to .20.205 by distcp over hdfs:// while 
 using hadoop.security.token.service.use_ip=false in core-site.xml. The copy 
 was successful but found error  
 org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: exception 
 in .20.205 JT.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-2764) Fix renewal of dfs delegation tokens

2011-10-14 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127798#comment-13127798
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-2764:
-

Created jira MAPREDUCE-3192 to fix the javadoc warning.

 Fix renewal of dfs delegation tokens
 

 Key: MAPREDUCE-2764
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Daryn Sharp
Assignee: Owen O'Malley
 Fix For: 0.20.205.0, 0.23.0, 0.24.0

 Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764-3.patch, 
 MAPREDUCE-2764-4.patch, MAPREDUCE-2764-trunk.patch, 
 MAPREDUCE-2764-trunk.patch, MAPREDUCE-2764-trunk.patch, MAPREDUCE-2764.patch, 
 delegation.patch, token-renew-trunk.patch, token-renew.patch, 
 token-renew.patch


 The JT may have issues renewing hftp tokens which disrupt long distcp jobs.  
 The problem is the JT's delegation token renewal code is built on brittle 
 assumptions.  The token's service field contains only the ip:port pair.  
 The renewal process assumes that the scheme must be hdfs.  If that fails due 
 to a {{VersionMismatchException}}, it tries https based on another assumption 
 that it must be hftp if it's not hdfs.  A number of other exceptions, most 
 commonly {{IOExceptions}}, can be generated which fouls up the renewal since 
 it won't fallback to https.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-2764) Fix renewal of dfs delegation tokens

2011-10-14 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128018#comment-13128018
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-2764:
-

Merged to 23 as well.

 Fix renewal of dfs delegation tokens
 

 Key: MAPREDUCE-2764
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Daryn Sharp
Assignee: Owen O'Malley
 Fix For: 0.20.205.0, 0.23.0, 0.24.0

 Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764-3.patch, 
 MAPREDUCE-2764-4.patch, MAPREDUCE-2764-trunk.patch, 
 MAPREDUCE-2764-trunk.patch, MAPREDUCE-2764-trunk.patch, MAPREDUCE-2764.patch, 
 delegation.patch, token-renew-trunk.patch, token-renew.patch, 
 token-renew.patch


 The JT may have issues renewing hftp tokens which disrupt long distcp jobs.  
 The problem is the JT's delegation token renewal code is built on brittle 
 assumptions.  The token's service field contains only the ip:port pair.  
 The renewal process assumes that the scheme must be hdfs.  If that fails due 
 to a {{VersionMismatchException}}, it tries https based on another assumption 
 that it must be hftp if it's not hdfs.  A number of other exceptions, most 
 commonly {{IOExceptions}}, can be generated which fouls up the renewal since 
 it won't fallback to https.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-2764) Fix renewal of dfs delegation tokens

2011-10-13 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13126421#comment-13126421
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-2764:
-

test patch results:

-1 overall.  

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 19 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version ) 
warnings.

-1 release audit.  The applied patch generated 1 release audit warnings 
(more than the trunk's current 0 warnings).

The release audit warning is related to Apache License header for 
hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml. It 
is unrelated to this patch.

 Fix renewal of dfs delegation tokens
 

 Key: MAPREDUCE-2764
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Daryn Sharp
Assignee: Owen O'Malley
 Fix For: 0.20.205.0, 0.23.0

 Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764-3.patch, 
 MAPREDUCE-2764-4.patch, MAPREDUCE-2764-trunk.patch, MAPREDUCE-2764.patch, 
 delegation.patch, token-renew-trunk.patch, token-renew.patch, 
 token-renew.patch


 The JT may have issues renewing hftp tokens which disrupt long distcp jobs.  
 The problem is the JT's delegation token renewal code is built on brittle 
 assumptions.  The token's service field contains only the ip:port pair.  
 The renewal process assumes that the scheme must be hdfs.  If that fails due 
 to a {{VersionMismatchException}}, it tries https based on another assumption 
 that it must be hftp if it's not hdfs.  A number of other exceptions, most 
 commonly {{IOExceptions}}, can be generated which fouls up the renewal since 
 it won't fallback to https.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-2764) Fix renewal of dfs delegation tokens

2011-10-13 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13127121#comment-13127121
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-2764:
-

I manually ran tests for common, hdfs, yarn. Following tests failed
  TestFailoverProxy,
  TestJMXJsonServlet
  TestDistributedFileSystem
  TestRestartDFS
  TestDfsOverAvroRpc
  TestDeleteBlockPool
  TestNMWebServer 

  They fail on trunk without this patch as well. 
  (TestDistributedFileSystem fails with java.io.IOException: Unknown protocol: 
org.apache.hadoop.hdfs.protocol.ClientProtocol)
 

 Fix renewal of dfs delegation tokens
 

 Key: MAPREDUCE-2764
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Daryn Sharp
Assignee: Owen O'Malley
 Fix For: 0.20.205.0, 0.23.0

 Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764-3.patch, 
 MAPREDUCE-2764-4.patch, MAPREDUCE-2764-trunk.patch, 
 MAPREDUCE-2764-trunk.patch, MAPREDUCE-2764.patch, delegation.patch, 
 token-renew-trunk.patch, token-renew.patch, token-renew.patch


 The JT may have issues renewing hftp tokens which disrupt long distcp jobs.  
 The problem is the JT's delegation token renewal code is built on brittle 
 assumptions.  The token's service field contains only the ip:port pair.  
 The renewal process assumes that the scheme must be hdfs.  If that fails due 
 to a {{VersionMismatchException}}, it tries https based on another assumption 
 that it must be hftp if it's not hdfs.  A number of other exceptions, most 
 commonly {{IOExceptions}}, can be generated which fouls up the renewal since 
 it won't fallback to https.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-3149) add a test to verify that buildDTAuthority works for cases with no authority.

2011-10-12 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125995#comment-13125995
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-3149:
-

+1 for the trunk patch.

 add a test to verify that buildDTAuthority works for cases with no authority.
 -

 Key: MAPREDUCE-3149
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3149
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 0.24.0
Reporter: John George
Assignee: John George
 Attachments: HADOOP-7602.patch


 Add a test to verify that buildDTAuthority works for cases with no Authority.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-2764) Fix renewal of dfs delegation tokens

2011-10-12 Thread Jitendra Nath Pandey (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13126263#comment-13126263
 ] 

Jitendra Nath Pandey commented on MAPREDUCE-2764:
-

I agree with Daryn regarding the use_ip change. Apart from that, the trunk 
patch looks good to me. +1

 Fix renewal of dfs delegation tokens
 

 Key: MAPREDUCE-2764
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2764
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Daryn Sharp
Assignee: Owen O'Malley
 Fix For: 0.20.205.0, 0.23.0

 Attachments: MAPREDUCE-2764-2.patch, MAPREDUCE-2764-3.patch, 
 MAPREDUCE-2764-4.patch, MAPREDUCE-2764.patch, delegation.patch, 
 token-renew-trunk.patch, token-renew.patch, token-renew.patch


 The JT may have issues renewing hftp tokens which disrupt long distcp jobs.  
 The problem is the JT's delegation token renewal code is built on brittle 
 assumptions.  The token's service field contains only the ip:port pair.  
 The renewal process assumes that the scheme must be hdfs.  If that fails due 
 to a {{VersionMismatchException}}, it tries https based on another assumption 
 that it must be hftp if it's not hdfs.  A number of other exceptions, most 
 commonly {{IOExceptions}}, can be generated which fouls up the renewal since 
 it won't fallback to https.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira