[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211020#comment-13211020 ] Mahadev konar commented on HDFS-2904: - bq. I was thinking it wouldn't, necessarily. Our current client side code has no notion of being able to list all of the potential NNs, only of being able to get a proxy to the active one. With ZK, you wouldn't necessarily have a list of the standbys. For example, it might be that when you submit the job, the SBN happened to be down for maintenance. Then when you bring it back up and do a failover to it, it wouldn't have been listed for DT renewal. I was thinking more along the lines of all the NN's would be listed as a persistent nodes on ZK (unless NN's unregister explicitly), so you would probably be able to get a list of NN's even when some are down, but again I am not an expert on what is going on in HA. I will let you folks decide on the right model. As far as MR is concerned I think it should be ok to deploy all NN configs on the RM as a stop gap solution. Longer term we should be able to get rid of that. > HA: Client support for getting delegation tokens to an HA cluster > - > > Key: HDFS-2904 > URL: https://issues.apache.org/jira/browse/HDFS-2904 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, hdfs client, name-node, security >Affects Versions: HA branch (HDFS-1623) >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh > > > Currently we have server-side support for delegation tokens in HA, and some > tests to verify it, but the client throws NPEs when trying to fetch a DT. > This is because the cluster doesn't have a single hostname, but instead a > logical nameservice name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211019#comment-13211019 ] Mahadev konar commented on HDFS-2904: - bq. I was thinking it wouldn't, necessarily. Our current client side code has no notion of being able to list all of the potential NNs, only of being able to get a proxy to the active one. bq. With ZK, you wouldn't necessarily have a list of the standbys. For example, it might be that when you submit the job, the SBN happened to be down for maintenance. Then when you bring it back up and do a failover to it, it wouldn't have been listed for DT renewal. > HA: Client support for getting delegation tokens to an HA cluster > - > > Key: HDFS-2904 > URL: https://issues.apache.org/jira/browse/HDFS-2904 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, hdfs client, name-node, security >Affects Versions: HA branch (HDFS-1623) >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh > > > Currently we have server-side support for delegation tokens in HA, and some > tests to verify it, but the client throws NPEs when trying to fetch a DT. > This is because the cluster doesn't have a single hostname, but instead a > logical nameservice name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209851#comment-13209851 ] Mahadev konar commented on HDFS-2904: - @Sanjay, I dont think you can obtain DT's from both NN's since HDFS has to log the txn's for DT's. > HA: Client support for getting delegation tokens to an HA cluster > - > > Key: HDFS-2904 > URL: https://issues.apache.org/jira/browse/HDFS-2904 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, hdfs client, name-node, security >Affects Versions: HA branch (HDFS-1623) >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh > > > Currently we have server-side support for delegation tokens in HA, and some > tests to verify it, but the client throws NPEs when trying to fetch a DT. > This is because the cluster doesn't have a single hostname, but instead a > logical nameservice name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2904) HA: Client support for getting delegation tokens to an HA cluster
[ https://issues.apache.org/jira/browse/HDFS-2904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209849#comment-13209849 ] Mahadev konar commented on HDFS-2904: - I didnt look at the patch very closely but having the RM deploy all the HA NN configs in the interim seems ok. Though having a comma separated list of hostnames in the DT would have eliminated the need. Todd, about the client not knowing all the hosts, it should be able to get all the hosts even when you use ZK, no? > HA: Client support for getting delegation tokens to an HA cluster > - > > Key: HDFS-2904 > URL: https://issues.apache.org/jira/browse/HDFS-2904 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, hdfs client, name-node, security >Affects Versions: HA branch (HDFS-1623) >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Attachments: hdfs-2904.txt, hdfs-2904.txt, hdfs-2904.txt, test-dt.sh > > > Currently we have server-side support for delegation tokens in HA, and some > tests to verify it, but the client throws NPEs when trying to fetch a DT. > This is because the cluster doesn't have a single hostname, but instead a > logical nameservice name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-442) dfsthroughput in test.jar throws NPE
[
https://issues.apache.org/jira/browse/HDFS-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191643#comment-13191643
]
Mahadev konar commented on HDFS-442:
+1 for the patch.
> dfsthroughput in test.jar throws NPE
>
>
> Key: HDFS-442
> URL: https://issues.apache.org/jira/browse/HDFS-442
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 0.20.1
>Reporter: Ramya Sunil
>Assignee: Harsh J
>Priority: Minor
> Attachments: HDFS-442.patch
>
>
> On running hadoop jar hadoop-test.jar dfsthroughput OR hadoop
> org.apache.hadoop.hdfs.BenchmarkThroughput, we get NullPointerException.
> Below is the stacktrace:
> {noformat}
> Exception in thread "main" java.lang.NullPointerException
> at java.util.Hashtable.put(Hashtable.java:394)
> at java.util.Properties.setProperty(Properties.java:143)
> at java.lang.System.setProperty(System.java:731)
> at
> org.apache.hadoop.hdfs.BenchmarkThroughput.run(BenchmarkThroughput.java:198)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hdfs.BenchmarkThroughput.main(BenchmarkThroughput.java:229)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2119) "hdfs version" should print out information similar to what "hadoop version" prints out
[ https://issues.apache.org/jira/browse/HDFS-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129909#comment-13129909 ] Mahadev konar commented on HDFS-2119: - @ravi, Can we not do this using shared version for all the 3 common/hdfs/mapred. I need to look into what the new tar assembly in hadoop-dist module is, but I think we could probably just calculate the version somewhere common and use it in hdfs/common/mapred. Will update after looking into the the poms. > "hdfs version" should print out information similar to what "hadoop version" > prints out > --- > > Key: HDFS-2119 > URL: https://issues.apache.org/jira/browse/HDFS-2119 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.23.0 >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Fix For: 0.23.0 > > Attachments: HDFS-2119.1.patch, HDFS-2119.2.patch, HDFS-2119.3.patch > > > Implement version in hdfs CLI -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2401) Running a set of methods in a Single Test Class
[ https://issues.apache.org/jira/browse/HDFS-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126398#comment-13126398 ] Mahadev konar commented on HDFS-2401: - +1 looks good to me. > Running a set of methods in a Single Test Class > --- > > Key: HDFS-2401 > URL: https://issues.apache.org/jira/browse/HDFS-2401 > Project: Hadoop HDFS > Issue Type: Improvement > Components: build >Affects Versions: 0.23.0 >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 0.23.0 > > Attachments: HDFS-2401.patch > > > Instead of running every test method in a class, limit to specific testing > methods as describe in the link below. > http://maven.apache.org/plugins/maven-surefire-plugin/examples/single-test.html > Upgrade to the latest version of maven-surefire-plugin that has this feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
