[jira] [Moved] (HADOOP-11719) [Fsshell]It's better to provide hdfs instead of hadoop in GenericOptionsParser
[ https://issues.apache.org/jira/browse/HADOOP-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J moved HDFS-3387 to HADOOP-11719: Component/s: (was: hdfs-client) scripts Affects Version/s: (was: 2.0.0-alpha) 2.0.0-alpha Issue Type: Improvement (was: Bug) Key: HADOOP-11719 (was: HDFS-3387) Project: Hadoop Common (was: Hadoop HDFS) > [Fsshell]It's better to provide hdfs instead of hadoop in GenericOptionsParser > -- > > Key: HADOOP-11719 > URL: https://issues.apache.org/jira/browse/HADOOP-11719 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 2.0.0-alpha >Reporter: Brahma Reddy Battula >Priority: Trivial > Labels: newbie > Attachments: HDFS-3387.patch, HDFS-3387_updated.patch > > > Scenario: > -- > Execute any fsshell command with invalid options > Like ./hdfs haadmin -transitionToActive... > Here it is logging as following.. > bin/hadoop command [genericOptions] [commandOptions]... > Expected: Here help message is misleading to user saying that bin/hadoop that > is not actually user ran > it's better to log bin/hdfs..Anyway hadoop is deprecated.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11719) [Fsshell] Remove bin/hadoop reference from GenericOptionsParser default help text
[ https://issues.apache.org/jira/browse/HADOOP-11719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-11719: - Summary: [Fsshell] Remove bin/hadoop reference from GenericOptionsParser default help text (was: [Fsshell]It's better to provide hdfs instead of hadoop in GenericOptionsParser) > [Fsshell] Remove bin/hadoop reference from GenericOptionsParser default help > text > - > > Key: HADOOP-11719 > URL: https://issues.apache.org/jira/browse/HADOOP-11719 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 2.0.0-alpha >Reporter: Brahma Reddy Battula >Priority: Trivial > Labels: newbie > Attachments: HDFS-3387.patch, HDFS-3387_updated.patch > > > Scenario: > -- > Execute any fsshell command with invalid options > Like ./hdfs haadmin -transitionToActive... > Here it is logging as following.. > bin/hadoop command [genericOptions] [commandOptions]... > Expected: Here help message is misleading to user saying that bin/hadoop that > is not actually user ran > it's better to log bin/hdfs..Anyway hadoop is deprecated.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11718) CHANGES.TXT in trunk is incorrect
[ https://issues.apache.org/jira/browse/HADOOP-11718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-11718: -- Description: As part of my auditing of JIRA fixversions, it's becoming clear that there are a few JIRAs listed as being only in trunk that were actually released as part of either 0.23 or 2.x. (was: There are a few JIRAs listed as being only in trunk that were actually released as part of 2.x.) > CHANGES.TXT in trunk is incorrect > - > > Key: HADOOP-11718 > URL: https://issues.apache.org/jira/browse/HADOOP-11718 > Project: Hadoop Common > Issue Type: Bug >Reporter: Allen Wittenauer > > As part of my auditing of JIRA fixversions, it's becoming clear that there > are a few JIRAs listed as being only in trunk that were actually released as > part of either 0.23 or 2.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11718) CHANGES.TXT in trunk is incorrect
Allen Wittenauer created HADOOP-11718: - Summary: CHANGES.TXT in trunk is incorrect Key: HADOOP-11718 URL: https://issues.apache.org/jira/browse/HADOOP-11718 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer There are a few JIRAs listed as being only in trunk that were actually released as part of 2.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-7947) Validate XMLs if a relevant tool is available, when using scripts
[ https://issues.apache.org/jira/browse/HADOOP-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362775#comment-14362775 ] Tsuyoshi Ozawa commented on HADOOP-7947: It sounds good to me. > Validate XMLs if a relevant tool is available, when using scripts > - > > Key: HADOOP-7947 > URL: https://issues.apache.org/jira/browse/HADOOP-7947 > Project: Hadoop Common > Issue Type: Wish > Components: scripts >Affects Versions: 2.7.0 >Reporter: Harsh J >Assignee: Kengo Seki > Labels: newbie > Attachments: HADOOP-7947.001.patch, HADOOP-7947.002.patch, > HADOOP-7947.003.patch > > > Given that we are locked down to using only XML for configuration and most of > the administrators need to manage it by themselves (unless a tool that > manages for you is used), it would be good to also validate the provided > config XML (*-site.xml) files with a tool like {{xmllint}} or maybe Xerces > somehow, when running a command or (at least) when starting up daemons. > We should use this only if a relevant tool is available, and optionally be > silent if the env. requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11667) Improve Credentials class for thread safe to avoid corruption for shared credentials.
[ https://issues.apache.org/jira/browse/HADOOP-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362677#comment-14362677 ] Hadoop QA commented on HADOOP-11667: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704706/HADOOP-11667.000.patch against trunk revision bc9cb3e. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ipc.TestRPCWaitForProxy Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5948//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5948//console This message is automatically generated. > Improve Credentials class for thread safe to avoid corruption for shared > credentials. > - > > Key: HADOOP-11667 > URL: https://issues.apache.org/jira/browse/HADOOP-11667 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HADOOP-11667.000.patch > > > Improve Credentials class for thread safe to avoid corruption for shared > credentials between Jobs. > The shared credentials corruption happened at cascading job client: > https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362656#comment-14362656 ] Larry McCay commented on HADOOP-11717: -- Hi [~wheat9] - good questions. I will try and address them. First a couple background point that may help in general: * JWT is a token that is gaining more and more acceptance as a great token to normalize authentication events that are a result of many different authentication servers/mechanisms. In fact, a jwt specific OAuth 2 profile is available: http://tools.ietf.org/html/draft-ietf-oauth-jwt-bearer-04 * This patch does not change the existing Hadoop sso tokens. It introduces a specific type of token - JWT - as a new mechanism for acquiring the hadoop.auth cookie. We shouldn't try and boil the ocean and do everything for everyone in this handler. * This patch provides a very specific behavior - WebSSO through redirect that results in a JWT token from which the current hadoop auth cookie is created. It also tries to make it possible for extensions to provide new implementations for various aspects of the token validation. That said, this is just a one option to be available where it is appropriate. It is certainly not being made the default option or a required one. * It will likely be extended to add support for other ways to get a JWT from the request at some point in the future. 1. OAuth 2.0 - there is nothing about this patch that precludes us from adding a similar handler for OAuth or any other protocol as we see fit. This patch introduces support for WebSSO type flows for UIs. Perhaps, OAuth 2.0 can be accommodated within this flow where it would result in a JWT token or perhaps we would add another handler altogether. 2. As I described about the auth cookie in Hadoop continues to be the existing cookie - this patch provides a new token that can be used like a credential for acquiring a hadoop auth cookie. We aren't changing how things currently work - just providing an alternative that allows for certain integration capabilities. 3. I don't see any real need to separate the redirecting capability of this patch from the specific type of token for a couple reasons: the redirecting capability alone is very simple and doesn't require an abstract or base class, the JWT processing available in the nimbus-jose-jwt library is quite succinct and easily understood and we can always refactor them apart later if the need arises. In the absence of other usecases, I think it is premature to provide the separation. 4. HMAC vs RSA - this is an interesting topic. This really comes down to HMAC vs PKI. While either can be used to provide integrity checking and establish trust relationships, HMAC requires a shared secret between the parties. This means that the secret must be available to both the signer and the consumer of the HMAC. In Hadoop this means that it must be available to many different processes/system users. The more such a secret is available the more easily it is compromised. PKI on the other hand only requires the public key be distributed to the consumers. It doesn't have to be kept completely secret like a shared secret because - it is public. You just have to know that you got it from a trusted party. An admin setting the PEM as a configuration element provides exactly that level of trust without requiring a secret distribution mechanism and encrypted storage of the key. I am actually considering providing a signer secret provider and some refactoring that is based on PKI as well. This would provide the same benefits for distribution and storage for the delegation and hadoop auth tokens. I hope that these points explain my design choices here. I plan to provide a design document for this improvement that spells out the configuration and implementation clearly. Thank you for the very good questions! > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch, > HADOOP-11717-3.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiratio
[jira] [Commented] (HADOOP-11716) Bump netty version to 4.1
[ https://issues.apache.org/jira/browse/HADOOP-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362655#comment-14362655 ] Hadoop QA commented on HADOOP-11716: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704696/HADOOP-11716.003.patch against trunk revision bc9cb3e. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5947//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5947//console This message is automatically generated. > Bump netty version to 4.1 > - > > Key: HADOOP-11716 > URL: https://issues.apache.org/jira/browse/HADOOP-11716 > Project: Hadoop Common > Issue Type: Bug >Reporter: Haohui Mai >Assignee: Haohui Mai > Attachments: HADOOP-11716.000.patch, HADOOP-11716.001.patch, > HADOOP-11716.002.patch, HADOOP-11716.003.patch > > > This jira proposes to bump the netty version from 4.0 to 4.1 so that it is > possible to leverage the HTTP/2 support from netty. > Note that this is a compatible change: the dependency of netty 4.0 is > introduced during the 2.7 timeframe and no release has been made during the > time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11667) Improve Credentials class for thread safe to avoid corruption for shared credentials.
[ https://issues.apache.org/jira/browse/HADOOP-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated HADOOP-11667: --- Status: Patch Available (was: Open) > Improve Credentials class for thread safe to avoid corruption for shared > credentials. > - > > Key: HADOOP-11667 > URL: https://issues.apache.org/jira/browse/HADOOP-11667 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HADOOP-11667.000.patch > > > Improve Credentials class for thread safe to avoid corruption for shared > credentials between Jobs. > The shared credentials corruption happened at cascading job client: > https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11667) Improve Credentials class for thread safe to avoid corruption for shared credentials.
[ https://issues.apache.org/jira/browse/HADOOP-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated HADOOP-11667: --- Attachment: HADOOP-11667.000.patch > Improve Credentials class for thread safe to avoid corruption for shared > credentials. > - > > Key: HADOOP-11667 > URL: https://issues.apache.org/jira/browse/HADOOP-11667 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HADOOP-11667.000.patch > > > Improve Credentials class for thread safe to avoid corruption for shared > credentials between Jobs. > The shared credentials corruption happened at cascading job client: > https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362630#comment-14362630 ] Yi Liu commented on HADOOP-11710: - {quote} I cherry-picked this to branch-2.7 {quote} Oh, I missed that. Thanks for committing to branch-2.7, [~ozawa]. > Make CryptoOutputStream behave like DFSOutputStream wrt synchronization > --- > > Key: HADOOP-11710 > URL: https://issues.apache.org/jira/browse/HADOOP-11710 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.6.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.7.0 > > Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, > HADOOP-11710.3.patch.txt > > > per discussion on parent, as an intermediate solution make CryptoOutputStream > behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362602#comment-14362602 ] Haohui Mai commented on HADOOP-11717: - I'm not an expert in the area, but got a couple questions and would appreciate some explanations: * How far off if I need to implement the OAuth 2.0 protocol? * Does it mean that JWT tokens are the format of auth cookie in Hadoop SSO cases? Many SSO implementation talks the OAuth 2.0, it doesn't seem that it specifies the token has to be in JSON. * Can you separate the mechanism (if there're no authentication token, then redirect) and the real implementation (JWT tokens)? I don't really follow why RSA / PEM are required if SSO is the end-goal -- looks like that only integrity is required here, and a simple HMAC would work as what we did in Hadoop delegation token. Thanks. > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch, > HADOOP-11717-3.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362599#comment-14362599 ] Hadoop QA commented on HADOOP-11717: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704694/HADOOP-11717-3.patch against trunk revision bc9cb3e. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6 warning messages. See https://builds.apache.org/job/PreCommit-HADOOP-Build/5946//artifact/patchprocess/diffJavadocWarnings.txt for details. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-auth. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5946//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5946//console This message is automatically generated. > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch, > HADOOP-11717-3.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11716) Bump netty version to 4.1
[ https://issues.apache.org/jira/browse/HADOOP-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HADOOP-11716: Attachment: HADOOP-11716.003.patch > Bump netty version to 4.1 > - > > Key: HADOOP-11716 > URL: https://issues.apache.org/jira/browse/HADOOP-11716 > Project: Hadoop Common > Issue Type: Bug >Reporter: Haohui Mai >Assignee: Haohui Mai > Attachments: HADOOP-11716.000.patch, HADOOP-11716.001.patch, > HADOOP-11716.002.patch, HADOOP-11716.003.patch > > > This jira proposes to bump the netty version from 4.0 to 4.1 so that it is > possible to leverage the HTTP/2 support from netty. > Note that this is a compatible change: the dependency of netty 4.0 is > introduced during the 2.7 timeframe and no release has been made during the > time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Larry McCay updated HADOOP-11717: - Status: Patch Available (was: Open) > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch, > HADOOP-11717-3.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Larry McCay updated HADOOP-11717: - Attachment: HADOOP-11717-3.patch New patch revision that addresses review comments from [~drankye]. > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch, > HADOOP-11717-3.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Larry McCay updated HADOOP-11717: - Status: Open (was: Patch Available) > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-7947) Validate XMLs if a relevant tool is available, when using scripts
[ https://issues.apache.org/jira/browse/HADOOP-7947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362544#comment-14362544 ] Allen Wittenauer commented on HADOOP-7947: -- The more I think about this, the more I think we may be over thinking it. I need to verify, but I suspect we already have entire directory checking: {code} $ hadoop --conf dir conftest {code} should check the whole directory, right? Perhaps what really needs to happen is: {code} $ hadoop conftest -conffile file {code} ... in other words, a rename of the option. This means we could commit this as-is and fix -conf to be a directory at a later date. > Validate XMLs if a relevant tool is available, when using scripts > - > > Key: HADOOP-7947 > URL: https://issues.apache.org/jira/browse/HADOOP-7947 > Project: Hadoop Common > Issue Type: Wish > Components: scripts >Affects Versions: 2.7.0 >Reporter: Harsh J >Assignee: Kengo Seki > Labels: newbie > Attachments: HADOOP-7947.001.patch, HADOOP-7947.002.patch, > HADOOP-7947.003.patch > > > Given that we are locked down to using only XML for configuration and most of > the administrators need to manage it by themselves (unless a tool that > manages for you is used), it would be good to also validate the provided > config XML (*-site.xml) files with a tool like {{xmllint}} or maybe Xerces > somehow, when running a command or (at least) when starting up daemons. > We should use this only if a relevant tool is available, and optionally be > silent if the env. requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11713) ViewFileSystem should support snapshot methods.
[ https://issues.apache.org/jira/browse/HADOOP-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362476#comment-14362476 ] Rakesh R commented on HADOOP-11713: --- Failure reason is, since this is depending on the AbstractFileSystem.java {{snapshot}} apis in HDFS-5640 patch, we should re-visit this jira after pushing HDFS-5640 implementation first. > ViewFileSystem should support snapshot methods. > --- > > Key: HADOOP-11713 > URL: https://issues.apache.org/jira/browse/HADOOP-11713 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0, 2.2.0 >Reporter: Chris Nauroth >Assignee: Rakesh R > Attachments: HDFS-5641-001.patch > > > Currently, {{ViewFileSystem}} does not dispatch snapshot methods through the > mount table. All snapshot methods throw {{UnsupportedOperationException}}, > even though the underlying mount points could be HDFS instances that support > snapshots. We need to update {{ViewFileSystem}} to implement the snapshot > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-10420) Add support to Swift-FS to support tempAuth
[ https://issues.apache.org/jira/browse/HADOOP-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362464#comment-14362464 ] Jim VanOosten commented on HADOOP-10420: I was not able to reproduce a build failure. With the patch applied, I executed mvn clean test -DskipTests -DHadoopPatchProcess -Pnative -Ptest-patch in the hadoop-tools directory: [INFO] Executed tasks [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop MapReduce Streaming . SUCCESS [11.182s] [INFO] Apache Hadoop Distributed Copy SUCCESS [8.727s] [INFO] Apache Hadoop Archives SUCCESS [5.091s] [INFO] Apache Hadoop Rumen ... SUCCESS [7.904s] [INFO] Apache Hadoop Gridmix . SUCCESS [8.774s] [INFO] Apache Hadoop Data Join ... SUCCESS [6.043s] [INFO] Apache Hadoop Ant Tasks ... SUCCESS [2.577s] [INFO] Apache Hadoop Extras .. SUCCESS [5.984s] [INFO] Apache Hadoop Pipes ... SUCCESS [8.081s] [INFO] Apache Hadoop OpenStack support ... SUCCESS [7.499s] [INFO] Apache Hadoop Amazon Web Services support . SUCCESS [4.464s] [INFO] Apache Hadoop Azure support ... SUCCESS [8.680s] [INFO] Apache Hadoop Scheduler Load Simulator SUCCESS [6.811s] [INFO] Apache Hadoop Tools Dist .. SUCCESS [0.185s] [INFO] Apache Hadoop Tools ... SUCCESS [0.048s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 1:33.651s [INFO] Finished at: Sun Mar 15 12:40:14 CDT 2015 [INFO] Final Memory: 23M/288M [ > Add support to Swift-FS to support tempAuth > --- > > Key: HADOOP-10420 > URL: https://issues.apache.org/jira/browse/HADOOP-10420 > Project: Hadoop Common > Issue Type: Improvement > Components: fs, fs/swift, tools >Affects Versions: 2.3.0 >Reporter: Jinghui Wang > Attachments: HADOOP-10420-002.patch, HADOOP-10420-003.patch, > HADOOP-10420-004.patch, HADOOP-10420-005.patch, HADOOP-10420-006.patch, > HADOOP-10420-007.patch, HADOOP-10420.patch > > > Currently, hadoop-openstack Swift FS supports keystone authentication. The > attached patch adds support for tempAuth. Users will be able to configure > which authentication to use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11713) ViewFileSystem should support snapshot methods.
[ https://issues.apache.org/jira/browse/HADOOP-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362459#comment-14362459 ] Hadoop QA commented on HADOOP-11713: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704421/HDFS-5641-001.patch against trunk revision bc9cb3e. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:red}-1 javac{color:red}. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5945//console This message is automatically generated. > ViewFileSystem should support snapshot methods. > --- > > Key: HADOOP-11713 > URL: https://issues.apache.org/jira/browse/HADOOP-11713 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.0.0, 2.2.0 >Reporter: Chris Nauroth >Assignee: Rakesh R > Attachments: HDFS-5641-001.patch > > > Currently, {{ViewFileSystem}} does not dispatch snapshot methods through the > mount table. All snapshot methods throw {{UnsupportedOperationException}}, > even though the underlying mount points could be HDFS instances that support > snapshots. We need to update {{ViewFileSystem}} to implement the snapshot > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11713) ViewFileSystem should support snapshot methods.
[ https://issues.apache.org/jira/browse/HADOOP-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HADOOP-11713: -- Status: Patch Available (was: Open) > ViewFileSystem should support snapshot methods. > --- > > Key: HADOOP-11713 > URL: https://issues.apache.org/jira/browse/HADOOP-11713 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.2.0, 3.0.0 >Reporter: Chris Nauroth >Assignee: Rakesh R > Attachments: HDFS-5641-001.patch > > > Currently, {{ViewFileSystem}} does not dispatch snapshot methods through the > mount table. All snapshot methods throw {{UnsupportedOperationException}}, > even though the underlying mount points could be HDFS instances that support > snapshots. We need to update {{ViewFileSystem}} to implement the snapshot > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11558) Fix dead links to doc of hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362418#comment-14362418 ] Hudson commented on HADOOP-11558: - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2083 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2083/]) HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. (ozawa: rev 7da136ecca4dafc83ef69b5d9980fa5b67ada084) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." (ozawa: rev b308a8d181416b5fe6bf77756e5f2c7b8fbd793c) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. (ozawa: rev 79426f3334ade5850fbf169764f540ede00fe366) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt > Fix dead links to doc of hadoop-tools > - > > Key: HADOOP-11558 > URL: https://issues.apache.org/jira/browse/HADOOP-11558 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Jean-Pierre Matsumoto >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11558.001.patch > > > Some links to dosc of hadoop-tools are dead by HADOOP-10976. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11558) Fix dead links to doc of hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362413#comment-14362413 ] Hudson commented on HADOOP-11558: - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #133 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/133/]) HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. (ozawa: rev 7da136ecca4dafc83ef69b5d9980fa5b67ada084) * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." (ozawa: rev b308a8d181416b5fe6bf77756e5f2c7b8fbd793c) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. (ozawa: rev 79426f3334ade5850fbf169764f540ede00fe366) * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md > Fix dead links to doc of hadoop-tools > - > > Key: HADOOP-11558 > URL: https://issues.apache.org/jira/browse/HADOOP-11558 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Jean-Pierre Matsumoto >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11558.001.patch > > > Some links to dosc of hadoop-tools are dead by HADOOP-10976. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362410#comment-14362410 ] Larry McCay commented on HADOOP-11717: -- Thanks for the review, Kai. I will address as many of the comments that you raise as appropriate for this iteration and provide a new patch. I'll try and address our comments/questions here to help clarify: * I believe that updating both poms is necessary in order to avoid putting the version of the new dependency in the hadoop-auth module. They get spelled out specifically in hadoop-project and referenced in the other modules. NOTE: bouncy castle is being excluded for this patch. There is nothing in the WebSSO usecase that requires it. * I agree with your assertion that the token should only be sent over SSL. This should be managed by the authentication server that creates the cookie. It must be able to be set to Secure only. * For WebSSO - I see the use of a cookie as fine and it aligns with the current usage of the hadoop.auth token in Hadoop. At some point later we could add a POST profile if there is a need. * I will limit the use of the word cookie as you suggest and ensure that it is just one way to acquire the token from the request. I already had this in mind for a later improvement - as I want to add support for JWT as a bearer token as well. * The bearer token usecase I mentioned above would be useful REST calls and is what I have in mind there. However, this patch does not introduce support for webhdfs or other REST servers yet. We will tackle them after this gets in. * Since the intent of the JWT token at this point is to allow for the creation of the hadoop.auth cookie, it can and should have a shorter lived expiration date. Just long enough to make sure that the normal hadoop cookie can be acquired. Tying their lifetimes together wouldn't add any value there. * I will refactor the handleJWTToken as you suggest * userName will be whatever the authentication server provides in the JWT. It will only ever be as unique as asserted by the issuer. * The message regarding the header and footer actually is correct. The required configuration is the PEM encoded certificate *without* the header and footer. This is actually the same way that public keys are configured in shibboleth and works well. The wording of the message needs to be improved to make sure that it is clear. * Yes, we only support a PEM configuration element for the public key in this patch. There is however a method for setting the RSAPublicKey directly that is only used in tests at the moment. We can add KeyProvider API support for getting the public key later - if that makes sense. I actually find the configuration approach preferable. It can easily be added through a management console, it is a public key - so the file permissions protection for the config file is plenty of protection. I will post a new patch today. Thanks again! > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11558) Fix dead links to doc of hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362405#comment-14362405 ] Hudson commented on HADOOP-11558: - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #124 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/124/]) HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. (ozawa: rev 7da136ecca4dafc83ef69b5d9980fa5b67ada084) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." (ozawa: rev b308a8d181416b5fe6bf77756e5f2c7b8fbd793c) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. (ozawa: rev 79426f3334ade5850fbf169764f540ede00fe366) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt > Fix dead links to doc of hadoop-tools > - > > Key: HADOOP-11558 > URL: https://issues.apache.org/jira/browse/HADOOP-11558 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Jean-Pierre Matsumoto >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11558.001.patch > > > Some links to dosc of hadoop-tools are dead by HADOOP-10976. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11558) Fix dead links to doc of hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362403#comment-14362403 ] Hudson commented on HADOOP-11558: - FAILURE: Integrated in Hadoop-Hdfs-trunk #2065 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2065/]) HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. (ozawa: rev 7da136ecca4dafc83ef69b5d9980fa5b67ada084) * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." (ozawa: rev b308a8d181416b5fe6bf77756e5f2c7b8fbd793c) * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-common-project/hadoop-common/CHANGES.txt HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. (ozawa: rev 79426f3334ade5850fbf169764f540ede00fe366) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm > Fix dead links to doc of hadoop-tools > - > > Key: HADOOP-11558 > URL: https://issues.apache.org/jira/browse/HADOOP-11558 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Jean-Pierre Matsumoto >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11558.001.patch > > > Some links to dosc of hadoop-tools are dead by HADOOP-10976. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11717) Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth
[ https://issues.apache.org/jira/browse/HADOOP-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362388#comment-14362388 ] Kai Zheng commented on HADOOP-11717: I read the non-trivial patch, it's really decent and of very good quality. A good job ! My comments are so far: 1. Why we need to add BC and nimbus library deps to hadoop-project, since they're already in hadoop-auth project ? 2. For secure protecting JWT token, we should use SSL for the web flow. We might need to add such security consideration texts in the new handler header comment. 3. I'm not sure we could avoid using cookie to pass the JWT token, since it's not a good practice. By post and putting it in the body instead ? 4. Anyway, please limit cookie just as one method to convey token, so better to avoid cookie stuffs in the many places (variables, words in logs and etc.). 5. I guess in somewhere we need document how to configure the new authentication handler, to feed the new properties like the login url. 6. Do we support the new mechanism for the both web UI and web hdfs ? Allow SSO between the two ? How would you go ? In HADOOP-10671, it allows the same configurations set for the both, thus SSO effect can be achieved. 7. Do we consider JWT token lifetime ? I thought maybe we should limit the lifetime of the resultant authentication token (hadoop-auth) to the lifetime of the JWT token. 8. Where {{originalUrl}} is used ? A constant for it ? 9. Can you construct {{loginURL}} only when necessary ? I thought it makes sense. 10. I thought {{handleJWTToken}} instead of {{handleJWTCookie}}. Anyway, for it: 1) Why we have a userName parameter ? Looks like not used. 2) Would we rewrite it for better reading and extension. Suggest: {code} handleJWTCookie(jwtToken) { boolean validated = validateToken(jwtToken); ... } validateToken(jwtToken) { validateSignature(jwtToken); validateAudiences(jwtToken); validateExpiration(jwtToken); } {code} Other effort like HADOOP-10959 can easily override validateToken method. 3) I thought the coding style here might be a little different from the project. 11. Only {{userName}} is used as the result of web sso, but I'm not sure that's enough to ensure its uniqueness. 12. Ref. below, the message isn't correct. By the way, looks like we only support PEM format. {code} + if (pem.startsWith(PEM_HEADER)) { +message = "CertificateException - do not include PEM header and footer"; + } {code} > Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth > - > > Key: HADOOP-11717 > URL: https://issues.apache.org/jira/browse/HADOOP-11717 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Larry McCay >Assignee: Larry McCay > Attachments: HADOOP-11717-1.patch, HADOOP-11717-2.patch > > > Extend AltKerberosAuthenticationHandler to provide WebSSO flow for UIs. > The actual authentication is done by some external service that the handler > will redirect to when there is no hadoop.auth cookie and no JWT token found > in the incoming request. > Using JWT provides a number of benefits: > * It is not tied to any specific authentication mechanism - so buys us many > SSO integrations > * It is cryptographically verifiable for determining whether it can be trusted > * Checking for expiration allows for a limited lifetime and window for > compromised use > This will introduce the use of nimbus-jose-jwt library for processing, > validating and parsing JWT tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11558) Fix dead links to doc of hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362358#comment-14362358 ] Hudson commented on HADOOP-11558: - SUCCESS: Integrated in Hadoop-Yarn-trunk #867 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/867/]) HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. (ozawa: rev 7da136ecca4dafc83ef69b5d9980fa5b67ada084) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." (ozawa: rev b308a8d181416b5fe6bf77756e5f2c7b8fbd793c) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. (ozawa: rev 79426f3334ade5850fbf169764f540ede00fe366) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md > Fix dead links to doc of hadoop-tools > - > > Key: HADOOP-11558 > URL: https://issues.apache.org/jira/browse/HADOOP-11558 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Jean-Pierre Matsumoto >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11558.001.patch > > > Some links to dosc of hadoop-tools are dead by HADOOP-10976. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11558) Fix dead links to doc of hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362352#comment-14362352 ] Hudson commented on HADOOP-11558: - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #133 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/133/]) HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. (ozawa: rev 7da136ecca4dafc83ef69b5d9980fa5b67ada084) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm * hadoop-common-project/hadoop-common/CHANGES.txt Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." (ozawa: rev b308a8d181416b5fe6bf77756e5f2c7b8fbd793c) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. (ozawa: rev 79426f3334ade5850fbf169764f540ede00fe366) * hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm > Fix dead links to doc of hadoop-tools > - > > Key: HADOOP-11558 > URL: https://issues.apache.org/jira/browse/HADOOP-11558 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Jean-Pierre Matsumoto >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11558.001.patch > > > Some links to dosc of hadoop-tools are dead by HADOOP-10976. -- This message was sent by Atlassian JIRA (v6.3.4#6332)