[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2233: Attachment: apache-yarn-2233.5.patch Uploaded new patch fixing findbug error. The test case failures are due to TestClientRMService.testForceKillApplication failing which lead to a whole bunch of subsequent tests to fail. > Implement web services to create, renew and cancel delegation tokens > > > Key: YARN-2233 > URL: https://issues.apache.org/jira/browse/YARN-2233 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev >Priority: Blocker > Attachments: apache-yarn-2233.0.patch, apache-yarn-2233.1.patch, > apache-yarn-2233.2.patch, apache-yarn-2233.3.patch, apache-yarn-2233.4.patch, > apache-yarn-2233.5.patch > > > Implement functionality to create, renew and cancel delegation tokens. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2233:
Attachment: apache-yarn-2233.4.patch
{quote}
bq.It seems to me that all API implementations should take the fulll
principle name if available.
I meant to replace all occurrences of getCallerUserGroupInformation(hsr), if
that makes sense.
{quote}
Fixed this. Use the principal everywhere
{quote}
bq.We should set all the fields of a DT - token, renewer, expiration-time
all the time - new-token, renew-token? renewDelegationToken only returns only
the expiry-time and getToken only returns the token. This is consistent with
RPCs. But I think in a followup, we should fix this.
Fixed.
bq. You meant we will fix this in a separate JIRA? I still see renewToken not
returning the entire token info. I'm okay doing it separately, just clarifying
what you said..
{quote}
I've fixed this for creating a new delegation token but I didn't fix it for
renew token. I think it's ok to fix it as part of a seperate JIRA.
> Implement web services to create, renew and cancel delegation tokens
>
>
> Key: YARN-2233
> URL: https://issues.apache.org/jira/browse/YARN-2233
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Varun Vasudev
>Assignee: Varun Vasudev
>Priority: Blocker
> Attachments: apache-yarn-2233.0.patch, apache-yarn-2233.1.patch,
> apache-yarn-2233.2.patch, apache-yarn-2233.3.patch, apache-yarn-2233.4.patch
>
>
> Implement functionality to create, renew and cancel delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2233:
Attachment: apache-yarn-2233.3.patch
bq.Shouldn't RMWebservices.DELEGATION_TOKEN_HEADER be a public constant? I
don't know what the right place is, though, may be the tokenID-itself..
I'm not sure what the right place is either.
bq.createDelegationToken: Null renewer check should be in ClientRMService?
Fixed.
bq.It seems to me that all API implementations should take the fulll
principle name if available.
Fixed.
bq.Call this API as alpha too?
Fixed.
bq.We should set all the fields of a DT - token, renewer, expiration-time
all the time - new-token, renew-token? renewDelegationToken only returns only
the expiry-time and getToken only returns the token. This is consistent with
RPCs. But I think in a followup, we should fix this.
Fixed.
bq. Is renewal better posted as POST /ws/v1/cluster/renew-delegation-token?
I've changed the API to make renewal POST
/ws/v1/cluster/delegation-token/expiration.
{quote}
assertTrue(tok.getNextExpirationTime() > oldExpirationTime);
You may want to put artificial sleeps, we have seen cases in the past where
they turn up being the same failing the test
{quote}
Fixed.
{quote}
testCancelDelegationToken
// Ideally, the owner should be able to cancel his own tokens but a bug
// prevent that
This is fixed now..
We should also look into the RM and validate that it is indeed
cancelled?
{quote}
Fixed.
> Implement web services to create, renew and cancel delegation tokens
>
>
> Key: YARN-2233
> URL: https://issues.apache.org/jira/browse/YARN-2233
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Varun Vasudev
>Assignee: Varun Vasudev
>Priority: Blocker
> Attachments: apache-yarn-2233.0.patch, apache-yarn-2233.1.patch,
> apache-yarn-2233.2.patch, apache-yarn-2233.3.patch
>
>
> Implement functionality to create, renew and cancel delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2233:
Attachment: apache-yarn-2233.2.patch
{quote}
1. This won't happen inside renewDelegationToken, as it is already validated
before.
{noformat}
+if (tokenData.getToken().isEmpty()) {
+ throw new BadRequestException("Empty token in request");
+}
{noformat}
2. It seems that some of the fields in DelegationToken are no longer necessary.
3. assertValidToken seems not to be necessary.
{quote}
Fixed all 3. I also fixed the FindBug warnings that were caused.
> Implement web services to create, renew and cancel delegation tokens
>
>
> Key: YARN-2233
> URL: https://issues.apache.org/jira/browse/YARN-2233
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Varun Vasudev
>Assignee: Varun Vasudev
>Priority: Blocker
> Attachments: apache-yarn-2233.0.patch, apache-yarn-2233.1.patch,
> apache-yarn-2233.2.patch
>
>
> Implement functionality to create, renew and cancel delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2233:
Attachment: apache-yarn-2233.1.patch
{quote}
1.
bq. It should be noted that when cancelling a token, the token to be cancelled
is specified by setting a header.
Any reason for specifying the token in head? If there's something
non-intuitive, maybe we should have some in-code comments for other developers?
{quote}
I've added comments to the code explaining why this is. Jetty doesn't allow
request bodies for DELETE methods.
{quote}
2. RPC get delegation token API doesn't have these fields, but it seems to be
nice have. We may want to file a Jira.
{noformat}
+long currentExpiration = ident.getIssueDate() + tokenRenewInterval;
+long maxValidity = ident.getMaxDate();
{noformat}
{quote}
Fixed this. I've left the fields out for now to match the RPC response. I'll
file tickets to add the information to both interfaces.
{quote}
3. Is it possible to reuse KerberosTestUtils in hadoop-auth?
{quote}
I missed this. hadoop-auth doesn't export test jars for us to use. I've changed
the pom.xml to start generating test-jars for hadoop-auth and used
KerberosTestUtils from there.
{quote}
4. Is this supposed to test invalid request body? It doesn't look like the
invalid body construction in the later tests.
{noformat}
+response =
+resource().path("ws").path("v1").path("cluster")
+ .path("delegation-token").accept(contentType)
+ .entity(dtoken, mediaType).post(ClientResponse.class);
+assertEquals(Status.BAD_REQUEST, response.getClientResponseStatus());
{noformat}
{quote}
This is actually a test with the renewer missing from the request body, hence
the BAD_REQUEST.
{quote}
1. No need of "== ture".
{noformat}
+if (usePrincipal == true) {
{noformat}
Similarly,
{noformat}
+if (KerberosAuthenticationHandler.TYPE.equals(authType) == false) {
{noformat}
{quote}
Fixed.
{quote}
2. If I remember it correctly, callerUGI.doAs will throw
UndeclaredThrowableException, which wraps the real raised exception. However,
UndeclaredThrowableException is an RE, this code cannot capture it.
{noformat}
+try {
+ resp =
+ callerUGI
+.doAs(new PrivilegedExceptionAction() {
+ @Override
+ public GetDelegationTokenResponse run() throws IOException,
+ YarnException {
+GetDelegationTokenRequest createReq =
+GetDelegationTokenRequest.newInstance(renewer);
+return rm.getClientRMService().getDelegationToken(createReq);
+ }
+});
+} catch (Exception e) {
+ LOG.info("Create delegation token request failed", e);
+ throw e;
+}
{noformat}
{quote}
I'm unsure about this. RE is a sub-class of Exception. Why won't this code work?
{quote}
3. Cannot return respToken simply? The framework should generate "OK" status
automatically, right?
{noformat}
+return Response.status(Status.OK).entity(respToken).build();
{noformat}
{quote}
There are a few cases where we need to send a FORBIDDEN response back and the
GenericExceptionHandler doesn't return FORBIDDEN responses.
{quote}
4. You can call tk.decodeIdentifier directly.
{noformat}
+RMDelegationTokenIdentifier ident = new RMDelegationTokenIdentifier();
+ByteArrayInputStream buf = new ByteArrayInputStream(tk.getIdentifier());
+DataInputStream in = new DataInputStream(buf);
+ident.readFields(in);
{noformat}
{quote}
Fixed. Thanks for this, cleaned up bunch of boilerplate code.
> Implement web services to create, renew and cancel delegation tokens
>
>
> Key: YARN-2233
> URL: https://issues.apache.org/jira/browse/YARN-2233
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Varun Vasudev
>Assignee: Varun Vasudev
>Priority: Blocker
> Attachments: apache-yarn-2233.0.patch, apache-yarn-2233.1.patch
>
>
> Implement functionality to create, renew and cancel delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-2233: -- Component/s: resourcemanager Priority: Blocker (was: Major) Target Version/s: 2.5.0 Marked for 2.5 and making it a blocker as I'd like to get it in to make RM web-services usable.. > Implement web services to create, renew and cancel delegation tokens > > > Key: YARN-2233 > URL: https://issues.apache.org/jira/browse/YARN-2233 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev >Priority: Blocker > Attachments: apache-yarn-2233.0.patch > > > Implement functionality to create, renew and cancel delegation tokens. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2233) Implement web services to create, renew and cancel delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2233: Attachment: apache-yarn-2233.0.patch Uploaded patch. > Implement web services to create, renew and cancel delegation tokens > > > Key: YARN-2233 > URL: https://issues.apache.org/jira/browse/YARN-2233 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: apache-yarn-2233.0.patch > > > Implement functionality to create, renew and cancel delegation tokens. -- This message was sent by Atlassian JIRA (v6.2#6252)
