[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690650#comment-13690650 ] Luke Lu commented on HADOOP-9421: - Filed HADOOP-9662 for more unit tests. +1 for the patch to unblock 2.1 scale testing. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690639#comment-13690639 ] Daryn Sharp commented on HADOOP-9421: - Oh, and I did manually test on a secure cluster, using all combinations of secure/insecure server + client using kerberos/simple/tokens. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690638#comment-13690638 ] Daryn Sharp commented on HADOOP-9421: - We agreed that guessing wasn't going to be implemented. There's no penalty for token auth because of my other optimization. Kerberos takes a tiny penalty but it's necessary to support subsequent IP failover changes. My existing SASL unit tests exercise the full matrix of client and server auths, both with and without tokens. Writing tests all the way down to the wire is going to further delay this patch. I think Arun is pretty eager to get this in. If we want wire tests, then I think we should defer to another jira and get 2.1 unblocked. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690626#comment-13690626 ] Luke Lu commented on HADOOP-9421: - You're right, I somehow read that it is trying to buildNegotiateResponse in that branch. Though the patch always sends client NEGOTIATE, which means a slight performance regression on typical clusters, I think the protocol is in a good shape now. Can you add some unit tests, as it's easy to break the code due to all the states? Cluster testing results would be appreciated as well. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690604#comment-13690604 ] Daryn Sharp commented on HADOOP-9421: - That's not a bug. The logic is "if the client sent an INITIATE which was a bad guess, only throw an exception if it has already been sent a NEGOTIATE". It's what allows you desire for the client to guess. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690600#comment-13690600 ] Luke Lu commented on HADOOP-9421: - In light of the bug, can you add a unit test for the INITIATE/NEGOTIATE/INITIATE path? The sentNegotiate state will also need to be kept per flow if you want to multiplex the connection. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690593#comment-13690593
]
Hadoop QA commented on HADOOP-9421:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12589114/HADOOP-9421.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2690//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2690//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690585#comment-13690585 ] Luke Lu commented on HADOOP-9421: - I'm fine with the new protocol. I agree that REINITIATE is not necessary if server keeps the sentNegotiate state for the connection. One bug in the patch: in Server case INITIATE, in the authMethod == null, the if (sentNegotiate) should be if (!sentNegotiate)... > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690558#comment-13690558 ] Luke Lu commented on HADOOP-9421: - bq. Server will not SASL respond to a client until it sends either NEGOTIATE or INITIATE. OK, use client NEGOTIATE to replace my empty INITIATE. That's cool. bq. If the client sends INITIATE, and guesses wrong, the server responds with NEGOTIATE. Again, the client now has one shot to send a valid INITIATE. I believe the code could get tricky without a REINITIATE. bq. Your patch would allow a client to spam INITIATE and keep the socket tied up indefinitely. My patch only sends one INITIATE after connection header, it either succeeds or get a NEGOTIATE to REINITIATE, which doesn't have a transition to NEGOTIATE again. The code is simple and succinct. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690539#comment-13690539 ] Luke Lu commented on HADOOP-9421: - bq. if the client decided to guess and blast a INITIATE, it could simply set a ignoreNegotiate boolean. When failover/upgrade happens, the INITIATE can fail and potentially two NEGOTIATE in row. You need to pick the right one. I think this increase client complexity unnecessarily. bq. A server upgrade is exactly one of the reasons the client shouldn't be guessing. Or one the reasons to REINTIATE? :) > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690513#comment-13690513 ] Daryn Sharp commented on HADOOP-9421: - It's really not worth debating anymore, but if the client decided to guess and blast a INITIATE, it could simply set a ignoreNegotiate boolean. A server upgrade is exactly one of the reasons the client shouldn't be guessing. During authentication the socket reader isn't "fair". As long as there is pending data from the client, it will keep reading until it gets the first RPC. It's not a problem today because the client is forced along a path that either ends in the connection being aborted, or the reading of an RPC. Your patch would allow a client to spam INITIATE and keep the socket tied up indefinitely. New patch arriving shortly... > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690510#comment-13690510 ] Luke Lu commented on HADOOP-9421: - bq. I'll just add it. Thank you! It's important to note that we don't need to add the optimizations for this jira, which should focus on fixing the protocol. Once the protocol is fixed, we can explore new non-blocking client in different languages and enable a Cambrian explosion of new use cases! bq. REINITIATE is only supposed to happen when mech cache goes stale. s/only//. System designer can make it happen rarely when mech cache is empty or stale and when serverId failover happens. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690496#comment-13690496 ] Luke Lu commented on HADOOP-9421: - bq. Ok, so now how would you handle improved serverId based tokens? Always take the REINITIATE hit? Maintain a persistent (serverId -> mech) cache in /tmp/hadoop-$user/rpc. Again: it should be choice of protocol users (system designers). bq. In this case, can't the client just blast the INITIATE before getting a NEGOTIATE? Like I said before, ignoring NEGOTIATE in client increase client complexity to ignore the right NEGOTIATE. What if server upgrade/security fix happened in between? bq. do you believe there will actually be a measurable performance difference? Depending on the use case, e.g. a console managing multiple hadoop clusters in multiple regions: the additional round trip could be in hundreds of ms! which is extremely annoying. bq. Will having the client ignore the inflight NEGOTIATE on a reconnect have a measurable latency Like I said, client will have to decide when to ignore the NEGOTIATE, which increase client complexity while still feel dirty. I've shown that this ugliness is not necessary. bq. If so, is the extra round trip for REINITIATEs a bad thing REINITIATE is only supposed to happen when mech cache goes stale. bq. I'll just add it. I'll re-leverage the existing states to avoid adding the reinitate. IMO, explicit REINITIATE make things a lot easier to reason about. bq. prevent a client seizing a connection thread. What're you talking about? The new protocol allows completely non-blocking client. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690351#comment-13690351 ] Daryn Sharp commented on HADOOP-9421: - Although I don't see a scenario in which this optimization is possible, I'll just add it. I'll re-leverage the existing states to avoid adding the reinitate, and prevent a client seizing a connection thread. With tcp nodelay, the two messages aren't combined, but I think that should be easy to fix. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690241#comment-13690241 ] Daryn Sharp commented on HADOOP-9421: - bq. I meant you'll be SOL to make the token optimization work. Your protocol requires an extra round trip to support SCRAM. Ok, so now how would you handle improved serverId based tokens? Always take the REINITIATE hit? bq. For most common hadoop auth work load: distributed containers/tasks, you don't need to guess, it's the delegation token with digest-md5/scram In a mixed security env including not just within the cluster but across different clusters with different versions, or during rolling upgrades when supported mechanisms are upgraded, it will always be a guess. You'll always have to do the REINITIATE? bq. For this contrived case, the client can catch exceptions for the preferred auth when generating the initial token, which would apply to fetching a service ticket for non-kerberos server In mixed security envs, or rolling upgrades, or use of IP failover are a contrived use case? In all these cases, we'll also take the penalty of an unnecessary roundtrip to the KDC for a non-existent service ticket? bq. cache the server auth/mechs for later use to save a round-trip later In this case, can't the client just blast the INITIATE before getting a NEGOTIATE? Actually, the client receiving a NEGOTIATE will allow the client to know in advance of the server reply whether its guess will fail and it can begin preparing the proper auth. bq. In summary, my protocol gives that choice to real system designers. Your protocol takes away that choice because you could not possibly think of all use cases, where auth latency matters. As I asked before, do you believe there will actually be a measurable performance difference? Will having the client ignore the inflight NEGOTIATE on a reconnect have a measurable latency? If so, is the extra round trip for REINITIATEs a bad thing? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690136#comment-13690136 ] Luke Lu commented on HADOOP-9421: - bq. There is nothing in the protocol that would prevent SCRAM being supported. I meant you'll be SOL to make the token optimization work. Your protocol *requires* an extra round trip to support SCRAM. bq. Guessing a supported auth/mechanism For most common hadoop auth work load: distributed containers/tasks, you don't need to guess, it's the delegation token with digest-md5/scram, as it's a framework internal token bootstrapped by other public facing mechanisms. For other use cases, you can use cached values. For the remaining use cases, the client can send an empty INITIATE and use NEGOTIATE and REINITIATE with the same total round-trip cost as yours in all cases. With the optional client initiate, my protocol gives the choices to the practicing system designers instead of the original protocol designers. bq. Dealing with the mishaps when the client blows itself up trying an auth the server doesn't even support INITIATE contains the same extra info like protocol and serverId for preferred auth. Server can simply send a NEGOTIATE if it decides that it cannot support the preferred auth choice, the client can then decide to REINITIATE or abort. bq. if it even attempts kerberos with a non-kerberos server. It won't even succeed far enough to send the INITIATE For this contrived case, the client can catch exceptions for the preferred auth when generating the initial token, which would apply to fetching service ticket for non-kerberos server, and send an empty INITIATE to NEGOTIATE and REINITIATE. Again for integration clients that need to talk to multiple servers with different auths it can simply use empty INITIATE to NEGOTIATE and cache the server auth/mechs for later use to save a round-trip later. Imagine a busy interactive web console that talk to multiple back-end Hadoop servers, it's not feasible to keep a connection per user open to all these servers, you often need to constantly creating new connections to the back-end servers (a connection pool helps), my protocol allows the web console to save a mandatory round-trip from yours, which can make the interactive user experience much better due to lowered latencies. In summary, my protocol gives that choice to real system designers. Your protocol takes away that choice because you could not possibly think of all use cases, where auth latency matters. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690066#comment-13690066
]
Daryn Sharp commented on HADOOP-9421:
-
bq. If we have to replace Digest-MD5 for security reasons, we'll be SOL.
That's completely untrue. There is nothing in the protocol that would prevent
SCRAM being supported.
{noformat}
C -> S connectionHeader(SASL)
C <- S NEGOTIATE { [TOKEN, SCRAM, proto, serverId], ... }
C -> S INITIATE [TOKEN] initial-response
{noformat}
bq. I merely want to leave the optional client initiate proto in the protocol
for future optimizations
In light of everything I've described, please detail what future optimization
is possible?
Please answer, how is the client capable of:
* Guessing a supported auth
* Guessing the supported mechanism for guessed auth
* Based on those guesses, reliably creating a SASL client to generate a SASL
response
* Dealing with the mishaps when the client blows itself up trying an auth the
server doesn't even support
Notably, describe how you would handle the problems I detailed regarding a
client failing if it even attempts kerberos with a non-kerberos server. It
won't even succeed far enough to send the INITIATE.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690051#comment-13690051 ] Luke Lu commented on HADOOP-9421: - bq. In a world of multiple auths and multiple mechanisms for an auth, requiring REINITIATE penalties is too expensive. If a client can't pick a mechanism, it could skip the initial token and send an empty INITIATE, REINITIATE is then not expensive, i.e. exactly equivalent to yours. Hadoop RPC foremost should serve its most common workload: delegation tokens. A performance regression for the most common workload in the name of integration is not acceptable. The specific optimization for Digest-MD5 (cramming a speculative challenge with a negotiate) doesn't work with modern client initiated auths like SCRAM. If we have to replace Digest-MD5 for security reasons, we'll be SOL. bq. Ignoring all the issues I've cited, your optimization doesn't appear to have a positive impact on performance. There is no optimization my patch, which merely leave the door for future optimization. In fact, there is performance bug in my impl for Kerberos. It's you who added a speculative optimization for Digest-MD5 that doesn't work with its future replacement SCRAM. bq. I feel like we've spent weeks haggling over an ill-suited pre-mature optimization that could been spent building upon this implementation. I merely want to leave the optional client initiate proto *in the protocol* for future optimizations. I feel like being forced to implement the optimization to show that it's straight forward and incremental. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690034#comment-13690034 ] Daryn Sharp commented on HADOOP-9421: - You seem to be trying to tailor a design that only considers today's implementation of tokens and kerberos. It seems "easy" when you assume there are only two choices. The optimization becomes more and more complicated, and in many cases impossible, instead of simply doing something the server tells you to do. When pluggable auth support allows a world of heterogenous security, such as Knox or Rhino, requiring REINITIATE penalties becomes very expensive. Sorry for the very long read, but these are topics I intended to address on the call that unfortunately didn't happen today. +IP failover+ Distinct service principals with IP failover isn't "insane". With a shared principal services can't be accessed directly because the host doesn't match the shared principal. So a different config with a hardcoded shared principal is needed. Similarly, DNs won't be able to heartbeat directly into HA NNs. I'm sure there are more problems than we've already discovered investigating that route. The root issue is the client must only use the hostname that appears in the kerberos service principal. Which means you can't access the service via all its interface, hostnames, or even pretty CNAMEs. If server advertises "this is who I am" via the NEGOTIATE, then the problem is solved. +Token selection issues+ Selecting tokens pre-connection based on the service as a host or ip port tuple is a problem. Let's take a few examples: Using the IP precludes multi-interface host support, for instance if you want to have a fast/private intra-cluster network and a separate public network. Tokens will contain the public IP, but clients using the private interface (different IP) can't find them. This isn't contrived, it's something Cloudera has wanted to do. You also can't use the IP because changing a service's IP will break clients using tokens with the old IP. In comes the bane of my creation, use_ip=false, to use the given hostname. But you can't allow non-fully qualified names because it will resolve differently on depending on the dns search path. There's a raft of reasons why the canonicalization isn't as straightforward as you'd think, which led to a custom NetUtils resolver and complicated path normalization. Likewise, any sort of public proxy or NAT-ing between an external client and a cluster service creates an unusable token service within the grid. HA token logic is unnecessarily convoluted to clone tokens from a logical uri into multiple tokens with each failover's service. _Solution_ A clean solution to all these problems is tokens contain a server generated opaque id. The server NEGOTIATE reports this id. The client looks for a token with that id. Now no matter what interface/IP/hostname/proxy/NAT is used, the client will always find the token. If you cut out the use of the NEGOTIATE, this ability is gone. +Supporting new auth methods+ Other new auths in the future may need the protocol/serverId hints from the NEGOTIATE to locate the required credentials. Guessing may not be an option. The RPC client shouldn't have to be modified to make a pre-connection guess for all the auth methods it supports. Because... Why should the client attempt an auth method before it _even knows if the server can do it_? Let's look at some hairy examples: The client tries to do kerberos, so it needs to generate the initial response to take advantage of your "optimization". But the server isn't kerberized. So either the client fails because it has no TGT, which it doesn't even need! Or fails to get a non-existent service principal. What if the client decides to use an SSO service, but the server doesn't do SSO? Take a REINITIATE penalty every time? +Supporting new mechanisms+ Let's say we add support for a new mechanism like SCRAM. Just because the client can do it doesn't mean all services across all clusters can do it. The server's NEGOTIATE will tell the client if it can do DIGEST-MD5, SCRAM, etc. Inter-cluster compatibility and rolling upgrades will introduce scenarios where the required mechanism differs, and penalizing the client to REINITIATE is not a valid option. --- In all of these scenarios, there aren't complex issues if the NEGOTIATE is used to chose an appropriate auth type. In a world of multiple auths and multiple mechanisms for an auth, requiring REINITIATE penalties is too expensive. Ignoring all the issues I've cited, your optimization doesn't appear to have a positive impact on performance. Even if it did shave a few milliseconds or even 100ms, will it have a measurable real-world impact? Considering how many RPC requests are performed over a single connection, will the negligible penal
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689980#comment-13689980
]
Luke Lu commented on HADOOP-9421:
-
bq. At best the client will fail to initiate GSSAPI with the wrong principle.
You'd have to assume an exception means solicit a negotiate message to recreate
the SASL client. At worst you get the wrong service ticket and fail the
negotiation.
I think we can do better with my protocol for the insane (separate server
principals for HA servers for a logical server) fail over case:
{code}
C -> S connectionHeader, INITIATE(old-token, [KERBEROS], old-host)
C <- S NEGOTIATE([TOKEN, KERBEROS], new-host) // no exception as the server can
detect that old-host and new-host being different.
C -> S REINITIATE(new-token, KERBEROS)
...
{code}
Client can cache the server name and in steady state case:
{code}
C -> S connectionHeader, INITIATE(new-token, [KERBEROS], new-host)
C <- S CHALLENGE(final-token)
{code}
Look ma, no NEGOTIATE!
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689976#comment-13689976 ] Daryn Sharp commented on HADOOP-9421: - No, you cannot try to further avoid the negotiate. W/o the negotiate to properly initialize the SASL client for the proper service host, IP failover will never work! > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689971#comment-13689971 ] Luke Lu commented on HADOOP-9421: - bq. Luke's patch is 8-11ms slower with Kerberos than mine Ah, that's probably due to my unoptimized patch does initial token twice (one in INITIATE and the other in REINITIATE), which causes an extra round trip to KDC to get service ticket. But I can fix this with no protocol changes and eliminate the difference. OTOH, I can further eliminate the NEGOTIATE round trip for Kerberos in normal/stead state cases. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689961#comment-13689961
]
Luke Lu commented on HADOOP-9421:
-
bq. Luke, Is there a sufficient need to add a new state REINITIATE ? The only
benefit is that it avoids a isSupportedInitiate() call.
Yes. It's the most straightforward way to avoid a initiate/negotiate loop
besides debuggability :)
Bad SASL to secure example:
{code}
C -> S connectionHeader(SASL), INITIATE([(INVALID, ...)])
C <- S NEGOTIATE([(TOKEN, DIGEST-MD5), (KERBEROS, GSSAPI), ...]) // due to
isSupportedInitiate returning false.
C -> S REINITIATE([(INVALID, ...)]) // note, there is no transition from
REINITIATE to NEGOTIATE
C <- S S RPC Exception "INVALID not supported", connection closed
{code}
The code is simple and hopefully not too subtle (see
Connection#saslReadAndProcess and ProcessSaslMessage).
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689959#comment-13689959 ] Daryn Sharp commented on HADOOP-9421: - You can do the feature that will never used by blasting the initiate, ignore the negotiate response, and hope for the best. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689955#comment-13689955 ] Daryn Sharp commented on HADOOP-9421: - If it's not obvious, I'm -1 on Luke's patch. The biggest issue is we must have IP failover support. At best the client will fail to initiate GSSAPI with the wrong principle. You'd have to assume an exception means solicit a negotiate message to recreate the SASL client. At worst you get the wrong service ticket and fail the negotiation. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689952#comment-13689952 ] Benoy Antony commented on HADOOP-9421: -- Luke, Is there a sufficient need to add a new state REINITIATE ? The only benefit is that it avoids a isSupportedInitiate() call. >From what I understood, Luke's change is a desirable optimization of Daryn's >protocol. Such optimization is quite common in protocols. I agree that it >complicates the design a bit. Tt may never be used with the subsequent changes >related to IP Failover etc, but why not keep it there as it should not cause >any impact on performance? Why would it cause change in performance ? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689950#comment-13689950 ] Daryn Sharp commented on HADOOP-9421: - We also need to consider the issues caused by guessing the auth during rolling upgrades that change the security mechanisms. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689943#comment-13689943 ] Luke Lu commented on HADOOP-9421: - My patch is ready to review, commit and scale testing. I'd appreciate fellow committer reviews and +1s :) I was hoping to nudge Daryn to add optional client initiate. My patch shows that adding optional client initiate is simple and future proof. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689940#comment-13689940 ] Daryn Sharp commented on HADOOP-9421: - I know Luke means well, but an initial initiate will never be used after subsequent changes. The negotiate will always be required for IP failover and improved token selection. The approach is trying to over optimize and weakens the design. I benched both patches, and Luke's token auth times are 1ms slower after 100 samples but let's say they are the same. However, Luke's patch is 8-11ms slower with Kerberos than mine. It also has a subset of the changes to make future changes compatible. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689895#comment-13689895 ] Arun C Murthy commented on HADOOP-9421: --- Guys, how far are we from getting this done? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689870#comment-13689870
]
Luke Lu commented on HADOOP-9421:
-
My "simple to \*" is equivalent Daryn's. Note, consecutive C -> S can be merged
into one TCP packet.
SASL to insecure
{code}
C -> S connectionHeader(SASL), INITIATE(optional initial token)
C <- S SUCCESS
C -> S connectionContext, RPC request
{code}
SASL to secure
{code}
C -> S connectionHeader(SASL), INITIATE(optional initial token, [(TOKEN,
DIGEST-MD5)])
C <- S CHALLENGE(challenge-token) or NEGOTIATE([(TOKEN, DIGEST-MD5), (KERBEROS,
GSSAPI), ...])
C -> S RESPONSE(response-token) or REINITIATE(initial token, [(TOKEN,
DIGEST-MD5)])
...
C <- S SUCCESS(final-token)
C -> S connectionContext, RPC request
{code}
Bottom line: my patch is a strict superset of Daryn's patch from protocol POV.
The keyward is *optional* client initiate. Daryn's protocol can *not* support
SCRAM (or any modern auths requiring client nonce) without an extra round-trip.
Most of the credit of my patch goes to Daryn, as adding optional client
initiate is simple (only a few extra lines).
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689823#comment-13689823
]
Daryn Sharp commented on HADOOP-9421:
-
+Simple to insecure+
{noformat}
C -> S connectHeader(SIMPLE)
C -> S connectionContext
C -> S RPC request
{noformat}
No change/overhead.
+Simple to secure+
{noformat}
C -> S connectHeader(SIMPLE)
C -> S connectionContext
C -> S RPC request
C <- S RPC exception "SIMPLE not supported", close connection
{noformat}
No change/overhead.
Note: There's an existing race condition. The server sent the exception in
response to the connect header, but the client already blasted the context and
request.
+SASL to insecure server+
{noformat}
C -> S connectHeader(SASL)
C <- S SUCCESS
C -> S connectionContext
C -> S RPC request
{noformat}
Immediate success message replaces switch to simple.
+SASL to secure server+
{noformat}
C -> S connectHeader(SASL)
C <- S NEGOTIATE {
[TOKEN, DIGEST-MD5, proto, realm, first-challenge-token]
[KERBEROS, GSSAPI, user, host, null]
}
C -> S INITIATE [ ] response-token
[... back and forth CHALLENGE/RESPONSE ...]
S <- C SUCCESS final-token
C -> S connectionContext
C -> S RPC request
{noformat}
The INITIATE for a token will use the first-challenge-token provided by the
server to cut out a roundtrip. Since the client cannot initiate, the existing
code wastes a round trip soliciting the first DIGEST-MD5 challenge that I'm now
immediately returning.
The INITIATE for kerberos cannot include a first challenge. The GSSAPI
mechanism cannot generate a challenge w/o an initial response from the client.
This is fine and expected.
With this design, a follow on change can allow the client to pick the first
auth type it supports instead of guessing in advance what it should try. It
will use the fields as provided, of course with safeguards to sanity check for
malicious behavior. Using the advertised fields is how the client will support
IP failover.
Walking the advertised auths will allow completely decoupling the auth type
code from the RPC client. The auth types can be implemented as pluggable
services that will be called on demand if the server requests that auth type.
The pluggable services may use the protocol/serverId fields to decide if they
have the credentials to even attempt the auth.
The client should never attempt an auth unless it knows the server supports the
auth, and the client actually has the credentials to do the auth.
+Bad SASL client to secure+
{noformat}
C -> S connectHeader(SASL)
C <- S NEGOTIATE {
[TOKEN, DIGEST-MD5, proto, realm, first-challenge-token]
[KERBEROS, GSSAPI, user, host, null]
}
C -> S INITIATE [ INVALID ] response-token
C <- S RPC Exception "INVALID not supported", connection closed
{noformat}
Well, you were supposed to reply with one of the advertised methods, not make a
blind guess... Sorry, game over.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689774#comment-13689774 ] Larry McCay commented on HADOOP-9421: - Just a thought... I have recently found the use of plantuml very useful for these sorts of discussions. Sequence diagrams may go a long way and they are refreshingly easy with that tool. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689763#comment-13689763 ] Daryn Sharp commented on HADOOP-9421: - bq. That is correct. BTW Client can guess the best default for initial auth since it is the default authentication for the Cluster (e.g. Kerberos or LDAP). Wrong. You can't guess in a heterogeneous security environment for different services. You can't even guess which mechanism a particular service will use for tokens. How will you simultaneously support accessing both LDAP and KERBEROS secured services? Or something pluggable like a SSO_TOKEN on one cluster, "normal" TOKEN on another. And either of them might be using DIGEST-MD5 or SCRAM or something else. How will you guess w/o a slew of config options? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689759#comment-13689759 ] Sanjay Radia commented on HADOOP-9421: -- Luke, Daryn can both of you please summarize the packet exchange for simple, kerberos, and hadoop-tokens for each of your approaches. ALso the exchange when server does not support auth-method X initiated by client and the client is suppose to use another auth-method from a list. Lets first get an agreement on the exchange. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689739#comment-13689739 ] Sanjay Radia commented on HADOOP-9421: -- bq. The client can't chose the best auth bq.Sometimes it's the only choice, eg. delegation token for tasks, which covers 99% of the auth workload in typical clusters. That is correct. BTW Client can guess the best default for initial auth since it is the default authentication for the Cluster (e.g. Kerberos or LDAP). > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689619#comment-13689619 ] Luke Lu commented on HADOOP-9421: - bq. I'm referring to the roundtrip your patch introduces by responding with negotiate if it's a non-token auth. The roundtrip is mandatory for non-token auth in your patch. It's optional in mine. bq. The client can't chose the best auth Sometimes it's the only choice, eg. delegation token for tasks, which covers 99% of the auth workload in typical clusters. bq. The roundtrip reduction "hack" is a feature that can be extended to any sasl mechanism that can initiate. No it cannot reduce round-trip for SCRAM or Kerberos, while my patch can. bq. The point you keep missing is the client can't guess an auth method but you keep focusing on retaining that behavior. The point you keep missing is that the client initiate proto is free and optional from network round-trip point of view, which opens door for future optimization for SCRAM etc. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689490#comment-13689490 ] Daryn Sharp commented on HADOOP-9421: - I'm referring to the roundtrip your patch introduces by responding with negotiate if it's a non-token auth. The client can't chose the best auth, or even know the supported auths, if it's already guessed prior to connection. How will the client know whether the server does DIGEST-MD5 or SCRAM for tokens? It won't work in a mixed environment. Eliminating use_ip is not related to the mech. A server hint is for the token selection itself instead of the fragile way tokens are currently selected. Tokens are completely sensitive to multi-interface hosts, and different hostnames for the same machine. IP failover with a shared principal isn't an option, at least for us. A shared principal prevents direct communication with the HA NNs because the client will use the actual host's principal, not the shared principal. Which also means DNs can't heartbeat into both NNs w/o hardcoding in the config, which may be problematic for federation + HA. The roundtrip reduction "hack" is a feature that can be extended to any sasl mechanism that can initiate. The point you keep missing is +the client can't guess an auth method+ but you keep focusing on retaining that behavior. We need to resolve this with the offline call today. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689451#comment-13689451 ] Luke Lu commented on HADOOP-9421: - Specifically: bq. Re-introduces the roundtrip I removed for tokens and usable by other auths in the future Not true. The initiate proto wrapped in rpc header is part of the connection header packet even though they're logically separate messages. bq. Appears to add yet another roundtrip for non-token auths Not true. See above. bq. Completely removes the ability for the client to chose the best or most preferred auth Not true. In fact the client initiate proto allows future auths without introduce new round-trip. bq. Ruins pluggable auths because the client now requires specific logic to "guess" if it can do the new auth Not true. initiate "allows" but not "requires" client specific logic, hence more extensible. bq. Prevents elimination of token use_ip Not applicable to any known token mechs: Digest-MD5 or SCRAM, as the former is always server initiated and the latter doesn't care. bq. Prevents supporting tokens for multi-interface, multi-A record, or CNAMEs Not true. Token auth don't care, see above. And the initiate proto is extensible for all kinds of auth metadata. bq. Breaks my ability to add IP failover support Not true. IP failover works with tokens as is and for Kerberos if server principal is shared among the servers for the same logical server. Can be extended to support insane cross server principal failover, while maintaining minimum round-trips in normal cases. bq. Allows clients to do complete DOS attacks by tying up the socket indefinitely with initiates Clients can already do the same by keeping RPC connections indefinitely. DoS is only significant if it requires client less resource to DoS a server, which is not the case. In summary, your patch changes the major flow of the current RPC with a new negotiate round-trip except for a round-trip reduction hack for Digest-MD5 tokens, since it disallows client to send any new auth metadata in the first packet. My patch is actually a (conceptually) small change to extend the capability to send arbitrary auth metadata in the first packet and allows server to intelligently respond with either challenge or negotiate, which allows round-trip optimization for all future auths besides Digest-MD5 tokens. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689336#comment-13689336 ] Luke Lu commented on HADOOP-9421: - bq. I don't understand the advantage of this patch. At a minimum, here are major problems. I think you misunderstood the patch and the current flow as well. In the ["Current flow"|https://issues.apache.org/jira/browse/HADOOP-9421?focusedCommentId=13688055&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13688055], step 1 and 2 is in the same C->S packet. My patch is an improvement over yours in that challenge or negotiation is not done blindly. It can support SCRAM/Kerberos round-trip reduction with no protocol branching while yours cannot. Conceptually I just replace the step 2 (still part of the connection header packet) with wrapped RpcSaslProto to allow future extensibility. So my patch feature wise is a superset of yours. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689283#comment-13689283 ] Daryn Sharp commented on HADOOP-9421: - I don't understand the advantage of this patch. At a minimum, here are major problems. * Re-introduces the roundtrip I removed for tokens and usable by other auths in the future * Appears to add yet another roundtrip for non-token auths * Completely removes the ability for the client to chose the best or most preferred auth * Ruins pluggable auths because the client now requires specific logic to "guess" if it can do the new auth * Prevents elimination of token use_ip * Prevents supporting tokens for multi-interface, multi-A record, or CNAMEs * Breaks my ability to add IP failover support * Allows clients to do complete DOS attacks by tying up the socket indefinitely with initiates Given my correction of the misunderstandings of my prior patch, what are the disadvantages? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689208#comment-13689208 ] Daryn Sharp commented on HADOOP-9421: - bq. Only works with token auths that use digest-md5, will require major protocol change to optimize for SCRAM (modern digest-md5 replacement) or Kerberos and anything SASL mechanisms that hasInitialResponse. A "major protocol change" will not be required for other auths. The client is properly coded to handle the server providing an initial challenge for any auth, but the server currently only does it for tokens. When the server auths become extensible, additional initial challenges can be added w/o changing the client. Ie. It's forward compatible. I did not generate an initial challenge for kerberos because the SASL mechanism does not support it. An exception is thrown if you try. This is intended to be a minimal change to provide a base implementation for future work. I thought everybody would be satisfied by removal of an existing round trip to offset the negotiate response? I'll look at your modifications to the patch. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689130#comment-13689130
]
Hadoop QA commented on HADOOP-9421:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588814/HADOOP-9421.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2684//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2684//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689025#comment-13689025
]
Hadoop QA commented on HADOOP-9421:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588781/HADOOP-9421.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ipc.TestSaslRPC
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2682//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2682//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688991#comment-13688991 ] Luke Lu commented on HADOOP-9421: - bq. As described, optimize token path. [Your patch|https://issues.apache.org/jira/secure/attachment/12588738/HADOOP-9421.patch] still doesn't have the proper client initiation support: # Only works with token auths that use digest-md5, will require major protocol change to optimize for SCRAM (modern digest-md5 replacement) or Kerberos and anything SASL mechanisms that hasInitialResponse. # fallback prevention seems broken as the patch didn't modify TestSaslRpc#testSimpleServerWith*Token and still pass these tests. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688801#comment-13688801
]
Hadoop QA commented on HADOOP-9421:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588738/HADOOP-9421.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2681//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2681//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688668#comment-13688668 ] Daryn Sharp commented on HADOOP-9421: - It'll be up before I go home. Ran into up-merge snag where the connection header had an incompatible change that sent me on a wild goose chase. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688657#comment-13688657 ] Luke Lu commented on HADOOP-9421: - I think it's close. It needs to be rebased against trunk for atm's security fix. I'm also adding two unit tests to make sure fallback prevention actually works. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688622#comment-13688622 ] Arun C Murthy commented on HADOOP-9421: --- How much more work are we talking about here? Is the patch ready to go? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688579#comment-13688579 ] Luke Lu commented on HADOOP-9421: - bq. We do not have a way to test such a major change this late in the game for 2.0. Well, Daryn will be doing such tests on Y clusters. I believe that we should try to make 2.0 friendly for backward compatibility. bq. a backward compatible change is possible because of the version number in the header. Switching on version number means that the community will be supporting some old cruft for a long time both in client and server. Not sure it's a worthy trade off for a week or two schedule. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688576#comment-13688576 ] Daryn Sharp commented on HADOOP-9421: - bq. We do not have a way to test such a major change this late in the game for 2.0. As with a number of things security related, Y! and by extension me are the test. :) bq. Let's leave the SASL part unchanged for V9 (ie Hadoop 2.x) and do a backwards compatible change in trunk; a backwards compatible change is possible because of the version number in the header. We really need to discard the current limited implementation before compatibility is a must. Supporting two implementations will be a huge burden. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688564#comment-13688564
]
Daryn Sharp commented on HADOOP-9421:
-
SASL just defines the format of the bytes used by mechanisms. It's left as an
exercise to the reader as to how to negotiate and wire encode the
challenge/response sequence. Per the SASL RFC
http://tools.ietf.org/html/rfc4422#section-4
{noformat}
4. Protocol Requirements
In order for a protocol to offer SASL services, its specification
MUST supply the following information:
1) A service name, to be selected from registry of "service" elements
for the Generic Security Service Application Program Interface
(GSSAPI) host-based service name form, as described in Section 4.1
of [RFC2743]. Note that this registry is shared by all GSSAPI and
SASL mechanisms.
2) Detail any mechanism negotiation facility that the protocol
provides (see Section 3.2).
A protocol SHOULD specify a facility through which the client may
discover, both before initiation of the SASL exchange and after
installing security layers negotiated by the exchange, the names
of the SASL mechanisms that the server makes available to the
client. The latter is important to allow the client to detect
downgrade attacks. This facility is typically provided through
the protocol's extensions or capabilities discovery facility.
3) Definition of the messages necessary for authentication exchange,
including the following:
a) A message to initiate the authentication exchange (see Section
3.3).
This message MUST contain a field for carrying the name of the
mechanism selected by the client.
This message SHOULD contain an optional field for carrying an
initial response. If the message is defined with this field,
the specification MUST describe how messages with an empty
initial response are distinguished from messages with no
initial response. This field MUST be capable of carrying
arbitrary sequences of octets (including zero-length sequences
and sequences containing zero-valued octets).
b) Messages to transfer server challenges and client responses
(see Section 3.4).
Each of these messages MUST be capable of carrying arbitrary
sequences of octets (including zero-length sequences and
sequences containing zero-valued octets).
c) A message to indicate the outcome of the authentication
exchange (see Section 3.6).
This message SHOULD contain an optional field for carrying
additional data with a successful outcome. If the message is
defined with this field, the specification MUST describe how
messages with an empty additional data are distinguished from
messages with no additional data. This field MUST be capable
of carrying arbitrary sequences of octets (including zero-
length sequences and sequences containing zero-valued octets).
{noformat}
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688558#comment-13688558 ] Sanjay Radia commented on HADOOP-9421: -- We do not have a way to test such a major change this late in the game for 2.0. Let's leave the SASL part unchanged for V9 (ie Hadoop 2.x) and do a backwards compatible change in trunk; a backwards compatible change is possible because of the version number in the header. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688556#comment-13688556 ] Luke Lu commented on HADOOP-9421: - SASL itself doesn't negotiate. Currently we have hack via -88 length to switch to SIMPLE. The proposal got rid of the hack and replace it with negotiate proto which contains a list of mechs that the server supports. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688514#comment-13688514 ] Sanjay Radia commented on HADOOP-9421: -- bq. packet 2: C<-S (token-challenge|success-for-simple|negotiate) Since SASL does not have a "negotiate" response are you proposing that we encode this in a -ve length of the token (just as SWITCH_TO_SIMPLE is encoded as -88)? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688472#comment-13688472 ] Luke Lu commented on HADOOP-9421: - bq. Simply wrapping sasl in protobuf headers would be sufficient to address our near future needs I meant "would not" > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688470#comment-13688470 ] Luke Lu commented on HADOOP-9421: - bq. it appears that this proposal is not wrapping sasl in frames but replacing sasl with an alternate protocol. It does, but it's further extended to allow negotiation of sasl mechanisms. Simply wrapping sasl in protobuf headers would be sufficient to address our near future needs. I believe that my latest patch addresses both performance for token auth and flexibility for mech negotiation. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688464#comment-13688464 ] Sanjay Radia commented on HADOOP-9421: -- Daryn, to fully understand you need to describe what happens for simple-auth, token, kerberos. Do we want to retain the current protocol's property where simpl-auth is not done within SASL but outside SASL. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688459#comment-13688459 ] Sanjay Radia commented on HADOOP-9421: -- Daryn, it appears that this proposal is not wrapping sasl in frames but replacing sasl with an alternate protocol. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688449#comment-13688449 ] Luke Lu commented on HADOOP-9421: - Looks like I need to merge with atm's fall-back-to-simple option commit (without a JIRA). > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688441#comment-13688441 ] Luke Lu commented on HADOOP-9421: - New patch flow: packet 1: C->S connectionHeader + blind-initiate packet 2: C<-S (token-challenge|success-for-simple|negotiate) ... Note, I refactored general purpose protobuf RpcWrapper into ProtobufHelper as they don't belong in ProtobufEngine. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688438#comment-13688438
]
Hadoop QA commented on HADOOP-9421:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588682/HADOOP-9421.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2678//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688055#comment-13688055
]
Daryn Sharp commented on HADOOP-9421:
-
I think I may be able to make everyone happy, by actually reducing the current
message exchange for tokens.
Current flow:
{noformat}
1: C -> S connectionHeader
2: C -> S TOKEN sasl:null
3: C <- S TOKEN sasl:token-challenge
4: C -> S TOKEN sasl:token-response
...
{noformat}
Patch flow:
{noformat}
1 : C -> S connectionHeader
1.1: C <- S [TOKEN, KERBEROS] sasl:null
2 : C -> S TOKEN sasl:null
3 : C <- S TOKEN sasl:token-challenge
4 : C -> S TOKEN sasl:token-response
...
{noformat}
How about:
{noformat}
1 : C -> S connectionHeader
1.1: C <- S [TOKEN, KERBEROS] sasl:token-challenge
4 : C -> S TOKEN sasl:token-response
...
{noformat}
I'm testing to see if this will work.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688005#comment-13688005 ] Daryn Sharp commented on HADOOP-9421: - Maybe I'm misunderstanding, but won't the server's response to accept/counter the client's proposal introduce the same delay? It also introduces a complication whereby the client is still guessing if it has the required credentials for the auth methods, even though it may need information from the server to determine if it has the credentials. For instance, to remove use_ip and support multiple interfaces, the server will need to provide a uniqueId to the client to locate a token. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687645#comment-13687645 ] Sanjay Radia commented on HADOOP-9421: -- bq. Client today does send/(send/read)+, and now it's send/read/(send/read)+. Daryn, I assume the first read in the new version is to read the server's list of auth methods. Generally the preferred approach is: client sends preferred auth with a list of alternatives client can do and server either accepts it or throws back a counter with one of the alternates that the client proposed. This would avoid an extra trip in the normal case. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687643#comment-13687643 ] Sanjay Radia commented on HADOOP-9421: -- bq. Client today does send/(send/read)+, and now it's send/read/(send/read)+. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687375#comment-13687375 ] Arun C Murthy commented on HADOOP-9421: --- Is there any reason to not commit this and possibly fix performance issues later? Will it change the protocols incompatibly to improve perf? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687233#comment-13687233 ] Daryn Sharp commented on HADOOP-9421: - Per Sanjay's request, here's a quick summary: This patch does not direct address deficiencies in negotiation process. It merely provides the flexibility to fix them in the near future. The basic changes are: * SASL protobufs * SASL protobufs wrapped with RPC header * Server advertised SASL auth methods - currently ignored by client Wrapping SASL negotiation in a RPC header avoids decoding errors. Ex. A secure client currently tries to decode a RPC exception from an incompatible connection header as a SASL message. The client also can't handle a RPC exception mid-stream during SASL negotiation. The RPC header allows the server and client to correctly decode the payload. Using the RPC headers, with the addition of a streamId, will also allow the _future potential_ to multiplex multiple UGIs over the same connection. Services like the NN can be overwhelmed by a stampede of connections. The NM may be an ideal candidate for aggregation of connections, or even a per-node multiplexor for tasks. Server advertisement of auth methods aims to address the current limitation of allowing 1 pre-determined auth or simple. Additional consideration is needed for a client-side whitelist to avoid server unexpectedly requesting weak authentication. In the current patch, the client ignores the server and blindly attempts the negotiation as it does today. +Other future features unlocked by this design+ * Brings us much closer to pluggable auth methods w/o changing server and client code * Client may select a server advertised auth method to: *# Support services and/or clusters with heterogenous auth methods (ex. thinking of knox, rhino, etc) *# *Critical to Y!*: IP failover, especially for HA with kerberos. The client needs the active NN's host to acquire a service ticket *# Simplify token selection by using an opaque identifier supplied by the server - will eliminate use_ip, and even the complex HA token logic *# Support accessing multi-interface hosts on all interfaces *# Support accessing services via any of their hostnames, ips, or CNAMEs * Ability to greatly reduce complexity of client/server auth code, and cleanly decouple SASL logic * As mentioned before, multiplexing of different UGIs over a shared connection > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp >Priority: Blocker > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686234#comment-13686234 ] Sanjay Radia commented on HADOOP-9421: -- Daryn, given that this jira discussed wide range of features, can you please summerize that the patch is attempting to do (and of the things discussed, not do). > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685656#comment-13685656 ] Daryn Sharp commented on HADOOP-9421: - Depending on how you define round trip, it's just a 1/2 trip for a new response. Client today does send/(send/read)\+, and now it's send/*read*/(send/read)\+. First file status RPC time increased from 340ms to 348ms. The overall times seem a bit high, so I'm going to test on a third cluster. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683963#comment-13683963 ] Luke Lu commented on HADOOP-9421: - bq. Given today's network speeds, I'm not going to fret over a handful of bytes that greatly enhance the design of the authentication layer. It's not the bytes but a whole new network/rpc round trip that can be easily avoided. In WAN and/or when the network is busy the round trip latency can easily be in hundreds of ms. bq. That patch resulted in an average 1% penalty and a median 2% penalty to the first RPC call using kerberos authentication - which translates to less than 10ms. 10ms / 1% means the total latency is 1 second! Something is not right here. bq. I can likely erase it by removing the unnecessary connection context I doubt it, the unnecessary connection context is in the noise compared with a network/rpc round trip. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683896#comment-13683896 ] Daryn Sharp commented on HADOOP-9421: - Given today's network speeds, I'm not going to fret over a handful of bytes that greatly enhance the design of the authentication layer. I did extensive testing with a more advanced follow on patch that instantiates the SASL client upon receipt of the NEGOTIATE. That patch resulted in an average 1% penalty and a median 2% penalty to the first RPC call using kerberos authentication - which translates to less than 10ms. That held true while issuing concurrent calls up until the number of socket readers. Once the number of socket readers is exceeded, performance drops off so sharply that any penalty is lost in the statistical noise. Assuming the penalty is entirely due to the additional NEGOTIATE response, I can likely erase it by removing the unnecessary connection context. We're passing null for the SASL authz user when it could be the effective user. SIMPLE is the only "auth" that actually requires the context context, but that could be a special case (as it already is in many ways) or could be replaced with SASL PLAIN which simply sends the effective and real user separated by null bytes. Note this patch does have a minor issue with kerberos where the client is confused about the state of negotiation after it completes. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683823#comment-13683823 ] Luke Lu commented on HADOOP-9421: - bq. I also have measured that the NEGOTIATE header is ~61 bytes + length of hostname. On my test cluster, it's 92 bytes. You forgot to count the [protocol overhead|http://sd.wareonearth.com/~phil/net/overhead/], which is between 78 bytes to 114 bytes per packet for TCP over ethernet. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683488#comment-13683488 ] Daryn Sharp commented on HADOOP-9421: - As a followup, I'm trying to get some performance numbers on authentication time. I'm testing a second cluster to ensure the results are valid. I also have measured that the NEGOTIATE header is ~61 bytes + length of hostname. On my test cluster, it's 92 bytes. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682366#comment-13682366 ] Daryn Sharp commented on HADOOP-9421: - The changes in this jira fundamentally boils down to: * SASL protobufs * RPC header wrapped SASL protobufs * Server advertised SASL auth methods There's not much additional complexity involved. Using this model opens the door to a lot of features such as easier support of the alternate auth methods being proposed, IP failover, token support for multi-interface and multi-hostnames for servers, multi-plexing UGIs over a single connection, etc. Sanjay, I'll contact you offline to bring you up to speed on this lengthy jira. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681607#comment-13681607 ] Sanjay Radia commented on HADOOP-9421: -- bq. I'm only suggest that we simply send a length + RpcSaslProto after the connection header (analogous to the current authmethod byte but more extensible), which open up many opportunities to reduce auth round-trips in the future without future client baggage. To echo this. This jira started out as adding just the length to the reply, then Protobuf and now a lot more. We can only afford a simple change for hadoop 2.x GA. Anything more adds too many changes to go into 2.x since it will require extensive testing including performance testing. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681564#comment-13681564 ] Luke Lu commented on HADOOP-9421: - bq. At the moment, I'm only trying to make the minimal changes to the base wire protocol to allow future extensibility. The current mechanism is a major change that introduce an extra round-trip of server processing (more chances of timeouts) that can potentially break a busy cluster. I'm only suggest that we simply send a length + RpcSaslProto after the connection header (analogous to the current authmethod byte but more extensible), which open up many opportunities to reduce auth round-trips in the future without future client baggage. bq. All SASL mechanisms are challenge/response, not just tokens... PLAIN is not... bq. I think you are predicating the optimization on the assumption the client can always pre-determine the AuthMethod to use. Not always but 99% of the time for most common workload, where container/tasks access HDFS with delegation tokens, which is not gonna change no matter how many mechs we'll support in the future, which are mostly to bootstrap a delegation token or equivalent. bq. we're holding up the base SASL RPCv9 changes to discuss a pre-mature optimization to avoid sending <100 bytes, compared to token negotiation requiring at least 5X more bytes, and kerberos requiring at least 20X more bytes. The mandatory negotiate response is way more than 100 bytes and will grow as the number of mechs/protocols grows. The most important thing here is not bytes (as long as it fits into one packet) but an extra round trip for the most common challenge response mechanism that currently requires only one extra round trip (over simple). I guess that I don't understand what's so hard to include the extra (length + RpcSaslProto) in the protocol. You don't even need to change your existing logic: always send a negotiate for RPC v9. But that little extra proto makes future evolution of auth so much easier. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681360#comment-13681360 ] Daryn Sharp commented on HADOOP-9421: - bq. I don't think ID_TOKEN or SSO_TOKEN being appropriate in SaslRpcServer.AuthMethod. No worries, this will eventually become dynamic and not live in the rpc server class since we can't afford to change the code every time we add a new AuthMethod. At the moment, I'm only trying to make the minimal changes to the base wire protocol to allow future extensibility. bq. Challenge response is the main performance use case that's worth optimizing for. All SASL mechanisms are challenge/response, not just tokens... We can continue on another jira, but I think you are predicating the optimization on the assumption the client can always pre-determine the AuthMethod to use. That's one of the fundamental flaws with the current implementation. When there are more supported auths and more supported tokens, the client's initial connection has no way to know which to chose until the server tells it what it needs. We might be able to optimize a re-connect but it's going to further complicate and delay this jira. To put this all in perspective, we're holding up the base SASL RPCv9 changes to discuss a pre-mature optimization to avoid sending <100 bytes, compared to token negotiation requiring at least 5X more bytes, and kerberos requiring at least 20X more bytes. Both of these are dwarfed by the real activity over the connection which why I feel we should defer to another jira. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680842#comment-13680842 ] Luke Lu commented on HADOOP-9421: - bq. The abstract name appears in the stringified UGI, so it's not applicable to call it CHALLENGE_RESPONSE. It's still really a mechanism name (vs KERBEROS, PLAIN etc.) and I don't think ID_TOKEN or SSO_TOKEN being appropriate in SaslRpcServer.AuthMethod. But I'm not insisting. bq. If we upgrade the digest mechanism for tokens, that new mechanism may also have an initial response Challenge response is the main performance use case that's worth optimizing for. Digest-MD5 doesn't have initial response. The main (only?) potential replacement SCRAM doesn't use server name at all (ie, no digest-uri issue in HA cases). The code can be generic sasl exchange, we just take advantage of the fact the token based auth is automatically optimized if you send the client initiation with connection header. The server can simply return a negotiate response with server name if the server name from the initial response doesn't match for non-token mechanisms. bq. Initial response is true for other mechanisms like GSSAPI and PLAIN, which means kerberos has just been penalized No. For normal situation (where the client's assumed server name matches server's), I save a negotiation round trip. For fail over situation, server can simply return a negotiation with server name, so the client can reinitialize the saslclient with the correct servername and send the correct "initial" response, which is the same number of round trip as your normal case. bq. Backing up, I thought we agreed earlier to defer reconnect optimizations to a future jira? I can definitely compromise for clear trade-offs. But I'd like to make sure we both fully understand the implications/alternatives before moving on. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680770#comment-13680770 ] Daryn Sharp commented on HADOOP-9421: - bq. As long as the mech is TOKEN (or better CHALLENGE_RESPONSE), The authMethods are mapping an abstract name (ie. TOKEN) to the SASL mechanism. With the other security designs being discussed, we might have an ID_TOKEN or SSO_TOKEN. The abstract name appears in the stringified UGI, so it's not applicable to call it CHALLENGE_RESPONSE. bq. why can't you instantiate the sasl client then (after receiving server challenge) with the info from server challenge? Why would an additional roundtrip be necessary unless the mech is not supported by server? That's precisely the convoluted logic I outlined in steps 1-4. It's a lot of additional complexity to pre-maturely optimize just a reconnect, and introduces much more complexity. Ex. If the client's INITIATE is invalid, now the server can't return an fatal error and close the connection. It has to return NEGOTIATE. However a second bad INITIATE should return a fatal error. The client has to know to create a SASL client on the first CHALLENGE, but not subsequent CHALLENGES. Other details rapidly add up. As opposed to INITIATE means the client and server both instantiate their SASL objects at that time. bq. Trying to see why you're not seeing what I'm seeing: perhaps it's not obvious that SaslClient#hasInitialResponse is always false for new connection with token (Digest-MD5 at least, cf. rfc-2831)? Initial response is true for other mechanisms like GSSAPI and PLAIN, which means kerberos has just been penalized. If we upgrade the digest mechanism for tokens, that new mechanism may also have an initial response. We can't design this around an internal detail of one particular mechanism (DIGEST-MD5). Backing up, I thought we agreed earlier to defer reconnect optimizations to a future jira? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680067#comment-13680067 ] Luke Lu commented on HADOOP-9421: - Trying to see why you're not seeing what I'm seeing: perhaps it's not obvious that SaslClient#hasInitialResponse is always false for new connection with token (Digest-MD5 at least, cf. [rfc-2831|https://tools.ietf.org/html/draft-ietf-sasl-rfc2831bis-12#section-2.1])? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679985#comment-13679985 ] Luke Lu commented on HADOOP-9421: - bq. The client can't generate an initial SASL response token since it hasn't instantiated the SASL client - leading to an additional roundtrip for SASL As long as the mech is TOKEN (or better CHALLENGE_RESPONSE), why can't you instantiate the sasl client then (after receiving server challenge) with the info from server challenge? Why would an additional roundtrip be necessary unless the mech is not supported by server? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679966#comment-13679966 ] Daryn Sharp commented on HADOOP-9421: - I'm open to improvements, but I'm having a hard reconciling how to add this capability. The SASL client and server must be instantiated with the exact same mechanism/protocol/serverId or the SASL server throws an exception. The initial connect will require a way for the client to solicit the NEGOTIATE otherwise it has no idea what to do. This is what incurs a roundtrip. Here's what it would take for a reconnect: # The client sends an INITIATE using the cached SaslAuth, but w/o instantiating it's SASL client until it receives the CHALLENGE response # The client can't generate an initial SASL response token since it hasn't instantiated the SASL client - leading to an additional roundtrip for SASL # The server's first CHALLENGE response must set the SaslAuth protobuf field which it currently doesn't, but not a big deal # Client now instantiates SASL client based on SaslAuth in CHALLENGE and processes the token > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679908#comment-13679908 ] Luke Lu commented on HADOOP-9421: - bq. By saying you can live with it, is that a tacit +1 or do you want further changes to this patch? Based on your response, I'm not completely sure that you fully considered my proposal (which is more evolution friendly and only requires a small change to your patch). But I'm fine with a scapegoat for RPC v9 :) > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679870#comment-13679870 ] Luke Lu commented on HADOOP-9421: - bq. This optimizes a reconnect, but the common case of initial connect will now take an additional round trip penalty by sending a second message to request the NEGOTIATE. No, for common case, server can detect that client selected the correct mechanism (note RpcSaslProto is extensible to contain the appropriate metadata needed for server to verify) and respond with a normal sasl challenge (again extensible to contain server principal) instead of negotiate. So my proposal actually works with majority of failover case (the mechanism doesn't change besides server name etc.) with no additional round trip as well. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679842#comment-13679842 ] Daryn Sharp commented on HADOOP-9421: - I believe you are proposing the client always sends two messages upon connect: the connect header and a SASL INITIATE. This optimizes a reconnect, but the common case of initial connect will now take an additional round trip penalty by sending a second message to request the NEGOTIATE. To allow for IP failover, I guess that means the server response to an invalid INITIATE is a NEGOTIATE instead of returning authentication failed and closing the connection? Presumably the second bad INITIATE will continue to return auth failed? I'm unclear what the next step is for this patch. I'm not too happy about evolution via AuthProtocol either but it's the only way I can think of to avoid penalizing the initial connect with another roundtrip. By saying you can live with it, is that a tacit +1 or do you want further changes to this patch? > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679738#comment-13679738 ] Luke Lu commented on HADOOP-9421: - bq. Shorting out the NEGOTIATE for a re-connect becomes a bit complicated. The connection header doesn't contain a length like other RPC packets You can always send the length + RpcSaslProto after the fixed connection header. Server can than send the appropriate challenge or negotiate accordingly. It seems more straight forward than the alternatives here. bq. the only way to signal the server is another authProtocol to not send a NEGOTIATE, It's not the only way (see above). But it could work, which is the saving grace of AuthProtocol :) though SASL2 or HSASL (pronounced as hassle) and its variant is kinda ugly. bq. The client needs the server's NEGOTIATE to correctly instantiate its SASL client. This negates the ability for the client to cache values for an immediate INITIATE. That's why it's called "cached" initiation, server can always send NEGOTIATE as it see fit after fail over. This is also why I prefer always sending RpcSaslProto first, so server can decide what to respond in a straight forward way. Fail over handling is not a common workload, the goal of client cached initiation is to reduce server side processing in common cases like container/task launching, when NN/RM are not failing over left and right. Anyway, though I'm not too happy with the evolution via AuthProtocol approach. I think I can live with it. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679663#comment-13679663 ] Daryn Sharp commented on HADOOP-9421: - Shorting out the NEGOTIATE for a re-connect becomes a bit complicated. The connection header doesn't contain a length like other RPC packets, so about the only way to signal the server is another authProtocol to not send a NEGOTIATE, but expect a subsequent INITIATE. As discussed earlier, I'd prefer to defer the reconnect optimization to a followup jira. BTW, I'm currently working on IP failover. We're blocked on HA deployment because we can't manage the logistics of updating confs and restarting every cluster & services like oozie and hdfsproxy, etc when one cluster is HA enabled or its HA config changes. IP failover is the answer but that's blocked too because the service principal changes when failover occurs. The client needs the server's NEGOTIATE to correctly instantiate its SASL client. This negates the ability for the client to cache values for an immediate INITIATE. The IP failover work is based upon this jira. bq. Though DIGEST is not exactly a precise word here, TOKEN is, IMO, even more nebulous. How about CHALLENGE_RESPONSE or simply CR? I'd prefer the rename to be in separate JIRA as well, as it doesn't really affect the wire protocol. Actually, it does affect the wire because that is a value passed over the wire. Ie. "TOKEN" via "DIGEST-MD5", or "TOKEN" via "SCRAM", etc. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679107#comment-13679107 ] Luke Lu commented on HADOOP-9421: - bq. As-is, it would be possible for the client to blindly send the connection header & INITIATE and just ignore the NEGOTIATE response. I'm concerned about the client code needed to maintain backward-compatibility. If client is enhanced to INITIATE first, client will need to always ignore the first NEGOTIATE and have to handle the second NEGOTIATE, which makes the client logic confusing and you cannot make the first dumb NEGOTIATE (which is size of O(number of mechs/protocols)) go away, it'll always be part of the protocol. The code is also replicated per client per language, which is a higher cost than just maintaining it at the server side. If client always sends a INITIATE, which could be empty, it is IMO easier to understand and evolve: server can handle the INITIATE and there will not be any wasted NEGOTIATE. In the era of YARN, where NN is resource strapped to handle the flood of requests from containers/tasks, I think it'd be worthwhile to leave room for optimization to reduce the amount of extra processing at the server side. The current approach will always have an extra NEGOTIATE that cannot be optimized away. Also, I noticed that you deprecated DIGEST and added TOKEN as an auth method (essentially renamed DIGEST to TOKEN) in SaslRpcServer. Though DIGEST is not exactly a precise word here, TOKEN is, IMO, even more nebulous. How about CHALLENGE_RESPONSE or simply CR? I'd prefer the rename to be in separate JIRA as well, as it doesn't really affect the wire protocol. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421.patch, HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678404#comment-13678404
]
Hadoop QA commented on HADOOP-9421:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12586780/HADOOP-9421.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2622//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2622//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421.patch, HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678356#comment-13678356 ] Daryn Sharp commented on HADOOP-9421: - bq. The main thing I'd like to have is be able to evolve client initiation (to save a round-trip with cached mechs) without having to change the RPC version again. Yes, and I haven't forgotten about that but planned to address in the future. As-is, it would be possible for the client to blindly send the connection header & INITIATE and just ignore the NEGOTIATE response. I need to consider how to properly handle IP failover when the cached info is stale. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678288#comment-13678288 ] Luke Lu commented on HADOOP-9421: - bq. The IpSerializationType change is orthogonal to this patch, so I'd like to defer to another jira if that's ok? Yes. HADOOP-9630 bq. I changed the authMethod in the connection header to specify the authentication protocol - in this case none, or sasl, to allow for future protocols. I think that's what you wanted? I wanted to get rid of the byte completely and let client always do a SASL call with RpcSaslProto with an initial auth method field, so that the client initiation is extendable. If you really want to use the byte for auth protocol NONE vs SASL (and potential other protocols), I'm fine (but change AuthCallId in the javadoc to AuthProtocol to be consistent). The main thing I'd like to have is be able to evolve client initiation (to save a round-trip with cached mechs) without having to change the RPC version again. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678045#comment-13678045 ] Daryn Sharp commented on HADOOP-9421: - Findbugs is grumping about a switch case not breaking. It's intentional so I'll update the ignore list. I don't know why findbugs didn't flag it in prior patches... > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13677706#comment-13677706
]
Hadoop QA commented on HADOOP-9421:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12586612/HADOOP-9421.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2616//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2616//console
This message is automatically generated.
> Convert SASL to use ProtoBuf and add lengths for non-blocking processing
>
>
> Key: HADOOP-9421
> URL: https://issues.apache.org/jira/browse/HADOOP-9421
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Sanjay Radia
>Assignee: Daryn Sharp
> Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch,
> HADOOP-9421-v2-demo.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13675923#comment-13675923 ] Daryn Sharp commented on HADOOP-9421: - Thanks Luke. Removing the AuthMethod proved to be a bit difficult. I was "cheating" by still using it to minimize the patch by avoiding changes to the pre-instantiation of the SASL client/server with disregard to the new handshake. I've had to do some of the changes I intended to do in a followup jira. That said, I'm almost done and should have a patch ready as early as this afternoon. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671898#comment-13671898 ] Luke Lu commented on HADOOP-9421: - Had a good call with Daryn, Thanks! I think I understand the (likely) future need to be able to multiplexing sasl sessions over a single connection. I think the basic approach is reasonable. I was a little thrown off by the wrapper changes in ProtobufRpcEngine. I think we're close enough protocol wise after the following fixes: # remove the AuthMethod byte and replace it with the sasl call with RpcSaslProto to initiate the exchange, so that client initiation is extensible. # remove the IpSerializationType byte. It's a useless historical vestige that's likely to be confused with RpcKind. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669723#comment-13669723 ] Luke Lu commented on HADOOP-9421: - bq. I still think there's confusion here. I'm not proposing per-call SASL I know what you're trying to do. You're overloading the protobuf rpc headers (a specific rpc impl detail) for sasl exchange, which I think is a confusing idea that prohibit clean separation of session/auth and rpc, which is required for multiple rpc engine support. bq. The session idea is interesting, but it closes the door on the server requiring multiple mechanisms. No it doesn't close the door. You can have arbitrary sasl mechanisms to establish an session. bq. If I have a process servicing multiple UGIs, I doesn't necessarily want that process to have hdfs "root" privileges I think you're confusing proxy user with hdfs "root". Proxy user has well defined mechanisms (ACLs) to restrict what a particular user can impersonate, which is already audit logged. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669681#comment-13669681 ] Daryn Sharp commented on HADOOP-9421: - I still think there's confusion here. I'm not proposing per-call SASL. The SASL negotiation sequence is wrapped in a fake callId in response to the connection header. It will simplify the client & server and open future possibilities. The session idea is interesting, but it closes the door on the server requiring multiple mechanism - ex. with all the other auth suggestions, maybe if the server auths a token, it will want to demand auth of an identity token or similar. I'm not sure using proxy/super-user authentication is desirable for multiplexing. If I have a process servicing multiple UGIs, I doesn't necessarily want that process to have hdfs "root" privileges. I want each multiplexed connection to use its specific token. For multiplexing to be feasible and remain async, the server must be able to determine what each received packet is - rpc call or sasl auth call. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669511#comment-13669511 ] Luke Lu commented on HADOOP-9421: - bq. This is our last shot for a long time to make RPC changes, so we should make it as flexible as possible. Agreed. But your proposal/patch actually makes it less flexible by making SASL depending on RPC engine implementation details. bq. SASL should ideally be a detail of RPC calls instead of only an initial connection detail (important for following points) My latest proposal decoupled the session concept from the underlying connection. By making the session explicit, you can have a more powerful and less leaky abstraction to handle all your use cases in a cleaner fashion. bq. Server should be able to issue SASL challenges mid-stream, perhaps after a connection has been open for an extended time, or to allow restricted super-user operations like chown This can be easily supported by simply close the session (with the connection remain open), and restart the session. bq. Out of band SASL auth (not distinguishable from actual RPC calls) precludes multiplexing multiple UGIs over a single connection - a concept y! has internally contemplated. SASL exchange could be much more expensive than typical RPC calls. Multiplexing multiple UGI over a single connection doesn't require per call SASL, which is an incredibly inefficient way of achieving the goal. A reasonable way to do it would be opening a proxy user session and add an optional user field per call to allow efficient multiple secure impersonation per call over the same session. Again the session concept is general enough to support per call SASL as well by optionally close the session per call. The session concept gives you the flexibility of amortizing the cost of SASL over multiple RPCs. bq. If different serializations are ever supported, we can change SASL at that time. This one of the major flaws of your proposal. We want to be able to support multiple RPC engines at the same time in the future to allow parallel development/deployment of RPC engines. We don't want to implement SASL again for each RPC engine. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669409#comment-13669409 ] Daryn Sharp commented on HADOOP-9421: - Of note, none of the benefits I mentioned is implemented. The negotiation isn't per-call. The current patch immediately initiates negotiation after connection header, and does it through the server's existing fake -33 call. > Convert SASL to use ProtoBuf and add lengths for non-blocking processing > > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Daryn Sharp > Attachments: HADOOP-9421.patch, HADOOP-9421.patch, > HADOOP-9421-v2-demo.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
