[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735860#comment-13735860 ] Hudson commented on HADOOP-9820: FAILURE: Integrated in Hadoop-Hdfs-trunk #1487 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1487/]) HADOOP-9820. RPCv9 wire protocol is insufficient to support multiplexing. Contributed by Daryn Sharp. (jitendra: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1512091) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Fix For: 3.0.0, 2.1.0-beta, 2.3.0, 2.1.1-beta Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734663#comment-13734663 ] Hudson commented on HADOOP-9820: SUCCESS: Integrated in Hadoop-Yarn-trunk #296 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/296/]) HADOOP-9820. RPCv9 wire protocol is insufficient to support multiplexing. Contributed by Daryn Sharp. (jitendra: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1512091) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Fix For: 3.0.0, 2.1.0-beta, 2.3.0, 2.1.1-beta Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734730#comment-13734730 ] Hudson commented on HADOOP-9820: ABORTED: Integrated in Hadoop-Hdfs-trunk #1486 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1486/]) HADOOP-9820. RPCv9 wire protocol is insufficient to support multiplexing. Contributed by Daryn Sharp. (jitendra: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1512091) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Fix For: 3.0.0, 2.1.0-beta, 2.3.0, 2.1.1-beta Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734833#comment-13734833 ] Hudson commented on HADOOP-9820: FAILURE: Integrated in Hadoop-Mapreduce-trunk #1513 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1513/]) HADOOP-9820. RPCv9 wire protocol is insufficient to support multiplexing. Contributed by Daryn Sharp. (jitendra: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1512091) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Fix For: 3.0.0, 2.1.0-beta, 2.3.0, 2.1.1-beta Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[
https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733644#comment-13733644
]
Hadoop QA commented on HADOOP-9820:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12596861/HADOOP-9820.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2945//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2945//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2945//console
This message is automatically generated.
RPCv9 wire protocol is insufficient to support multiplexing
---
Key: HADOOP-9820
URL: https://issues.apache.org/jira/browse/HADOOP-9820
Project: Hadoop Common
Issue Type: Bug
Components: ipc, security
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9820.patch, HADOOP-9820.patch
RPCv9 is intended to allow future support of multiplexing. This requires all
wire messages to be tagged with a RPC header so a demux can decode and route
the messages accordingly.
RPC ping packets and SASL QOP wrapped data is known to not be tagged with a
header.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733685#comment-13733685 ] Daryn Sharp commented on HADOOP-9820: - Findbugs still not caused by this patch. I'm supposed to be on vacation today, so please commit if the patch is acceptable. Thanks! RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734052#comment-13734052 ] Jitendra Nath Pandey commented on HADOOP-9820: -- +1. looks good to me. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734064#comment-13734064 ] Sanjay Radia commented on HADOOP-9820: -- +1 with a minor nit in java comment. // decode message if it's SASL wrapped should be // Must be SASL wrapped, verify and decode. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734179#comment-13734179 ] Hudson commented on HADOOP-9820: SUCCESS: Integrated in Hadoop-trunk-Commit #4231 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4231/]) HADOOP-9820. RPCv9 wire protocol is insufficient to support multiplexing. Contributed by Daryn Sharp. (jitendra: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1512091) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/RpcHeader.proto RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734307#comment-13734307 ] Jitendra Nath Pandey commented on HADOOP-9820: -- I have addressed Sanjay's nit and committed this to trunk, branch-2, branch-2.1-beta and branch-2.1.0-beta. Thanks to Daryn. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch, HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732103#comment-13732103 ] Daryn Sharp commented on HADOOP-9820: - Currently, yes, all RPC packets are wrapped and processed on the other side after unwrapping. Therein lies the possible problem with sending control messages. Let's say we do throw an exception as suggested. If the server cannot unwrap the SASL data, the client unlikely to be able to unwrap. The server would need to send a non-wrapped exception response to notify the client that wrapping isn't working, which this patch allows. Perhaps a better approach is to only allow out-of-band RPC (negative callIds) to be sent unwrapped over a wrapped stream? RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732156#comment-13732156 ] Sanjay Radia commented on HADOOP-9820: -- bq. Let's say we do throw an exception as suggested. If the server cannot unwrap the SASL data, ... Note the exception is not being thrown because the server cannot unwrap, the exception is being thrown because *currently* the only header that is acceptable when wrapping is enabled is the RPC-header-callId=sasl *with* the SASL-state=wrapped header. If you don't get that then throw the exception (which will go with its own response header). Later when we add multiplexing we will allow RPC-header-callId=sasl *with* start new rpc-stream and here comes its its SASL-authenticate exchange. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732137#comment-13732137 ] Sanjay Radia commented on HADOOP-9820: -- The RPC header and the SASL header after the it (but before the wrapped data) are not wrapped. The wrapped reply also has unwrapped headers (RPC and SASL). So the exception (if say the RPC header or the SASL is incorrect) will pass through fine. Indeed that is the beauty of the headers to the wrapped data - it does allow throwing an exception at the outer layer. The only problem is that if there is an exception at the RPC layer (above the wrapped layer) then the client has to be able to unwrap in order to read the exception. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732175#comment-13732175 ] Luke Lu commented on HADOOP-9820: - IMO, when the stream is being wrapped, no unwrapped exception should be thrown across RPC for the stream, as it's a breach of confidentiality. Server should log such exceptions. Minor nits: # SaslRpcClient.SaslRpc*Stream should be named SaslRpcClient.Wrapped*Stream. # The default stream buffer size should be configurable instead of hard coded 64*1024. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732752#comment-13732752 ] Luke Lu commented on HADOOP-9820: - bq. Client and server are using mismatched ciphers. That should not happen after the SASL negotiation is done. Given that even timing difference can leak information, we should not even tell a potentially adversarial client the fact that unwrap failed. We should log the exception at the server side for debugging purpose and close the connection after waiting for a random interval. bq. That's the spec default if the buffer size isn't negotiated so it can't be a configurable option. It needs to be a constant (with a pointer to the rfc) instead of literals for future maintenance. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[
https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732965#comment-13732965
]
Sanjay Radia commented on HADOOP-9820:
--
bq. The more specific cases I had in mind: ... Server wants to send a
non-sensitive control messages like is session alive or close session.
Requiring non-sensitive messages to be wrapped/unwrapped seems overkill.
I am in agreement with you here. But I was never proposing that we need to wrap
such stuff in the future. Since you are responding to an issue I never raised,
perhaps you are misreading my concern.
All I am just saying: in SaslRpcClient#SaslRpcInputStream.readNextRpcPacket
line 569:
{code}
if (headerBuilder.getCallId() != AuthProtocol.SASL.callId) {...
throw an exception, perhaps close the connection with fatal exception
{code}
In the future when we have out-of-band messages we can enumerate the ones that
are allowed.
RPCv9 wire protocol is insufficient to support multiplexing
---
Key: HADOOP-9820
URL: https://issues.apache.org/jira/browse/HADOOP-9820
Project: Hadoop Common
Issue Type: Bug
Components: ipc, security
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9820.patch
RPCv9 is intended to allow future support of multiplexing. This requires all
wire messages to be tagged with a RPC header so a demux can decode and route
the messages accordingly.
RPC ping packets and SASL QOP wrapped data is known to not be tagged with a
header.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732968#comment-13732968 ] Sanjay Radia commented on HADOOP-9820: -- +1 modulo my comment on the exception and my comment on the javadoc. I like Luke's nit. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[
https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730769#comment-13730769
]
Daryn Sharp commented on HADOOP-9820:
-
Findbugs flagged two bad practices completely unrelated to this patch:
{{Class org.apache.hadoop.metrics2.lib.DefaultMetricsSystem defines
non-transient non-serializable instance field mBeanNames}}
RPCv9 wire protocol is insufficient to support multiplexing
---
Key: HADOOP-9820
URL: https://issues.apache.org/jira/browse/HADOOP-9820
Project: Hadoop Common
Issue Type: Bug
Components: ipc, security
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9820.patch
RPCv9 is intended to allow future support of multiplexing. This requires all
wire messages to be tagged with a RPC header so a demux can decode and route
the messages accordingly.
RPC ping packets and SASL QOP wrapped data is known to not be tagged with a
header.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730917#comment-13730917 ] Daryn Sharp commented on HADOOP-9820: - Summary of offline messages to Sanjay who is reviewing: # Sasl wrapped message are sent in RPC/SASL protobufs like other SASL messages # Code shifted in Server to decode SASL wrapped packets in the code path invoked by processRpcOutOfBandRequest which handles other SASL packets # Client's SaslInputStream replacement unwraps SASL wrapped messages, leaves others alone # Client's SaslOutputStream replacement adds the RPC header because existing one does length/encrypted-payload only # Replacement SaslOutputStream correctly uses a buffered stream of the SASL negotiated size for wrapping. Existing SaslOutputStream impl was wrong but accidentally worked because of a smaller buffered stream atop it. # Slight optimization that Client isn't unnecessarily given sasl streams (that are no-ops) when wrapping isn't being done # Per comments, would be cleaner to decode all RPC packets in Client and route SASL messages to SaslRpcClient, but decoding is currently split across Client/SaslRpcClient. SaslRpcClient handles RPC decoding during authentication, but then Client decodes the rest of the stream with no knowledge of SASL. In the future, Client should decode all RPC packets and route SASL to SaslRpcClient. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[
https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731144#comment-13731144
]
Sanjay Radia commented on HADOOP-9820:
--
In SaslRpcClient#SaslRpcInputStream.readNextRpcPacket line 569: if
(headerBuilder.getCallId() == AuthProtocol.SASL.callId) {...
Since SaslRpcInputStream is only used when sasl-wrapped, shouldn't it throw an
exception if the callId is not SASL.callId?
RPCv9 wire protocol is insufficient to support multiplexing
---
Key: HADOOP-9820
URL: https://issues.apache.org/jira/browse/HADOOP-9820
Project: Hadoop Common
Issue Type: Bug
Components: ipc, security
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9820.patch
RPCv9 is intended to allow future support of multiplexing. This requires all
wire messages to be tagged with a RPC header so a demux can decode and route
the messages accordingly.
RPC ping packets and SASL QOP wrapped data is known to not be tagged with a
header.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731150#comment-13731150 ] Sanjay Radia commented on HADOOP-9820: -- You have optimized as per item 6 on your comment. Hence the javadoc for getInputStream and getOutputStream are incorrect. It should say something like Get SASL wrapped xxxputStreeam if it is sasl wrapped otherwise return original stream. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731234#comment-13731234 ] Daryn Sharp commented on HADOOP-9820: - bq. Since SaslRpcInputStream is only used when sasl-wrapped, shouldn't it throw an exception if the callId is not SASL.callId? I did consider if an exception should be thrown. However, it would preclude the server sending any control messages to a given session. Non-SASL messages might be something like a server sent ping to see if the client session is still alive. Or maybe to forcibly close the session, etc. I erred on the side of future flexibility. Thoughts? RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731254#comment-13731254 ] Sanjay Radia commented on HADOOP-9820: -- bq. I did consider if an exception should be thrown. However, it would preclude the server sending any control messages to a given session. If that is the case then we should enumerate the messages explicitly in the code. However, Non-sasl messages will have their own header and will be wrapped - they will be parsed by the next layer and the SASL layer will not see them. If you agree then at this stage throw the exception. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730042#comment-13730042 ] Sanjay Radia commented on HADOOP-9820: -- Actually one could argue that ping is part of the connection not the session. However your fix in Hadoop-9832 is an improvement over using a length of -1. So I will +1 the proposal in Hadoop-9832 and we can discuss whether the ping should be per-session or per-connection when we add multiplexed sessions to rpc. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730191#comment-13730191 ] Daryn Sharp commented on HADOOP-9820: - No tests are included because expanded test suites in prior jiras provide code coverage. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker Attachments: HADOOP-9820.patch RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[
https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730225#comment-13730225
]
Hadoop QA commented on HADOOP-9820:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12596247/HADOOP-9820.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2929//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2929//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2929//console
This message is automatically generated.
RPCv9 wire protocol is insufficient to support multiplexing
---
Key: HADOOP-9820
URL: https://issues.apache.org/jira/browse/HADOOP-9820
Project: Hadoop Common
Issue Type: Bug
Components: ipc, security
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9820.patch
RPCv9 is intended to allow future support of multiplexing. This requires all
wire messages to be tagged with a RPC header so a demux can decode and route
the messages accordingly.
RPC ping packets and SASL QOP wrapped data is known to not be tagged with a
header.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727747#comment-13727747 ] Arun C Murthy commented on HADOOP-9820: --- [~daryn] I'd really appreciate (and buy you a few cases) if you could get this in asap, thanks! RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727756#comment-13727756 ] Daryn Sharp commented on HADOOP-9820: - #1 priority. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9820) RPCv9 wire protocol is insufficient to support multiplexing
[ https://issues.apache.org/jira/browse/HADOOP-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727142#comment-13727142 ] Daryn Sharp commented on HADOOP-9820: - Sanjay and I spoke that this will be incompatible to fix in the future. RPCv9 wire protocol is insufficient to support multiplexing --- Key: HADOOP-9820 URL: https://issues.apache.org/jira/browse/HADOOP-9820 Project: Hadoop Common Issue Type: Bug Components: ipc, security Affects Versions: 3.0.0, 2.1.0-beta Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Blocker RPCv9 is intended to allow future support of multiplexing. This requires all wire messages to be tagged with a RPC header so a demux can decode and route the messages accordingly. RPC ping packets and SASL QOP wrapped data is known to not be tagged with a header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
