[
https://issues.apache.org/jira/browse/HDFS-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16803440#comment-16803440
]
Chen Liang commented on HDFS-13699:
-----------------------------------
Thanks for the review [~shv]! Updated v008 patch. For {code}Should use assert :
instead of{code} my impression has been that assert is used in tests but prod
code uses Perconditions (e.g. DFSClient code has many Perconditions checks).
So I'm still leaving it Perconditions in the v008 patch. Please let me know if
I'm wrong on this. All other comments are addressed.
Regarding upgrade, the only potentially incompatible change is the change of
datatransfer.proto with a new field added, which is the handshake secret. It is
added as an optional field, so serialization/deserialization does not break.
On Client side, client parses the token it got from NN, to create a
DataTransferEncryptorMessage to send to DN.
For new client + old NN:
The patch already has logic that if secret is missing from the token, client
just sends the message without the secret.
For old client + new NN:
The old client will not include the secret from NN and sends the message
without secret.
Either way, DN receives a token without the secret so even the new DN stays the
same way as of today.
On DN side, with the patch, DN parses the token to get the secret, decrypts the
secret to get new QOP and overwrites to use this QOP when establishing
connection with client. DN can also be optionally configured to overwrite QOP
for downstream DNs:
For new DN not getting the secret:
The patch already has DN logic so that DN just proceeds without overwriting
QOP, which makes it behave the same way as of today.
For old DN receiving the secret:
The DN will simply not process the secret and proceeds as of today.
For new DN talking to downstream old DN:
The new DN may overwrite the secret with new QOP encrypted, but old DN will
ignore it and DN proceeds the same way.
For old DN talking to downstream new DN:
Again, new DN has the logic to handle the absence of the secret, so it behaves
the same way as old DN.
The only problematic scenarios I can think of now are when NN and DN have
different QOP settings, for example, NN configured to use encryption (privacy),
client also configured to use encryption (privacy), but DNs are configured not
to use encryption (auth). Then client can talk to NN but only the upgraded DNs
will accept client calls. Because the upgraded DNs will take the same QOP as NN
(encryption), while non-upgraded DNs will reject the calls because there is no
common QOP to agree when doing handshake. But such cases do not seem to be
valid use case to me, and can cause problem even without this feature...more
like misconfigurations on client side.
Still though, I will try to get some bandwidth to actually verify different
versions can work together.
> Add DFSClient sending handshake token to DataNode, and allow DataNode
> overwrite downstream QOP
> ----------------------------------------------------------------------------------------------
>
> Key: HDFS-13699
> URL: https://issues.apache.org/jira/browse/HDFS-13699
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Chen Liang
> Assignee: Chen Liang
> Priority: Major
> Attachments: HDFS-13699.001.patch, HDFS-13699.002.patch,
> HDFS-13699.003.patch, HDFS-13699.004.patch, HDFS-13699.005.patch,
> HDFS-13699.006.patch, HDFS-13699.007.patch, HDFS-13699.008.patch,
> HDFS-13699.WIP.001.patch
>
>
> Given the other Jiras under HDFS-13541, this Jira is to allow DFSClient to
> redirect the encrypt secret to DataNode. The encrypted message is the QOP
> that client and NameNode have used. DataNode decrypts the message and enforce
> the QOP for the client connection. Also, this Jira will also include
> overwriting downstream QOP, as mentioned in the HDFS-13541 design doc.
> Namely, this is to allow inter-DN QOP that is different from client-DN QOP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
