I don't know the answer to the question off the top of my head. Tracking the source code, it looks like the data transfer encryption does not really depend on Kerberos.
That said, (1) the Hadoop data transfer encryption relies on the data encryption key distributed by the NameNode. If a client can't validate authenticity of the NameNode, it may not make much sense to encrypt. (2) (With my Cloudera's hat on) If you use CDH, CM warns you that encryption is not effective if kerberos is not on, and which means this configuration is unsupported by Cloudera. You can use packet sniffer to validate the encryption. On Tue, Feb 4, 2020 at 3:44 PM Daniel Howard <danny...@toldme.com> wrote: > Hello, > > My scenario is running Hadoop in an environment without multiple users in > a secure datacenter. Nevertheless, we prefer to have encrypted data > transfers for activity between nodes. We have determined that we do not > need to set up Kerberos, so I am working through getting encryption going > on block data transfer and web services. > > I appear to have DFS encryption enabled thanks to the following settings > in *hdfs-site.xml*: > <!-- SECURITY --> > <property> > <name>dfs.encrypt.data.transfer</name> > <value>true</value> > </property> > <property> > <name>dfs.block.access.token.enable</name> > <value>true</value> > </property> > > Indeed, I was getting handshake errors on the datanodes with > dfs.encrypt.data.transfer enabled until I also set > dfs.block.access.token.enable. > > Filesystem operations work great now, but I still see plenty of this: > > 2020-02-04 15:25:59,492 INFO sasl.SaslDataTransferClient: SASL encryption > trust check: localHostTrusted = false, remoteHostTrusted = false > 2020-02-04 15:25:59,862 INFO sasl.SaslDataTransferClient: SASL encryption > trust check: localHostTrusted = false, remoteHostTrusted = false > 2020-02-04 15:26:00,054 INFO sasl.SaslDataTransferClient: SASL encryption > trust check: localHostTrusted = false, remoteHostTrusted = false > > I reckon that SASL is a Kerberos feature that I shouldn't ever expect to > see reported as true. Does that sound right? > > Is there a way to verify that DFS is encrypting data between nodes? (I > could get a sniffer out...) > > Thanks, > -danny > > -- > http://dannyman.toldme.com >
