?Not sure about HDFS special setup, but in general, to use HTTPs, you should
have your keystore/truststore generated and config ssl-client.xml and
ssl-server.xml properly.
- Zhijie
________________________________
From: Colin Ma <colin...@gmail.com>
Sent: Friday, July 03, 2015 1:37 AM
To: user@hadoop.apache.org
Subject: Problem when configure the security in hadoop
Hi,
I do the security configuration for Hadoop these days, the Kerberos
works fine, but there maybe has some problems on sasl configuration.
The following is the related configuration in hdfs-site.xml:
<property>
<name>dfs.http.policy</name>
<value>HTTPS_ONLY</value>
</property>
<property>
<name>dfs.data.transfer.protection</name>
<value>authentication</value>
</property>
There is no problem to execute the command like: hdfs dfs -ls /
But when I execute the command: hdfs dfs -copyToLocal /temp/test.txt . The
following exception will be thrown:
015-07-03 14:02:54,715 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Added
bpid=BP-271423801-192.168.20.28-1423724265164 to blockPoolScannerMap, new size=1
2015-07-03 14:03:39,963 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
server-511:50010:DataXceiver error processing unknown operation src:
/192.168.20.28:58422<http://192.168.20.28:58422> dst:
/192.168.20.28:50010<http://192.168.20.28:50010>
java.io.EOFException: Premature EOF: no length prefix available
at
org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2203)
at
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiationCipherOptions(DataTransferSaslUtil.java:233)
at
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.doSaslHandshake(SaslDataTransferServer.java:369)
at
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.getSaslStreams(SaslDataTransferServer.java:297)
at
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.receive(SaslDataTransferServer.java:124)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:183)
at java.lang.Thread.run(Thread.java:745)
2015-07-03 15:34:39,917 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Sent 1 blockreports 145 blocks total. Took 1 msec to generate and 6 msecs for
RPC and NN processing. Got back commands
org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@1b3bce82<mailto:org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@1b3bce82>
Just take a look the method doSaslHandshake() of
SaslDataTransferClient.java and SaslDataTransferServer.java, maybe
SaslDataTransferClient send a empty response cause this exception, and I think
some mistakes in the configuration caused this problem.
Is there anyone can help to check this problem?
Thanks for your help.
Best regards,
Colin Ma
