Krishna Gundamaraju created HAWQ-1791:
-----------------------------------------
Summary: Kerberos HTTP SPNEGO Authentication not supported in
LibHdfs3
Key: HAWQ-1791
URL: https://issues.apache.org/jira/browse/HAWQ-1791
Project: Apache HAWQ
Issue Type: Bug
Components: libhdfs
Reporter: Krishna Gundamaraju
Assignee: Ruilong Huo
Hi, I have integrated the latest LibHdfs3 from the git hub repo at
[https://github.com/apache/hawq/tree/master/depends/libhdfs3] with my
application that is expected to work like a HDFS Client.
I have verified that the following use cases work in my test setup:
1) Transparent Data Encryption (TDE) works when I configure the Hadoop cluster
and KMS to use simple authentication. My application is able to both read files
under an encryption zone and create and write to new files under an encryption
zone. So all of the interactions between my application and the Hadoop KMS work
as expected.
2) Non TDE use cases with a kerberized Hadoop cluster work as well. My
application can successfully authenticate itself with the Hadoop cluster that
is configured to use Kerberos. It can read and write files from this Hadoop
cluster.
What doesn't work is when my application tries to read files under an
encryption zone from a Hadoop cluster that is configured to use Kerberos
authentication. I have created a HTTP service principal on the KDC and
generated a keytab and installed it on the Linux host where my application
runs. I have verified that using this keytab file my application is able to
successfully get a TGT from the KDC.
I stepped through the LibHdfs3 code and I see that the
KmsClientProvider::buildKmsUrl() function throws the following exception when
Authentication method is set to Kerberos.
** if (method == AuthMethod::KERBEROS) {
** // todo
** THROW(InvalidParameter, "KmsClientProvider : Not support kerberos
yet.");
** } else if (method == AuthMethod::SIMPLE) {
My question is about whether LibHdfs3 supports Kerberos HTTP SPNEGO
Authentication or not? If the answer is yes, then can you please help me in
debugging this issue by pointing me to any relevant literature/documentation or
by providing any other hints on what I could be missing? I can provide pcaps
that show the packets exchanged between my application and the Hadoop KMS and I
can also provide pcaps that show the packets exchanged between my application
and the Hadoop NameNode.
Thanks in advance
Krishna
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
