[
https://issues.apache.org/jira/browse/HDFS-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoyu Yao updated HDFS-10643:
------------------------------
Attachment: HDFS-10643.05.patch
[~xiaochen], thanks for the review. This reason for not repro with the unit
test is not HDFS-9405. There are 150 key
(hadoop.security.kms.client.encrypted.key.cache.size*hadoop.security.kms.client.encrypted.key.cache.low-watermark
= 500*0.3=150) getting pre-created upon encryption zone creation. Change the
unit test to hadoop.security.kms.client.encrypted.key.cache.size = 4 and
hadoop.security.kms.client.encrypted.key.cache.low-watermark=0.5 so that the
refill of EDEK cache happen upon the 3rd file creation.
Update the unit test based on that and now we can repro the original issue
without the code fix in the patch with the exception stack below.
{code}
Caused by: org.apache.hadoop.security.authorize.AuthorizationException: User:
hdfs/localh...@example.com is not allowed to impersonate oozie_user
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at
org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:616)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:574)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.access$200(KMSClientProvider.java:91)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider$EncryptedQueueRefiller.fillQueueForKey(KMSClientProvider.java:146)
at
org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:299)
{code}
Also update the unit test to remove the try/catch as suggested. Please review
and let me know your thoughts. Thanks!
> HDFS namenode should always use service user (hdfs) to generateEncryptedKey
> ---------------------------------------------------------------------------
>
> Key: HDFS-10643
> URL: https://issues.apache.org/jira/browse/HDFS-10643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: encryption, namenode
> Affects Versions: 2.6.0
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Attachments: HDFS-10643.00.patch, HDFS-10643.01.patch,
> HDFS-10643.02.patch, HDFS-10643.03.patch, HDFS-10643.04.patch,
> HDFS-10643.05.patch
>
>
> KMSClientProvider is designed to be shared by different KMS clients. When
> HDFS Namenode as KMS client talks to KMS to generateEncryptedKey for new file
> creation from proxy user (hive, oozie), the proxyuser handling for
> KMSClientProvider in this case is unnecessary, which cause 1) an extra proxy
> user configuration allowing hdfs user to proxy its clients and 2) KMS acls to
> allow non-hdfs user for GENERATE_EEK operation.
> This ticket is opened to always use HDFS namenode login user (hdfs) when
> talking to KMS to generateEncryptedKey for new file creation. This way, we
> have a more secure KMS based HDFS encryption (we can set kms-acls to allow
> only hdfs user for GENERATE_EEK) with less configuration hassle for KMS to
> allow hdfs to proxy other users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
