[
https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612575#comment-16612575
]
Xiao Chen commented on HDFS-13697:
----------------------------------
Thanks for the update [~daryn].
As far as I can find, there were similar discussions initially on HADOOP-10771
/ HADOOP-10880. I agree it's a day0 issue that's rather fundamental in
authentication, and more difficult to fix. However, it seems to be a somewhat
separate problem than what we are tackling so far. The existing code seems to
have met requirement from all use cases so far, except the oozie one
[~zvenczel] reported here. I believe Zsolt is setting up a cluster with the
latest patch, to test the various cases to be sure. Assuming that goes well,
would you be ok with this jira focusing on the kms client cache issue, and
having another jira focusing on the hadoop-auth family improvements (which
presumably impacts kms, httpfs, and maybe oozie)?
I may not fully understand your intention, and the fix may turn out to be easy
enough to come in this together, please help elaborate if that's the case.
> DFSClient should instantiate and cache KMSClientProvider using UGI at
> creation time for consistent UGI handling
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-13697
> URL: https://issues.apache.org/jira/browse/HDFS-13697
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Zsolt Venczel
> Assignee: Zsolt Venczel
> Priority: Major
> Attachments: HDFS-13697.01.patch, HDFS-13697.02.patch,
> HDFS-13697.03.patch, HDFS-13697.04.patch, HDFS-13697.05.patch,
> HDFS-13697.06.patch, HDFS-13697.07.patch, HDFS-13697.08.patch,
> HDFS-13697.09.patch, HDFS-13697.10.patch, HDFS-13697.11.patch,
> HDFS-13697.12.patch, HDFS-13697.prelim.patch
>
>
> While calling KeyProviderCryptoExtension decryptEncryptedKey the call stack
> might not have doAs privileged execution call (in the DFSClient for example).
> This results in loosing the proxy user from UGI as UGI.getCurrentUser finds
> no AccessControllerContext and does a re-login for the login user only.
> This can cause the following for example: if we have set up the oozie user to
> be entitled to perform actions on behalf of example_user but oozie is
> forbidden to decrypt any EDEK (for security reasons), due to the above issue,
> example_user entitlements are lost from UGI and the following error is
> reported:
> {code}
> [0]
> SERVER[xxx] USER[example_user] GROUP[-] TOKEN[] APP[Test_EAR]
> JOB[0020905-180313191552532-oozie-oozi-W]
> ACTION[0020905-180313191552532-oozie-oozi-W@polling_dir_path] Error starting
> action [polling_dir_path]. ErrorType [ERROR], ErrorCode [FS014], Message
> [FS014: User [oozie] is not authorized to perform [DECRYPT_EEK] on key with
> ACL name [encrypted_key]!!]
> org.apache.oozie.action.ActionExecutorException: FS014: User [oozie] is not
> authorized to perform [DECRYPT_EEK] on key with ACL name [encrypted_key]!!
> at
> org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:463)
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:441)
> at
> org.apache.oozie.action.hadoop.FsActionExecutor.touchz(FsActionExecutor.java:523)
> at
> org.apache.oozie.action.hadoop.FsActionExecutor.doOperations(FsActionExecutor.java:199)
> at
> org.apache.oozie.action.hadoop.FsActionExecutor.start(FsActionExecutor.java:563)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:232)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
> at org.apache.oozie.command.XCommand.call(XCommand.java:286)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.security.authorize.AuthorizationException: User
> [oozie] is not authorized to perform [DECRYPT_EEK] on key with ACL name
> [encrypted_key]!!
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:607)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:565)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:832)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:209)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:205)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:205)
> at
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
> at
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1440)
> at
> org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1542)
> at
> org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1527)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:408)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:401)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:401)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:344)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:923)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:904)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:801)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:790)
> at
> org.apache.oozie.action.hadoop.FsActionExecutor.touchz(FsActionExecutor.java:519){code}
> The operation should have succeeded as [example_user] is the owner of the
> [encrypted_key]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
