[
https://issues.apache.org/jira/browse/HBASE-18620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18620:
-----------------------------------
Fix Version/s: (was: 1.5.1)
3.0.0
Status: Open (was: Patch Available)
No progress, cancelling patch
> Secure bulkload job fails when HDFS umask has limited scope
> -----------------------------------------------------------
>
> Key: HBASE-18620
> URL: https://issues.apache.org/jira/browse/HBASE-18620
> Project: HBase
> Issue Type: Bug
> Components: security
> Reporter: Pankaj Kumar
> Assignee: Pankaj Kumar
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18620-branch-1-v2.patch,
> HBASE-18620-branch-1-v3.patch, HBASE-18620-branch-1.patch
>
>
> By default "hbase.fs.tmp.dir" parameter value is
> /user/$\{user.name}/hbase-staging.
> RegionServer creates the staging directory (hbase.bulkload.staging.dir,
> default value is hbase.fs.tmp.dir) during opening a region as below when
> SecureBulkLoadEndpoint configured in hbase.coprocessor.region.classes,
> {noformat}
> drwx------ - hbase hadoop 0 2017-08-12 13:55 /user/xyz
> drwx--x--x - hbase hadoop 0 2017-08-12 13:55 /user/xyz/hbase-staging
> drwx--x--x - hbase hadoop 0 2017-08-12 13:55
> /user/xyz/hbase-staging/DONOTERASE
> {noformat}
> Here,
> 1. RegionServer is started using "xyz" linux user.
> 2. HDFS umask (fs.permissions.umask-mode) has been set as 077, so file/dir
> permission will not be wider than 700. "/user/xyz" directory (doesn't exist
> earlier) permission will be 700 and "/user/xyz/hbase-staging" will be 711 as
> we are just setting permission of staging directory not the parent
> directories which are created (fs.mkdirs()) by RegionServer.
> Secure bulkload will fail as other user doesn't have EXECUTE permission on
> "/user/xyz" directory.
> *Steps to reproduce:*
> ==================
> 1. Configure org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> in "hbase.coprocessor.region.classes" at client side.
> 2. Login to machine as "root" linux user.
> 3. kinit to any kerberos user except RegionServer kerberos user (say admin).
> 4. ImportTSV will create the user temp directory (hbase.fs.tmp.dir) while
> writing partition file,
> {noformat}
> drwxrwxrwx - admin hadoop 0 2017-08-12 14:52 /user/root
> drwxrwxrwx - admin hadoop 0 2017-08-12 14:52 /user/root/hbase-staging
> {noformat}
> 4. During LoadIncrementalHFiles job,
> - a. prepareBulkLoad() step - Random dir will be created by RegionServer
> credentials,
> {noformat}
> drwxrwxrwx - hbase hadoop 0 2017-08-12 14:58
> /user/xyz/hbase-staging/hbase__t1__e67b23m2ghe6fkn1bqrb95ak41ferj8957cdhsep4ebmpohm22nvi54vh8g3qh1
> {noformat}
> - b. secureBulkLoadHFiles() step - Family dir existence check and creation is
> done by using client user credentials. Here client operation will fail as
> below,
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
> Permission denied: user=admin, access=EXECUTE,
> inode="/user/xyz/hbase-staging/admin__t1__e1f3m4r2prud9117thg5pdg91lkg0le0fdvtbbpg03epqg0f14lv54j8sqd8s0n6/cf1":hbase:hadoop:drwx------
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:342)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:223)
> at
> com.huawei.hadoop.adapter.hdfs.plugin.HWAccessControlEnforce.checkPermission(HWAccessControlEnforce.java:69)
> {noformat}
> So the root cause is "admin" user doesn't have EXECUTE permission over
> "/user/xyz", because RegionServer has created this intermediate parent
> directory during opening (SecureBulkLoadEndpoint) a region where the default
> permission is set as 700 based on the hdfs UMASK 077.
> *Solution:*
> =========
> However it can be handled by the creating /user/xyz manually and setting
> sufficient permission explicitly. But we should handle this by setting
> sufficient permission to intermediate staging directories which is created by
> RegionServer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
