Re: ACLs/Quotas for HBase structures
Thanks, Esteban for the quick reply. I'll work on implementing your suggestions and see how far it goes toward meeting our requirement.
Re: ACLs/Quotas for HBase structures
Hello Manoj, Thats a very interesting requirement, unfortunately the existing HBase directory structure needs to be owned by the user that started HBase (usually the 'hbase' user) and HBase will handle all the permissions and ACL rules without exposing details from HDFS to the client API. Even if you change the ownership of Hfiles under a specific namespace or table, compactions or splits will create new files as the HBase superuser. Also, running multiple HBase clusters under the same HDFS cluster to go around this limitation should be avoided since it can make things really hard to manage. Regarding the quota requirement, probably HBASE-8410/HBASE-11999 (work in progress) will be the best option in order to support quotas in a reliable way. Keep in mind that using HDFS quotas with HBase doesn't work well since compactions and splits can increase the space utilization beyond the quota sometimes and that could trigger failures in the Region Servers. Probably your only option for now is to monitor the namespace utilization, use Kerberos for authentication and specify a set of ACL rules that will guarantee that users cannot access namespaces that are not allowed to. cheers, esteban. -- Cloudera, Inc. On Fri, Dec 19, 2014 at 10:31 PM, Manoj Murumkar manoj.murum...@gmail.com wrote: Folks, We are trying to control space usage and manage security at HBase namespace level. Think of it in terms of a RDBMS (database and superuser for a database). Is there a simple way to do this? This is what I have in mind. Does it make sense? - Space quotas: Namespace is managed under /apps/hbase/data/data/namespace directory. We'll use dfsadmin -setQuota to set appropriate quotas. - Make superuser owner of /apps/hbase/data/data/namespace directory in hdfs. Thanks for any pointers from folks who have done this kind of thing. Appreciate the time. Manoj
Re: ACLs/Quotas for HBase structures
Manoj: HBASE-8410 is under active development. If you have time, please go over the feature to see if it fits your need. Cheers On Dec 19, 2014, at 11:42 PM, Esteban Gutierrez este...@cloudera.com wrote: Hello Manoj, Thats a very interesting requirement, unfortunately the existing HBase directory structure needs to be owned by the user that started HBase (usually the 'hbase' user) and HBase will handle all the permissions and ACL rules without exposing details from HDFS to the client API. Even if you change the ownership of Hfiles under a specific namespace or table, compactions or splits will create new files as the HBase superuser. Also, running multiple HBase clusters under the same HDFS cluster to go around this limitation should be avoided since it can make things really hard to manage. Regarding the quota requirement, probably HBASE-8410/HBASE-11999 (work in progress) will be the best option in order to support quotas in a reliable way. Keep in mind that using HDFS quotas with HBase doesn't work well since compactions and splits can increase the space utilization beyond the quota sometimes and that could trigger failures in the Region Servers. Probably your only option for now is to monitor the namespace utilization, use Kerberos for authentication and specify a set of ACL rules that will guarantee that users cannot access namespaces that are not allowed to. cheers, esteban. -- Cloudera, Inc. On Fri, Dec 19, 2014 at 10:31 PM, Manoj Murumkar manoj.murum...@gmail.com wrote: Folks, We are trying to control space usage and manage security at HBase namespace level. Think of it in terms of a RDBMS (database and superuser for a database). Is there a simple way to do this? This is what I have in mind. Does it make sense? - Space quotas: Namespace is managed under /apps/hbase/data/data/namespace directory. We'll use dfsadmin -setQuota to set appropriate quotas. - Make superuser owner of /apps/hbase/data/data/namespace directory in hdfs. Thanks for any pointers from folks who have done this kind of thing. Appreciate the time. Manoj
