[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245329#comment-16245329 ] Hudson commented on PHOENIX-4198: - SUCCESS: Integrated in Jenkins build Phoenix-master #1870 (See [https://builds.apache.org/job/Phoenix-master/1870/]) PHOENIX-4198 Remove the need for users to have access to the Phoenix (ankitsinghal59: rev 217867c78108b29d991794726c01c1eefb49b828) * (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/TableDDLPermissionsIT.java * (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixMetaDataCoprocessorHost.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/SchemaUtil.java * (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseMetaDataEndpointObserver.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java * (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java * (add) phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/RpcUtil.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java * (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointObserver.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexFailurePolicy.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/stats/StatisticsWriter.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.14.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245272#comment-16245272 ] Karan Mehta commented on PHOENIX-4198: -- Thanks [~an...@apache.org]. Also checkout PHOENIX-672 and let me know if you have any suggestions. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.14.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245269#comment-16245269 ] Thomas D'Silva commented on PHOENIX-4198: - Thanks [~ankit.singhal] > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.14.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245255#comment-16245255 ] Ankit Singhal commented on PHOENIX-4198: Sorry guys, was holding this up for RC. Just committed to master and 4.x-HBase-1.x branches. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.14.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244989#comment-16244989 ] Thomas D'Silva commented on PHOENIX-4198: - ping [~ankit.singhal] > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.14.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16242887#comment-16242887
]
Karan Mehta commented on PHOENIX-4198:
--
[~an...@apache.org]
{code}
public TableDDLPermissionsIT(final boolean isNamespaceMapped) throws
Exception {
this.isNamespaceMapped = isNamespaceMapped;
Map clientProps = Maps.newHashMapWithExpectedSize(1);
clientProps.put(QueryServices.IS_NAMESPACE_MAPPING_ENABLED, "true");
}
{code}
Nit: The clientProps is not used anywhere.
Also, when do you plan to commit this patch? I want to reuse the code from the
{{TableDDLPermissionsIT}} in writing tests for PHOENIX-672.
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch,
> PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238116#comment-16238116 ] Thomas D'Silva commented on PHOENIX-4198: - bq. I'm fine, if PHOENIX-672 can keep the users of views, Indexes and data table in sync but let's keep this code until PHOENIX-672 is checked-in with similar functionality. OK, sounds good. I am +1 on the patch. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Labels: namespaces, security > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16237575#comment-16237575 ] Ankit Singhal commented on PHOENIX-4198: bq. In PhoenixAccessController. preCreateTable, for indexes we need authorizeOrGrantAccessToUsers() to automatically grant all users/groups who have any permissions on the parent table the same permissions on the index table. So I think we don't need the isAutomaticGrantEnabled option. bq. This change can also be done as part of PHOENIX-672, if you think that makes more sense. I'm fine, if PHOENIX-672 can keep the users of views, Indexes and data table in sync but let's keep this code until PHOENIX-672 is checked-in with similar functionality. bq. Related to this, why is automatic grant not allowed for groups? Just had a thought, that giving an automatic grant to groups could be risky as they affect a lot of users, and on the other hand, groups would anyways be easy to manage by Admin. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Labels: namespaces, security > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227372#comment-16227372
]
Hadoop QA commented on PHOENIX-4198:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12895021/PHOENIX-4198_v7.patch
against master branch at commit 54a8f27304908f2259109eb230a2d3f261c2270a.
ATTACHMENT ID: 12895021
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+final UserGroupInformation superUser =
UserGroupInformation.createUserForTesting(SUPERUSER, new String[0]);
+final UserGroupInformation superUser2 =
UserGroupInformation.createUserForTesting("superuser", new String[0]);
+final UserGroupInformation regularUser =
UserGroupInformation.createUserForTesting("user", new String[0]);
+final UserGroupInformation groupUser =
UserGroupInformation.createUserForTesting("user2", new String[] {
GROUP_SYSTEM_ACCESS });
+final UserGroupInformation unprivilegedUser =
UserGroupInformation.createUserForTesting("unprivilegedUser",
+config.set("hbase.regionserver.wal.codec",
"org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec");
+grantPermissions(regularUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES, Action.READ,
+grantPermissions(unprivilegedUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(AuthUtil.toGroupEntry(GROUP_SYSTEM_ACCESS),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(regularUser.getShortUserName(),
Collections.singleton("SYSTEM:SEQUENCE"), Action.WRITE,
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1599//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1599//console
This message is automatically generated.
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch,
> PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227350#comment-16227350
]
Thomas D'Silva commented on PHOENIX-4198:
-
bq. I'm not sure if we can do it as a part of PHOENIX-672 as how can we give
access to the tables which doesn't exist. For eg: User1 has RX access on the
table and CREATE access on the namespace, but he should not be allowed to
create an Index until all users of the data table have access to the new Index
table.
Once PHOENIX-672 is complete, we should only use GRANT/REVOKE statements to
grant or revoke permissions. The GRANT/REVOKE command will only allow you to
grant or revoke permissions on a table. If the table has any indexes or if it
has a view that has any indexes the same permissions are automatically granted
or revoked from the index tables.
In PhoenixAccessController. preCreateTable, for indexes we need
authorizeOrGrantAccessToUsers() to automatically grant all users/groups who
have any permissions on the parent table the same permissions on the index
table. So I think we don't need the isAutomaticGrantEnabled option.
This change can also be done as part of PHOENIX-672, if you think that makes
more sense.
Related to this, why is automatic grant not allowed for groups?
{code}
+
if(AuthUtil.isGroupPrincipal(Bytes.toString(userPermission.getUser({
+AUDITLOG.warn("Users of GROUP:" +
Bytes.toString(userPermission.getUser())
++ " will not have following access
" + requireAccess
++ " to the newly created index " +
toTable
++ ", Automatic grant is not yet
allowed on Groups");
+continue;
+}
{code}
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch,
> PHOENIX-4198_v6.patch, PHOENIX-4198_v7.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226737#comment-16226737
]
Hadoop QA commented on PHOENIX-4198:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12894959/PHOENIX-4198_v6.patch
against master branch at commit 54a8f27304908f2259109eb230a2d3f261c2270a.
ATTACHMENT ID: 12894959
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+final UserGroupInformation superUser =
UserGroupInformation.createUserForTesting(SUPERUSER, new String[0]);
+final UserGroupInformation superUser2 =
UserGroupInformation.createUserForTesting("superuser", new String[0]);
+final UserGroupInformation regularUser =
UserGroupInformation.createUserForTesting("user", new String[0]);
+final UserGroupInformation groupUser =
UserGroupInformation.createUserForTesting("user2", new String[] {
GROUP_SYSTEM_ACCESS });
+final UserGroupInformation unprivilegedUser =
UserGroupInformation.createUserForTesting("unprivilegedUser",
+config.set("hbase.regionserver.wal.codec",
"org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec");
+grantPermissions(regularUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES, Action.READ,
+grantPermissions(unprivilegedUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(AuthUtil.toGroupEntry(GROUP_SYSTEM_ACCESS),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(regularUser.getShortUserName(),
Collections.singleton("SYSTEM:SEQUENCE"), Action.WRITE,
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterTableWithViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ChildViewsUseParentViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterMultiTenantTableWithViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ViewIndexIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1598//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1598//console
This message is automatically generated.
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch,
> PHOENIX-4198_v6.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226642#comment-16226642 ] Ankit Singhal commented on PHOENIX-4198: Thanks a lot [~tdsilva] for the great review. Updated the patch with the review comments. bq. Karan Mehta is working on PHOENIX-672 which will handle this case, so maybe some of this code can be used for that JIRA. As part of that JIRA we will only allow grant/revoke on the parent physical table for views and indexes, and keep the permissions on the index and view index physical tables in sync with the parent table, so I don't think we need to have an automatic grant option. We should always keep the index tables in sync with the parent. I'm not sure if we can do it as a part of PHOENIX-672 as how can we give access to the tables which doesn't exist. For eg: User1 has RX access on the table and CREATE access on the namespace, but he should not be allowed to create an Index until all users of the data table have access to the new Index table. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch, > PHOENIX-4198_v6.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225560#comment-16225560
]
Thomas D'Silva commented on PHOENIX-4198:
-
> Actually, this is needed when a NEW user is creating a view and Admin has
> just given READ/EXEC access to the user on the data table.
[~karanmehta93] is working on PHOENIX-672 which will handle this case, so maybe
some of this code can be used for that JIRA.
As part of that JIRA we will only allow grant/revoke on the parent physical
table for views and indexes, and keep the permissions on the index and view
index physical tables in sync with the parent table, so I don't think we need
to have an automatic grant option. We should always keep the index tables in
sync with the parent. FYI [~jamestaylor]
In MetadataEndpointImpl, you should always check that the user has the required
permissions on the parent table indexes (since they are added to the ptable of
child views see MetaDataClient.addIndexesFromParentTable )
{code}
+if (parentPhysicalSchemaTableNames[1] != null) {
+parentTableKey =
SchemaUtil.getTableKey(ByteUtil.EMPTY_BYTE_ARRAY,
+parentPhysicalSchemaTableNames[0],
parentPhysicalSchemaTableNames[1]);
+PTable parentTable = loadTable(env, parentTableKey, new
ImmutableBytesPtr(parentTableKey),
+clientTimeStamp, clientTimeStamp, clientVersion);
+cParentPhysicalName =
parentTable.getPhysicalName().getBytes();
+if (parentSchemaTableNames[1] != null
+&& Bytes.compareTo(parentSchemaTableNames[1],
parentPhysicalSchemaTableNames[1]) != 0) {
+// parent table is a view
+
indexes.add(TableName.valueOf(MetaDataUtil.getViewIndexPhysicalName(cParentPhysicalName)));
+} else {
+for (PTable index : parentTable.getIndexes()) {
+
indexes.add(TableName.valueOf(index.getPhysicalName().getBytes()));
+}
+}
+
+} else {
+// Mapped View
+cParentPhysicalName =
SchemaUtil.getTableNameAsBytes(schemaName, tableName);
}
{code}
Also the view index physical table might not exist (if that view doesn't have
an index on it), so you only need to check for the permission if it exists.
Apart from these, LGTM. Great work!
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
>
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222631#comment-16222631
]
Hadoop QA commented on PHOENIX-4198:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12894388/PHOENIX-4198_v5.patch
against master branch at commit fe13b257e5dfe29581b1c3265d79596f194954cd.
ATTACHMENT ID: 12894388
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+final UserGroupInformation superUser =
UserGroupInformation.createUserForTesting(SUPERUSER, new String[0]);
+final UserGroupInformation superUser2 =
UserGroupInformation.createUserForTesting("superuser", new String[0]);
+final UserGroupInformation regularUser =
UserGroupInformation.createUserForTesting("user", new String[0]);
+final UserGroupInformation groupUser =
UserGroupInformation.createUserForTesting("user2", new String[] {
GROUP_SYSTEM_ACCESS });
+final UserGroupInformation unprivilegedUser =
UserGroupInformation.createUserForTesting("unprivilegedUser",
+config.set("hbase.regionserver.wal.codec",
"org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec");
+grantPermissions(regularUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES, Action.READ,
+grantPermissions(unprivilegedUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(AuthUtil.toGroupEntry(GROUP_SYSTEM_ACCESS),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(regularUser.getShortUserName(),
Collections.singleton("SYSTEM:SEQUENCE"), Action.WRITE,
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ColumnEncodedImmutableTxStatsCollectorIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1580//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1580//console
This message is automatically generated.
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222559#comment-16222559
]
Hadoop QA commented on PHOENIX-4198:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12894379/PHOENIX-4198_v5.patch
against master branch at commit fe13b257e5dfe29581b1c3265d79596f194954cd.
ATTACHMENT ID: 12894379
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+final UserGroupInformation superUser =
UserGroupInformation.createUserForTesting(SUPERUSER, new String[0]);
+final UserGroupInformation superUser2 =
UserGroupInformation.createUserForTesting("superuser", new String[0]);
+final UserGroupInformation regularUser =
UserGroupInformation.createUserForTesting("user", new String[0]);
+final UserGroupInformation groupUser =
UserGroupInformation.createUserForTesting("user2", new String[] {
GROUP_SYSTEM_ACCESS });
+final UserGroupInformation unprivilegedUser =
UserGroupInformation.createUserForTesting("unprivilegedUser",
+config.set("hbase.regionserver.wal.codec",
"org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec");
+grantPermissions(regularUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES, Action.READ,
+grantPermissions(unprivilegedUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(AuthUtil.toGroupEntry(GROUP_SYSTEM_ACCESS),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(regularUser.getShortUserName(),
Collections.singleton("SYSTEM:SEQUENCE"), Action.WRITE,
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SetPropertyOnEncodedTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConcurrentMutationsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.PartialIndexRebuilderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.UpdateCacheIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SetPropertyOnNonEncodedTableIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1579//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1579//console
This message is automatically generated.
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222456#comment-16222456
]
Hadoop QA commented on PHOENIX-4198:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12894376/PHOENIX-4198_v5.patch
against master branch at commit fe13b257e5dfe29581b1c3265d79596f194954cd.
ATTACHMENT ID: 12894376
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+final UserGroupInformation superUser =
UserGroupInformation.createUserForTesting(SUPERUSER, new String[0]);
+final UserGroupInformation superUser2 =
UserGroupInformation.createUserForTesting("superuser", new String[0]);
+final UserGroupInformation regularUser =
UserGroupInformation.createUserForTesting("user", new String[0]);
+final UserGroupInformation groupUser =
UserGroupInformation.createUserForTesting("user2", new String[] {
GROUP_SYSTEM_ACCESS });
+final UserGroupInformation unprivilegedUser =
UserGroupInformation.createUserForTesting("unprivilegedUser",
+config.set("hbase.regionserver.wal.codec",
"org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec");
+grantPermissions(regularUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES, Action.READ,
+grantPermissions(unprivilegedUser.getShortUserName(),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(AuthUtil.toGroupEntry(GROUP_SYSTEM_ACCESS),
PHOENIX_NAMESPACE_MAPPED_SYSTEM_TABLES,
+grantPermissions(regularUser.getShortUserName(),
Collections.singleton("SYSTEM:SEQUENCE"), Action.WRITE,
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.GlobalMutableTxIndexIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1578//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1578//console
This message is automatically generated.
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch, PHOENIX-4198_v5.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1679#comment-1679 ] Ankit Singhal commented on PHOENIX-4198: Thanks [~tdsilva] and [~elserj] for the review.Updated the patch(v5) with the review comments. bq. When automatic grant is enabled you automatically grant the user who is creating the view access on the index table if they have access on the parent table. Is this required since when an index is created all users who have RWXCA access on the parent physical table are given corresponding access on the data table? Actually, this is needed when a NEW user is creating a view and Admin has just given READ/EXEC access to the user on the data table. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217473#comment-16217473
]
Josh Elser commented on PHOENIX-4198:
-
{code}
+try {
+Call rpcContext = RpcUtil.getRpcContext();
+//Setting RPC context as null so that user can be
resetted
+RpcUtil.setRpcContext(null);
+region.mutateRowsWithLocks(mutations, rowsToLock,
nonceGroup, nonce);
+//Setting RPC context back to original context of the
RPC
+RpcUtil.setRpcContext(rpcContext);
+} catch (Throwable e) {
+throw new IOException(e);
+}
{code}
[~ankit.singhal], would suggest you tweak this to use a try/finally block just
in case. e.g.
{code}
final Call rpcContext = RpcUtil.getRpcContext();
try {
//Setting RPC context as null so that user can be
resetted
RpcUtil.setRpcContext(null);
region.mutateRowsWithLocks(mutations, rowsToLock,
nonceGroup, nonce);
} catch (Throwable e) {
throw new IOException(e);
} finally {
//Setting RPC context back to original context of the
RPC
RpcUtil.setRpcContext(rpcContext);
}
{code}
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces, security
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208797#comment-16208797 ] Thomas D'Silva commented on PHOENIX-4198: - [~ankit.singhal] sorry for the delayed response. The patch is looking pretty good. Just a few additional comments. In PhoenixAccessController.preCreateTable where you check if you have access to the physical table indexes, you could skip any local indexes since you already check for access on the parent physical table. In order to handle view hierarchies (a view with a child view), should you also check if you have access to the view indexes of the parent view(s), if any present. You would only have to check for the physical table of a view index (since all view indexes are stored in a single table). When automatic grant is enabled you automatically grant the user who is creating the view access on the index table if they have access on the parent table. Is this required since when an index is created all users who have RWXCA access on the parent physical table are given corresponding access on the data table? A minor nit in MetadataEndpointImpl can you remove mutateRowsWithLocksWithEndpoint() and DEFAULT_ROWLOCK_WAIT_DURATION since they aren't used any more. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203162#comment-16203162 ] Ankit Singhal commented on PHOENIX-4198: Ping [~giacomotaylor] and [~tdsilva] > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces, security > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch, > PHOENIX-4198_v3.patch, PHOENIX-4198_v4.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197052#comment-16197052
]
Ankit Singhal commented on PHOENIX-4198:
I'm not sure we should require that a user who creates an index or view have
read and exec access on the parent table as well as all indexes on the parent
table. Its possible that we might have a user that only executes ddl commands.
bq. User need READ(and EXEC) access on data table to build the index, although
it may not hold true for async indexes if they are built by another user. And,
the purpose why user requires READ access on data table, is to restrict from
anybody to create views(As we don't have any other special permission in HBase
for logical entities).
bq. In PhoenixAccessController.authorizeOrGrantAccessToUsers should the boolean
haveAccess=false be within the loop on requiredActionsOnTable ?
yep, Thanks
bq. Also when you check if the user already has the action in {{if
(permToTable.implies(action)) {}} should you also check that the
permToTable.getUser() equals userPermission.getUser() ?
Actually, we are only getting the table permission of particular user of data
table only. so, we don't need to check it again.
{code}
List permsToTable =
getPermissionForUser(permissionsOnTheTable,userPermission.getUser());
{code}
bq. Can you add a test that verifies that users that have access the data table
also have access to an index after its created?
Added a test for the same in a new patch(v4).
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193663#comment-16193663
]
Thomas D'Silva commented on PHOENIX-4198:
-
I'm not sure we should require that a user who creates an index or view have
read and exec access on the parent table as well as all indexes on the parent
table. Its possible that we might have a user that only executes ddl commands .
WDYT [~jamestaylor] ?
In {{PhoenixAccessController.authorizeOrGrantAccessToUsers}} should the
{{boolean haveAccess=false}} be within the loop on requiredActionsOnTable ?
Also when you check if the user already has the action in {{if
(permToTable.implies(action)) {}} should you also check that the
permToTable.getUser() equals userPermission.getUser() ?
Can you add a test that verifies that users that have access the data table
also have access to an index after its created?
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces
> Fix For: 4.13.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch,
> PHOENIX-4198_v3.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191011#comment-16191011 ] Ankit Singhal commented on PHOENIX-4198: bq. Instead of using HTable to make an RPC to change the user context, is to possible to set the rpc context to null before running as the login user? Thanks [~tdsilva], it seems to be a good idea. I've updated the patch to accommodate this. please review. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.13.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189290#comment-16189290
]
Thomas D'Silva commented on PHOENIX-4198:
-
[~an...@apache.org]
Instead of using HTable to make an RPC to change the user context, is to
possible to set the rpc context to null before running as the login user? Then
when {{AccessController.getActiveUser()}} is called it would return the login
user and not the remote user. You can then just use
{{region.mutateRowsWithLocks}} and don't need the LockManager.
You would need a util class that is in the same package as {{RpcServer}}
{code}
package org.apache.hadoop.hbase.ipc;
import org.apache.hadoop.hbase.ipc.RpcServer.Call;
public class RpcUtil {
public static Call getRpcContext() {
return RpcServer.CurCall.get();
}
public static void setRpcContext(Call c){
RpcServer.CurCall.set(c);
}
}
{code}
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Labels: namespaces
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179140#comment-16179140 ] Ankit Singhal commented on PHOENIX-4198: bq. Since we already acquire the required locks before calling HTable.Batch would that be atomic as well ? Actually, still it will not be atomic, I changed the implementation to use MultiRowMutationEndpoint instead of HTable.batch to fix it.(Thanks [~devaraj] for the pointer) bq.If a user has create access on the namespace can he create a view, or does he also need read access on the data table to create the view? User still needs read access on table. bq.I think for creating an index a user should have create access for the schema/namespace. While creating an index users that have read access on the data table should also be granted read access on the index. Users that have write access on the table should be granted write access on the index, and for mutable indexes they should be given execute access on the data table (so that the index metadata can be sent to the server). Yep, that is already covered under PhoenixAccessController#grantAccessToUsers. (this will be an automatic grant which can be turned off if suspected as security issue) I'm done with the fixes (including above issues mentioned by Josh and Sergey). (Busy with day's job but will upload v3 soon.) > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172655#comment-16172655 ] Sergey Soldatov commented on PHOENIX-4198: -- There is an another problem. There is no check for the superuser. So if the table was created by user X and hbase user tries to create an index, the query would fail because we check only for the table permissions (which has only user X, but not hbase). > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172574#comment-16172574 ] Sergey Soldatov commented on PHOENIX-4198: -- [~an...@apache.org] there is a problem in kerberos setup. In hasAccess you are checking for the User.getName value, but hbase acl uses short names, so in the index scenario when user has namespace access we are able to create tables, but unable to create index tables (actually physical table is created, but after that fails during the check that user has access to the parent table). > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172319#comment-16172319 ] Thomas D'Silva commented on PHOENIX-4198: - bq. Actually, if ACL access is not enabled, we do it with the earlier logic(region.mutateRowsWithLocks) where region server acquiring read locks again during mutation. Not sure why we had required this with a single region of the system.catalog table as we were already taking lock before, does it ensure that all rows are written atomically which Htable.batch will not? are there any consequences of writing set of mutations with HTable.Batch which is not there in region.mutateRowsWithLocks? Since we already acquire the required locks before calling HTable.Batch would that be atomic as well ? If a user has create access on the namespace can he create a view, or does he also need read access on the data table to create the view? I think for creating an index a user should have create access for the schema/namespace. While creating an index users that have read access on the data table should also be granted read access on the index. Users that have write access on the table should be granted write access on the index, and for mutable indexes they should be given execute access on the data table (so that the index metadata can be sent to the server). [~karanmehta93] is working on PHOENIX-672 which can be used to grant users permissions to the index when permissions are granted on the data table. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170792#comment-16170792 ] Josh Elser commented on PHOENIX-4198: - bq. Just don't want to destabilize anything so close to when we're cutting 4.12 Yup, completely understood and no animosity is held. A close eye on this would be appreciated from (all of) you fine folks as the $dayjob train keeps rolling. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170722#comment-16170722 ] James Taylor commented on PHOENIX-4198: --- bq. For $dayjob reasons, I'd like to see this one get in 4.12, but not for any technical reason. I understand this not holding up 4.12. Just don't want to destabilize anything so close to when we're cutting 4.12. Lot's of stabilization fixes I'd like to get released as soon as possible. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170634#comment-16170634 ] Josh Elser commented on PHOENIX-4198: - bq. will this cooperate with PHOENIX-3534 which enables the SYSTEM.CATALOG to be split across more than one RS? Will ultimately defer to Ankit, but my understanding is "yes". We're not actually deferring to exclusion as a part of the "single JVM" of the current model, just tweaking our call to the AccessController to be less exclusive in which clients (really, which clients with certain permissions) are allowed to perform some operations. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170629#comment-16170629 ] Josh Elser commented on PHOENIX-4198: - bq. How about we target this JIRA for 4.13? I'd like to get a 4.12 RC up in a few days. For $dayjob reasons, I'd like to see this one get in 4.12, but not for any technical reason. I understand this not holding up 4.12. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170149#comment-16170149 ] James Taylor commented on PHOENIX-4198: --- How about we target this JIRA for 4.13? I'd like to get a 4.12 RC up in a few days. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch, PHOENIX-4198_v2.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170069#comment-16170069 ] Ankit Singhal commented on PHOENIX-4198: Thanks [~apurtell],[~elserj],[~tdsilva],[~sergey.soldatov] for the comments. bq. Make sure to add a JUnit Category annotation to TableDDLPermissionsIT otherwise you'll run into PHOENIX-4191. Does it make sense to consolidate that test with SystemTablePermissionsIT? Annotated with a category for now. Yes, we probably can remove SystemTablePermissionsIT now. bq. mutateRowsWithLocks(...) seems a little weird – when we're doing the new code-path, we don't explicitly grab any locks whereas I'd assume that does happen in the other codepath. Is that OK? Actually, if ACL access is not enabled, we do it with the earlier logic(region.mutateRowsWithLocks) where region server acquiring read locks again during mutation. Not sure why we had required this with a single region of the system.catalog table as we were already taking lock before, does it ensure that all rows are written atomically which Htable.batch will not? are there any consequences of writing set of mutations with HTable.Batch which is not there in region.mutateRowsWithLocks? bq. but mandating them is IMHO mandating user lock-in on something external to HBase+Phoenix for what is a core functional concern (protection of SYSTEM tables) Yes [~apurtell], as [~elserj] also clear my point that we are not making Phoenix dependent on any third party component for access control for DDLs operation. The purpose is just to use the right set of public API so that third-party component can also work automatically in future along with HBase Native access control if they implement the APIs required for it. bq. I believe he did some legwork to realize that one of these third-party components didn't actually implement the full interface, and would thus not actually work. I'll let Ankit confirm, but I think you are of the same opinion that I am – Ranger (or Sentry) not implementing the full interface is not our problem and not something we should care about. yes, Ranger or Sentry will not work until they implement the AccessControlService.Interface#getUserPermissions API bq. Thinking about this more, even if we use this approach to not require that a user has write access to SYSTEM.SEQUENCE a user can still access any sequence by using SELECT NEXT VALUE for, so maybe that should be addressed separately. Yes, We may need to take it in separate JIRA, as we need to treat sequences as another entity which currently doesn't exist in HBase bq. Are the changes in MetadataEndpointImpl that use LockManager required because even if you run as the HBase user calling region.mutateRowsWithLock still uses the original user context ? Yes [~tdsilva] that's why we need to create a new RPC using HTable to change the user context. bq. In QueryServicesOptions.java there is a new import of net.sf.ehcache.config.TerracottaConfiguration.Consistency. Is it incorrect IDE autoimport ? [~sergey soldatov], removed from newer version > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165832#comment-16165832 ] Sergey Soldatov commented on PHOENIX-4198: -- [~an...@apache.org] In QueryServicesOptions.java there is a new import of net.sf.ehcache.config.TerracottaConfiguration.Consistency. Is it incorrect IDE autoimport ? > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165199#comment-16165199 ] Thomas D'Silva commented on PHOENIX-4198: - [~ankit.singhal] The user only needs write access to SYSTEM.SEQUENCE to get the next value for a sequence. Thinking about this more, even if we use this approach to not require that a user has write access to SYSTEM.SEQUENCE a user can still access any sequence by using SELECT NEXT VALUE for, so maybe that should be addressed separately. Are the changes in MetadataEndpointImpl that use LockManager required because even if you run as the HBase user calling region.mutateRowsWithLock still uses the original user context ? > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165100#comment-16165100 ] Andrew Purtell commented on PHOENIX-4198: - I think Phoenix should use HBase native security features, there is no need to require use of a third party component like Ranger or Sentry. If Ranger or Sentry offer something compelling, fine, a user is free to pick them up and use them, but mandating them is IMHO mandating user lock-in on something external to HBase+Phoenix for what is a core functional concern (protection of SYSTEM tables). That is the wrong approach. HBase+Phoenix should natively provide protection of SYSTEM tables as required. Also, there are already a lot of downstream dependencies for a Phoenix build due to all of the connector modules, and the cross-product of these already present a challenge to DIY-ers trying to orchestrate a whole stack for production. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164785#comment-16164785 ] Josh Elser commented on PHOENIX-4198: - bq. I'm still looking for the API(available in both Ranger and HBase) to check read access on the base table. I seek help from Josh Elser/Andrew Purtell on this in my last comment. I can see AccessControlService.Interface has some APIs but they are not yet implemented by Ranger. Sorry, I forgot to address this point yesterday. [~enis] might also be good to bother. I'm of the opinion that if HBase exposes an API for the purpose of what you need to accomplish a task but Ranger doesn't implement that method, that's a Ranger problem. Doesn't mean that is the wrong API call to use :) > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164255#comment-16164255 ] Ankit Singhal commented on PHOENIX-4198: Thanks [~tdsilva] for taking the look. bq. If a user has read access to a base table, will they be able to create a view? Or will the user need CREATE access on the namespace? I'm still looking for the API(available in both Ranger and HBase) to check read access on the base table. I seek help from [~elserj]/[~apurtell] on this in my last comment. I can see AccessControlService.Interface has some APIs but they are not yet implemented by Ranger. bq. Any thoughts on if we should be doing something similar for SequenceRegionObserver, currently users need a write access to SYSTEM.SEQUENCE in order to use sequences. Currently, we are using the authorization provided by HBase/Ranger so do you know which user access( on schema/table) is required by the regular user to create/increment the sequences. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163805#comment-16163805 ] Thomas D'Silva commented on PHOENIX-4198: - If a user has read access to a base table, will they be able to create a view? Or will the user need CREATE access on the namespace? Any thoughts on if we should be doing something similar for SequenceRegionObserver, currently users need write access to SYSTEM.SEQUENCE in order to use sequences. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Labels: namespaces > Fix For: 4.12.0 > > Attachments: PHOENIX-4198.patch > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[
https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163061#comment-16163061
]
Josh Elser commented on PHOENIX-4198:
-
Thanks for working on this one, Ankit. This has definitely been a repeated ask
I've seen come across my "desk".
Make sure to add a JUnit Category annotation to {{TableDDLPermissionsIT}}
otherwise you'll run into PHOENIX-4191. Does it make sense to consolidate that
test with {{SystemTablePermissionsIT}}?
{{mutateRowsWithLocks(...)}} seems a little weird -- when we're doing the new
code-path, we don't explicitly grab any locks whereas I'd assume that does
happen in the other codepath. Is that OK?
Looks OK at a high-level to me, but I should bring it down and play with it
locally too :)
FYI [~jamestaylor]
> Remove the need for users to have access to the Phoenix SYSTEM tables to
> create tables
> --
>
> Key: PHOENIX-4198
> URL: https://issues.apache.org/jira/browse/PHOENIX-4198
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4198.patch
>
>
> Problem statement:-
> A user who doesn't have access to a table should also not be able to modify
> Phoenix Metadata. Currently, every user required to have a write permission
> to SYSTEM tables which is a security concern as they can
> create/alter/drop/corrupt meta data of any other table without proper access
> to the corresponding physical tables.
> [~devaraj] recommended a solution as below.
> 1. A coprocessor endpoint would be implemented and all write accesses to the
> catalog table would have to necessarily go through that. The 'hbase' user
> would own that table. Today, there is MetaDataEndpointImpl that's run on the
> RS where the catalog is hosted, and that could be enhanced to serve the
> purpose we need.
> 2. The regionserver hosting the catalog table would do the needful for all
> catalog updates - creating the mutations as needed, that is.
> 3. The coprocessor endpoint could use Ranger to do necessary authorization
> checks before updating the catalog table. So for example, if a user doesn't
> have authorization to create a table in a certain namespace, or update the
> schema, etc., it can reject such requests outright. Only after successful
> validations, does it perform the operations (physical operations to do with
> creating the table, and updating the catalog table with the necessary
> mutations).
> 4. In essence, the code that implements dealing with DDLs, would be hosted in
> the catalog table endpoint. The client code would be really thin, and it
> would just invoke the endpoint with the necessary info. The additional thing
> that needs to be done in the endpoint is the validation of authorization to
> prevent unauthorized users from making changes to someone else's
> tables/schemas/etc. For example, one should be able to create a view on a
> table if he has read access on the base table. That mutation on the catalog
> table would be permitted. For changing the schema (adding a new column for
> example), the said user would need write permission on the table... etc etc.
> Thanks [~elserj] for the write-up.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4198) Remove the need for users to have access to the Phoenix SYSTEM tables to create tables
[ https://issues.apache.org/jira/browse/PHOENIX-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163001#comment-16163001 ] Ankit Singhal commented on PHOENIX-4198: PFA, a wip patch for the same. Currently, Create table/index/view * A user who has a CREATE access on schema/namespace can create table/view/index. We also need read access to the data table for view and index but not sure which API (common in Ranger/native authorization) we can use. [~elserj]/[~apurtell] any idea? Drop table/index/view * A user who has CREATE access on physical/data table can drop table/index/view. Same we need write access to the physical table, not sure which API I can use? Alter(Add/drop column,properties) table/index/view * A user who has CREATE access on physical/data table can alter table/index/view. Create/drop schema * A user who has GLOBAL ADMIN access can create and drop the schema. > Remove the need for users to have access to the Phoenix SYSTEM tables to > create tables > -- > > Key: PHOENIX-4198 > URL: https://issues.apache.org/jira/browse/PHOENIX-4198 > Project: Phoenix > Issue Type: Bug >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Fix For: 4.12.0 > > > Problem statement:- > A user who doesn't have access to a table should also not be able to modify > Phoenix Metadata. Currently, every user required to have a write permission > to SYSTEM tables which is a security concern as they can > create/alter/drop/corrupt meta data of any other table without proper access > to the corresponding physical tables. > [~devaraj] recommended a solution as below. > 1. A coprocessor endpoint would be implemented and all write accesses to the > catalog table would have to necessarily go through that. The 'hbase' user > would own that table. Today, there is MetaDataEndpointImpl that's run on the > RS where the catalog is hosted, and that could be enhanced to serve the > purpose we need. > 2. The regionserver hosting the catalog table would do the needful for all > catalog updates - creating the mutations as needed, that is. > 3. The coprocessor endpoint could use Ranger to do necessary authorization > checks before updating the catalog table. So for example, if a user doesn't > have authorization to create a table in a certain namespace, or update the > schema, etc., it can reject such requests outright. Only after successful > validations, does it perform the operations (physical operations to do with > creating the table, and updating the catalog table with the necessary > mutations). > 4. In essence, the code that implements dealing with DDLs, would be hosted in > the catalog table endpoint. The client code would be really thin, and it > would just invoke the endpoint with the necessary info. The additional thing > that needs to be done in the endpoint is the validation of authorization to > prevent unauthorized users from making changes to someone else's > tables/schemas/etc. For example, one should be able to create a view on a > table if he has read access on the base table. That mutation on the catalog > table would be permitted. For changing the schema (adding a new column for > example), the said user would need write permission on the table... etc etc. > Thanks [~elserj] for the write-up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
