[jira] [Commented] (SENTRY-2160) Add owner in create table notification event
[
https://issues.apache.org/jira/browse/SENTRY-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409064#comment-16409064
]
Hadoop QA commented on SENTRY-2160:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12915543/SENTRY-2160.003.patch
against master.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationWithHA
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3708/console
This message is automatically generated.
> Add owner in create table notification event
>
>
> Key: SENTRY-2160
> URL: https://issues.apache.org/jira/browse/SENTRY-2160
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-2160.001.patch, SENTRY-2160.002.patch,
> SENTRY-2160.003.patch
>
>
> When creating notification event in SentryJSONCreateTableMessage, save the
> owner of the table, so sentry knows the owner and can create implicit
> privileges for the owner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408653#comment-16408653 ] Alexander Kolbasov commented on SENTRY-2154: [~LinaAtAustin] [~kkalyan] Can you post proposed changes to the model Java classes and package.jdo file? > Update schema to grant privileges to user > - > > Key: SENTRY-2154 > URL: https://issues.apache.org/jira/browse/SENTRY-2154 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add new DB table to support grant user to privileges > Also, a flag should be added in privilege table to indicate the privilege is > created by user, or created by sentry implicitly. User can view the implicit > privileges, but cannot change it directly -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408641#comment-16408641 ] Alexander Kolbasov commented on SENTRY-2154: We had some discussion a while ago about the relationship between roles and privileges. Currently it is M:N meaning that there is a pool of roles and a pool of privileges and they refer to each other in some way. I was suggesting to change this to a different model where each role may have a bunch of privileges so it is more like 1:N relationship - in the current model privileges do not make much sense outside of a role. Currently Sentry uses role-based model - only roles have privileges. This proposal changes this - now users can have privileges and users are not roles. So now we need to define relationship between users, groups, roles and privileges. Can someone summarize the proposed relationships between all these? > Update schema to grant privileges to user > - > > Key: SENTRY-2154 > URL: https://issues.apache.org/jira/browse/SENTRY-2154 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add new DB table to support grant user to privileges > Also, a flag should be added in privilege table to indicate the privilege is > created by user, or created by sentry implicitly. User can view the implicit > privileges, but cannot change it directly -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2186) Support HDFS ACL for fine grained privileges
kalyan kumar kalvagadda created SENTRY-2186: --- Summary: Support HDFS ACL for fine grained privileges Key: SENTRY-2186 URL: https://issues.apache.org/jira/browse/SENTRY-2186 Project: Sentry Issue Type: Sub-task Components: Sentry Affects Versions: 2.1.0 Reporter: kalyan kumar kalvagadda we need to extend the HDFS sync to fine grained privileges like create, drop and alter. I will be updating some more details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408622#comment-16408622 ] Na Li commented on SENTRY-2154: --- [~kkalyan] We already have MSentryUser, MSentryGroup, MSentryPrivilege, MSentryGMPrivilege. To support granting privilege to user in DB schema (scope of this jira), we just need to add mapping table between user and privilege, which is the same approach as before (like granting privilege to role is mapped by SENTRY_ROLE_DB_PRIVILEGE_MAP table). If we need to grant privilege to group directly, we can add a mapping table between group and privilege. DataNucleus does lazy loading and we can control the depth of recursive loading. So circular reference is not an issue. http://www.datanucleus.org/products/accessplatform_4_1/jdo/fetchgroup.html 1) Drawbacks of using generic entity table If we use the generic entity table, will we migrate the tables to that entity table: SENTRY_USER, SENTRY_GROUP, SENTRY_ROLE? That will cause a lot of upgrade issues. Also, having a generic entity table will cause performance issue because each query has to be changed to add filter for the entity type. It is also easier to make mistake if the wrong entries are pulled. In addition, code maintenance is much harder. If we use generic entity table, we still need to add mapping table. Having a generic approach has more flexibility, but at the cost of complexity. We need to weight the benefits and drawbacks. 2) Benefits of using generic entity table Flexibility to support new entity types including user. When a new type is introduced, no need to change DB schema. 3) My preference I don't see much benefits by using a generic entity table, but many drawbacks. So I prefer to just adding a mapping table between user and privilege to support granting privilege to user directly. [~akolb] [~btowles] [~spena] Can you post your opinions? We need to decide the approach as soon as possible. I am already coding in the approach I prefer. Delayed decision will definitely delay the progress of this task. Thanks! > Update schema to grant privileges to user > - > > Key: SENTRY-2154 > URL: https://issues.apache.org/jira/browse/SENTRY-2154 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add new DB table to support grant user to privileges > Also, a flag should be added in privilege table to indicate the privilege is > created by user, or created by sentry implicitly. User can view the implicit > privileges, but cannot change it directly -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2184) Performance Issue: MPath is queried for each MAuthzPathsMapping in full snapshot
[
https://issues.apache.org/jira/browse/SENTRY-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2184:
--
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
> Performance Issue: MPath is queried for each MAuthzPathsMapping in full
> snapshot
>
>
> Key: SENTRY-2184
> URL: https://issues.apache.org/jira/browse/SENTRY-2184
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Critical
> Fix For: 2.1.0
>
> Attachments: SENTRY-2184.001.patch, SENTRY-2184.002.patch,
> SENTRY-2184.003.patch
>
>
> MAuthzPathsMapping contains list of MPath instances. From log message, when
> getting path full snapshot at SentryStore.retrieveFullPathsImageCore(),
> DataNucleus issues a query for all MPath instances associated with each
> MAuthzPathsMapping. Therefore, getting full path image may take a very long
> time.
> The solution is to get MPath in a batch when getting full path image.
> Log Message when DataNucleus issues a query for all MPath instances
> associated with each MAuthzPathsMapping
> {code:java}
> 1) Initially, all MAuthzPathsMapping entries for current snapshot is queried.
> 2018-03-14 11:51:23,999 (main) [DEBUG -
> org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)] SELECT
> 'org.apache.sentry.provider.db.service.model.MAuthzPathsMapping' AS
> NUCLEUS_TYPE,A0.AUTHZ_OBJ_NAME,A0.AUTHZ_SNAPSHOT_ID,A0.CREATE_TIME_MS,A0.AUTHZ_OBJ_ID
> FROM AUTHZ_PATHS_MAPPING A0 WHERE A0.AUTHZ_SNAPSHOT_ID = <1>
> 2) call authzToPaths.getPathStrings() causes MPath to be queried for each
> AUTHZ_OBJ_ID
> 2018-03-14 11:52:27,700 (main) [DEBUG -
> org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)] SELECT
> 'org.apache.sentry.provider.db.service.model.MPath' AS
> NUCLEUS_TYPE,A0.PATH_NAME,A0.PATH_ID FROM AUTHZ_PATH A0 WHERE A0.AUTHZ_OBJ_ID
> = <1>{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408587#comment-16408587 ] kalyan kumar kalvagadda commented on SENTRY-2154: - we need to have the capability of grating privilege to user and groups in future. This is feature that we have plans to implement in short term. Using entity table makes that the logic generic for both user and group privileges. > Update schema to grant privileges to user > - > > Key: SENTRY-2154 > URL: https://issues.apache.org/jira/browse/SENTRY-2154 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add new DB table to support grant user to privileges > Also, a flag should be added in privilege table to indicate the privilege is > created by user, or created by sentry implicitly. User can view the implicit > privileges, but cannot change it directly -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408581#comment-16408581 ] kalyan kumar kalvagadda commented on SENTRY-2154: - [~LinaAtAustin] and [~spena] i understand your point but we need to consider making change to the database which are generic enough for future developments not looking at immediate requirements. Unless there is complexity invoked it is worth having the change generic. > Update schema to grant privileges to user > - > > Key: SENTRY-2154 > URL: https://issues.apache.org/jira/browse/SENTRY-2154 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add new DB table to support grant user to privileges > Also, a flag should be added in privilege table to indicate the privilege is > created by user, or created by sentry implicitly. User can view the implicit > privileges, but cannot change it directly -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2184) Performance Issue: MPath is queried for each MAuthzPathsMapping in full snapshot
[
https://issues.apache.org/jira/browse/SENTRY-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408531#comment-16408531
]
Hadoop QA commented on SENTRY-2184:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12915520/SENTRY-2184.003.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3707/console
This message is automatically generated.
> Performance Issue: MPath is queried for each MAuthzPathsMapping in full
> snapshot
>
>
> Key: SENTRY-2184
> URL: https://issues.apache.org/jira/browse/SENTRY-2184
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Critical
> Attachments: SENTRY-2184.001.patch, SENTRY-2184.002.patch,
> SENTRY-2184.003.patch
>
>
> MAuthzPathsMapping contains list of MPath instances. From log message, when
> getting path full snapshot at SentryStore.retrieveFullPathsImageCore(),
> DataNucleus issues a query for all MPath instances associated with each
> MAuthzPathsMapping. Therefore, getting full path image may take a very long
> time.
> The solution is to get MPath in a batch when getting full path image.
> Log Message when DataNucleus issues a query for all MPath instances
> associated with each MAuthzPathsMapping
> {code:java}
> 1) Initially, all MAuthzPathsMapping entries for current snapshot is queried.
> 2018-03-14 11:51:23,999 (main) [DEBUG -
> org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)] SELECT
> 'org.apache.sentry.provider.db.service.model.MAuthzPathsMapping' AS
> NUCLEUS_TYPE,A0.AUTHZ_OBJ_NAME,A0.AUTHZ_SNAPSHOT_ID,A0.CREATE_TIME_MS,A0.AUTHZ_OBJ_ID
> FROM AUTHZ_PATHS_MAPPING A0 WHERE A0.AUTHZ_SNAPSHOT_ID = <1>
> 2) call authzToPaths.getPathStrings() causes MPath to be queried for each
> AUTHZ_OBJ_ID
> 2018-03-14 11:52:27,700 (main) [DEBUG -
> org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)] SELECT
> 'org.apache.sentry.provider.db.service.model.MPath' AS
> NUCLEUS_TYPE,A0.PATH_NAME,A0.PATH_ID FROM AUTHZ_PATH A0 WHERE A0.AUTHZ_OBJ_ID
> = <1>{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2160) Add owner in create table notification event
[ https://issues.apache.org/jira/browse/SENTRY-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kalyan kumar kalvagadda updated SENTRY-2160: Attachment: SENTRY-2160.003.patch > Add owner in create table notification event > > > Key: SENTRY-2160 > URL: https://issues.apache.org/jira/browse/SENTRY-2160 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2160.001.patch, SENTRY-2160.002.patch, > SENTRY-2160.003.patch > > > When creating notification event in SentryJSONCreateTableMessage, save the > owner of the table, so sentry knows the owner and can create implicit > privileges for the owner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2160) Add owner in create table notification event
[
https://issues.apache.org/jira/browse/SENTRY-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408263#comment-16408263
]
Hadoop QA commented on SENTRY-2160:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12915495/SENTRY-2160.002.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3706/console
This message is automatically generated.
> Add owner in create table notification event
>
>
> Key: SENTRY-2160
> URL: https://issues.apache.org/jira/browse/SENTRY-2160
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-2160.001.patch, SENTRY-2160.002.patch
>
>
> When creating notification event in SentryJSONCreateTableMessage, save the
> owner of the table, so sentry knows the owner and can create implicit
> privileges for the owner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2184) Performance Issue: MPath is queried for each MAuthzPathsMapping in full snapshot
[
https://issues.apache.org/jira/browse/SENTRY-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2184:
--
Attachment: SENTRY-2184.003.patch
> Performance Issue: MPath is queried for each MAuthzPathsMapping in full
> snapshot
>
>
> Key: SENTRY-2184
> URL: https://issues.apache.org/jira/browse/SENTRY-2184
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Critical
> Attachments: SENTRY-2184.001.patch, SENTRY-2184.002.patch,
> SENTRY-2184.003.patch
>
>
> MAuthzPathsMapping contains list of MPath instances. From log message, when
> getting path full snapshot at SentryStore.retrieveFullPathsImageCore(),
> DataNucleus issues a query for all MPath instances associated with each
> MAuthzPathsMapping. Therefore, getting full path image may take a very long
> time.
> The solution is to get MPath in a batch when getting full path image.
> Log Message when DataNucleus issues a query for all MPath instances
> associated with each MAuthzPathsMapping
> {code:java}
> 1) Initially, all MAuthzPathsMapping entries for current snapshot is queried.
> 2018-03-14 11:51:23,999 (main) [DEBUG -
> org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)] SELECT
> 'org.apache.sentry.provider.db.service.model.MAuthzPathsMapping' AS
> NUCLEUS_TYPE,A0.AUTHZ_OBJ_NAME,A0.AUTHZ_SNAPSHOT_ID,A0.CREATE_TIME_MS,A0.AUTHZ_OBJ_ID
> FROM AUTHZ_PATHS_MAPPING A0 WHERE A0.AUTHZ_SNAPSHOT_ID = <1>
> 2) call authzToPaths.getPathStrings() causes MPath to be queried for each
> AUTHZ_OBJ_ID
> 2018-03-14 11:52:27,700 (main) [DEBUG -
> org.datanucleus.util.Log4JLogger.debug(Log4JLogger.java:58)] SELECT
> 'org.apache.sentry.provider.db.service.model.MPath' AS
> NUCLEUS_TYPE,A0.PATH_NAME,A0.PATH_ID FROM AUTHZ_PATH A0 WHERE A0.AUTHZ_OBJ_ID
> = <1>{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2160) Add owner in create table notification event
[ https://issues.apache.org/jira/browse/SENTRY-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kalyan kumar kalvagadda updated SENTRY-2160: Attachment: SENTRY-2160.002.patch > Add owner in create table notification event > > > Key: SENTRY-2160 > URL: https://issues.apache.org/jira/browse/SENTRY-2160 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2160.001.patch, SENTRY-2160.002.patch > > > When creating notification event in SentryJSONCreateTableMessage, save the > owner of the table, so sentry knows the owner and can create implicit > privileges for the owner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
