[jira] [Updated] (SENTRY-2299) NPE In Sentry HDFS Sync Plugin
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2299:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> NPE In Sentry HDFS Sync Plugin
> --
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2299.001.patch
>
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2312) Update owner privileges for table when owner is changed.
[ https://issues.apache.org/jira/browse/SENTRY-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543636#comment-16543636 ] Na Li commented on SENTRY-2312: --- should the command be more specific "alter table set owner"? What is the difference between this jira and "SENTRY-2274: Grant and revoke owner privileges based on HMS updates(server-side)", which updates the owner privilege when owner changes > Update owner privileges for table when owner is changed. > > > Key: SENTRY-2312 > URL: https://issues.apache.org/jira/browse/SENTRY-2312 > Project: Sentry > Issue Type: Sub-task >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > > owner privilege should be updated when the owner for a table is changed using > "alter table" command -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2310) Sentry is not be able to fetch full update subsequently, when there is HMS restart in the snapshot process.
[ https://issues.apache.org/jira/browse/SENTRY-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543627#comment-16543627 ] Na Li commented on SENTRY-2310: --- [~kkalyan] We have SentryHMSClient to get full snapshot (have bug) and HiveNotificationFetcher for delta change (not have bug). It is easy to create bug and forget to update all places. What's the reason we have two classes instead of one? If it is relative big change to marge two classes into one, we can create a jira for that for now, and merge them later. > Sentry is not be able to fetch full update subsequently, when there is HMS > restart in the snapshot process. > > > Key: SENTRY-2310 > URL: https://issues.apache.org/jira/browse/SENTRY-2310 > Project: Sentry > Issue Type: Bug >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Attachments: SENTRY-2310.001.patch, SENTRY-2310.002.patch, > SENTRY-2310.003.patch > > > If the communication between sentry and HMS goes down for any reason while > sentry is fetching full update from HMS, SentryHMSClient in HMSFollower would > be left with a reference to closed socket. As sentry is not handling the > failure and closing the SentryHMSClient, it continues using the same > SentryHMSClient. This will result in "java.net.SocketException: Broken pipe" > as the client tries to write on socket that is closed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2299) NPE In Sentry HDFS Sync Plugin
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543366#comment-16543366
]
Na Li commented on SENTRY-2299:
---
Sentry client at NameNode does not create entry for a table whose location is
not with in the sentry managed prefix's . When user rename this table with
different path, null exception happens. The fix is to check if old entry exists
or not. If exists, move old paths to new table. Otherwise, skip that step.
> NPE In Sentry HDFS Sync Plugin
> --
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2299.001.patch
>
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassi
[jira] [Commented] (SENTRY-2310) Sentry is not be able to fetch full update subsequently, when there is HMS restart in the snapshot process.
[ https://issues.apache.org/jira/browse/SENTRY-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543338#comment-16543338 ] Na Li commented on SENTRY-2310: --- [~kkalyan] I prefer the option 1 to avoid unnecessary overhead of creating HiveMetaStoreClient when connection is in good state. 1) In HiveNotificationFetcher, whenever gets exception in request, the HiveMetaStoreClient is closed and set to null. So getting delta changes does not have this issue. 2) SentryHMSClient.getFullSnapshot has this issue because the HiveMetaStoreClient is not closed at exception, and will be used again in next pulling. So getting full snapshot has this issue. > Sentry is not be able to fetch full update subsequently, when there is HMS > restart in the snapshot process. > > > Key: SENTRY-2310 > URL: https://issues.apache.org/jira/browse/SENTRY-2310 > Project: Sentry > Issue Type: Bug >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Attachments: SENTRY-2310.001.patch, SENTRY-2310.002.patch, > SENTRY-2310.003.patch > > > If the communication between sentry and HMS goes down for any reason while > sentry is fetching full update from HMS, SentryHMSClient in HMSFollower would > be left with a reference to closed socket. As sentry is not handling the > failure and closing the SentryHMSClient, it continues using the same > SentryHMSClient. This will result in "java.net.SocketException: Broken pipe" > as the client tries to write on socket that is closed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2158) Update notification handler to update privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542251#comment-16542251 ] Na Li commented on SENTRY-2158: --- Right now, we do not expose public API at Sentry server for granting or revoking user privileges. Therefore, this jira cannot be done until we expose such API, like alter_sentry_user_grant_privilege and alter_sentry_user_revoke_privilege, in SentryPolicyService. > Update notification handler to update privileges to user > > > Key: SENTRY-2158 > URL: https://issues.apache.org/jira/browse/SENTRY-2158 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing > permission related commands. We should update notification handler in the > following files when granting privileges to user. > When authorizable changes, need to change user privileges too > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2158) Update notification handler to update privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2158: -- Attachment: (was: SENTRY-2158.001.patch) > Update notification handler to update privileges to user > > > Key: SENTRY-2158 > URL: https://issues.apache.org/jira/browse/SENTRY-2158 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing > permission related commands. We should update notification handler in the > following files when granting privileges to user. > When authorizable changes, need to change user privileges too > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2158) Update notification handler to update privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2158: -- Issue Type: New Feature (was: Sub-task) Parent: (was: SENTRY-2151) > Update notification handler to update privileges to user > > > Key: SENTRY-2158 > URL: https://issues.apache.org/jira/browse/SENTRY-2158 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing > permission related commands. We should update notification handler in the > following files when granting privileges to user. > When authorizable changes, need to change user privileges too > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2158) Update notification handler to update privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2158: -- Status: In Progress (was: Patch Available) > Update notification handler to update privileges to user > > > Key: SENTRY-2158 > URL: https://issues.apache.org/jira/browse/SENTRY-2158 > Project: Sentry > Issue Type: New Feature > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing > permission related commands. We should update notification handler in the > following files when granting privileges to user. > When authorizable changes, need to change user privileges too > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2299) NPE In Sentry HDFS Sync Plugin
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2299:
--
Summary: NPE In Sentry HDFS Sync Plugin (was: NPE In Sentry HDFS Sync Plug)
> NPE In Sentry HDFS Sync Plugin
> --
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2299.001.patch
>
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2299:
--
Status: Patch Available (was: Open)
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2299.001.patch
>
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2299:
--
Attachment: SENTRY-2299.001.patch
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2299.001.patch
>
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2255) alter table/database set owner command can be executed only by user with proper privilege
[
https://issues.apache.org/jira/browse/SENTRY-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540724#comment-16540724
]
Na Li commented on SENTRY-2255:
---
* The hive operation code for "ALTER TABLE SET OWNER" is
ALTERTABLE_OWNER("ALTERTABLE_OWNER", null, null). The means we need to require
DB level ALL for HiveOperation.ALTERTABLE_OWNER in HiveAuthzPrivilegesMap.
* Need to check grant option in SentryStore.updateOwnerPrivilege() by calling
grantOptionCheck() to verify that the user has grant option
* Need to update grantOptionCheck to check user's directly associated
privilege, not just its roles for grant option.
> alter table/database set owner command can be executed only by user with
> proper privilege
> -
>
> Key: SENTRY-2255
> URL: https://issues.apache.org/jira/browse/SENTRY-2255
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> Need to set sentry privilege mapping to make sure it can be done only by user
> with "all with grant option" at parent obj
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540673#comment-16540673
]
Na Li commented on SENTRY-2299:
---
[~Tagar] To verify this is the case, can you enable debug in NameNode, so we
can see the details of the path and table that causes this issue? Before the
exception, you can see debug message containing "renameAuthzObject" before
exception that shows the old name and path as well as the new name and path.
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540660#comment-16540660
]
Na Li commented on SENTRY-2299:
---
[~Tagar] do you have a lot of external tables, and rename them a lot with
different paths? From following code, sentry skips creating entry for those
tables.
The most likely reason why oldEntry is null is because the entry is not created
due to its path outside of the prefix, not because some delta change was lost.
This also explains why restarting NN and getting full path snapshot from Sentry
server does not help because the entry of an external table will still be null
for rename.
Then the fix will be to check if oldEntry is null or not. If no, then move its
entry to the new path.
{code}
public Entry createAuthzObjPath(List pathElements, String authzObj)
{
Entry entry = null;
Entry prefix = findPrefixEntry(pathElements);
if (prefix != null) {
// we only create the entry if is under a prefix, else we ignore it
entry = createChild(pathElements, EntryType.AUTHZ_OBJECT, authzObj);
} else {
if (LOG.isDebugEnabled()) {
LOG.debug(String.format("%s: createAuthzObjPath(%s, %s): outside of
prefix, skipping",<- skip creating entry for external tables
this, authzObj, pathElements));
}
}
return entry;
}
{code}
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
[jira] [Commented] (SENTRY-2304) Optimize the time taken for HMS snapshot creation.
[ https://issues.apache.org/jira/browse/SENTRY-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539090#comment-16539090 ] Na Li commented on SENTRY-2304: --- If customer has 1 million tables in a thought databases, the performance difference will be quite big * Option 1: query the tables in a database per task. This requires a thousand queries. * Option 2: query all tables in all databases. This requires a single queries The performance for a single query with or without "WHERE clause" may be small, but the difference of time spent on having a thousand queries versus a single query is significant. > Optimize the time taken for HMS snapshot creation. > -- > > Key: SENTRY-2304 > URL: https://issues.apache.org/jira/browse/SENTRY-2304 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Priority: Major > > First get all the database names and their locations. > # Create Table Task for each database by providing the db name > # DB Task will get the names and locations for all the tables in that > database. > # Create Partition task for each table in that database by providing the > database and table names. > ## Table task will get the locations of all the partitions in that table. > This approach needs new API’s implemented in HMS to get the names and > locations of the databases/tables/partitions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2300) Move Permission Update due to DDL to HMS Post Event Listener
[
https://issues.apache.org/jira/browse/SENTRY-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539032#comment-16539032
]
Na Li commented on SENTRY-2300:
---
One case Sentry could run into problem when updating permission for DDL update
through notification events:
* Sentry gets full snapshot from HMS.
* During that time, customer issues table rename command. The notificaition ID
is 2.
* When full snapshot is done, the current notification ID is 10. So Sentry does
not fetch the table rename event nor process it.
* Then Sentry has permission associated with old table, not the new table.
* Following query on the renamed table will fail authorization in Sentry
There could be multiple reasons why Sentry gets full snapshot from HMS. One
reason is
* Customer decides to disable HDFS sync, disable updating permission based on
DDL update. As a result, sentry does not start HMS follower, and does not fetch
notification events
* Then customer enables HDFS sync, and enables updating permission based on DDL
update.
> Move Permission Update due to DDL to HMS Post Event Listener
>
>
> Key: SENTRY-2300
> URL: https://issues.apache.org/jira/browse/SENTRY-2300
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> There was a code in MetastorePlugin that modified Sentry privileges on table
> Create/Drop and database Create/Drop. As part of Sentry HA work we moved all
> this logic from Sentry plugin to be driven by notifications which required
> the extra synchronization between HMS and Sentry.
> It should be possible to do permission changes in the post event listener
> itself to avoid blocking for Sentry. This requires some experiments though
> because it may cause strange artifacts since at the time these DDL operations
> are done Sentry may not be aware of the current state - for example you may
> try to change permissions of a table that Sentry doesn’t know about, which
> seems to be OK.
> This update will have the following benefits:
> {code}
> * HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update
> will remove this delay
> * Sentry knows every DDL update, and therefore can update permission
> correctly. In current approach using notification processing, Sentry could
> miss updates if full snapshot is fetched from HMS, and permission is not
> updated correctly. In the case of table rename, when mission DDL update event
> because of full snapshot, sentry will not move the permissions associated
> with old table to the new table. And the authorization on queries on the
> renamed table will fail.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2300) Move Permission Update due to DDL to HMS Post Event Listener
[
https://issues.apache.org/jira/browse/SENTRY-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2300:
--
Issue Type: Bug (was: Improvement)
> Move Permission Update due to DDL to HMS Post Event Listener
>
>
> Key: SENTRY-2300
> URL: https://issues.apache.org/jira/browse/SENTRY-2300
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> There was a code in MetastorePlugin that modified Sentry privileges on table
> Create/Drop and database Create/Drop. As part of Sentry HA work we moved all
> this logic from Sentry plugin to be driven by notifications which required
> the extra synchronization between HMS and Sentry.
> It should be possible to do permission changes in the post event listener
> itself to avoid blocking for Sentry. This requires some experiments though
> because it may cause strange artifacts since at the time these DDL operations
> are done Sentry may not be aware of the current state - for example you may
> try to change permissions of a table that Sentry doesn’t know about, which
> seems to be OK.
> This update will have the following benefits:
> {code}
> * HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update
> will remove this delay
> * Sentry knows every DDL update, and therefore can update permission
> correctly. In current approach using notification processing, Sentry could
> miss updates if full snapshot is fetched from HMS, and permission is not
> updated correctly. In the case of table rename, when mission DDL update event
> because of full snapshot, sentry will not move the permissions associated
> with old table to the new table. And the authorization on queries on the
> renamed table will fail.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2300) Move Permission Update due to DDL to HMS Post Event Listener
[
https://issues.apache.org/jira/browse/SENTRY-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2300:
--
Description:
There was a code in MetastorePlugin that modified Sentry privileges on table
Create/Drop and database Create/Drop. As part of Sentry HA work we moved all
this logic from Sentry plugin to be driven by notifications which required the
extra synchronization between HMS and Sentry.
It should be possible to do permission changes in the post event listener
itself to avoid blocking for Sentry. This requires some experiments though
because it may cause strange artifacts since at the time these DDL operations
are done Sentry may not be aware of the current state - for example you may try
to change permissions of a table that Sentry doesn’t know about, which seems to
be OK.
This update will have the following benefits:
{code}
* HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update
will remove this delay
* Sentry knows every DDL update, and therefore can update permission correctly.
In current approach using notification processing, Sentry could miss updates if
full snapshot is fetched from HMS, and permission is not updated correctly. In
the case of table rename, when mission DDL update event because of full
snapshot, sentry will not move the permissions associated with old table to the
new table. And the authorization on queries on the renamed table will fail.
{code}
was:
There was a code in MetastorePlugin that modified Sentry privileges on table
Create/Drop and database Create/Drop. As part of Sentry HA work we moved all
this logic from Sentry plugin to be driven by notifications which required the
extra synchronization between HMS and Sentry.
It should be possible to do permission changes in the post event listener
itself to avoid blocking for Sentry. This requires some experiments though
because it may cause strange artifacts since at the time these DDL operations
are done Sentry may not be aware of the current state - for example you may try
to change permissions of a table that Sentry doesn’t know about, which seems to
be OK.
This update will have the following benefits:
{code}
* HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update
will remove this delay
* Sentry knows every DDL update, and therefore can update permission correctly.
In current approach using notification processing, Sentry could miss updates if
full snapshot is fetched from HMS, and permission is not updated correctly.
{code}
> Move Permission Update due to DDL to HMS Post Event Listener
>
>
> Key: SENTRY-2300
> URL: https://issues.apache.org/jira/browse/SENTRY-2300
> Project: Sentry
> Issue Type: Improvement
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> There was a code in MetastorePlugin that modified Sentry privileges on table
> Create/Drop and database Create/Drop. As part of Sentry HA work we moved all
> this logic from Sentry plugin to be driven by notifications which required
> the extra synchronization between HMS and Sentry.
> It should be possible to do permission changes in the post event listener
> itself to avoid blocking for Sentry. This requires some experiments though
> because it may cause strange artifacts since at the time these DDL operations
> are done Sentry may not be aware of the current state - for example you may
> try to change permissions of a table that Sentry doesn’t know about, which
> seems to be OK.
> This update will have the following benefits:
> {code}
> * HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update
> will remove this delay
> * Sentry knows every DDL update, and therefore can update permission
> correctly. In current approach using notification processing, Sentry could
> miss updates if full snapshot is fetched from HMS, and permission is not
> updated correctly. In the case of table rename, when mission DDL update event
> because of full snapshot, sentry will not move the permissions associated
> with old table to the new table. And the authorization on queries on the
> renamed table will fail.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (SENTRY-2159) Add e2e tests for granting owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li resolved SENTRY-2159. --- Resolution: Fixed This testing case is cover by [SENTRY-2247|https://issues.apache.org/jira/browse/SENTRY-2247]. I don't mark it as duplicate because SENTRY-2247 contains more testing cases that what's specified in this jira. > Add e2e tests for granting owner privileges > --- > > Key: SENTRY-2159 > URL: https://issues.apache.org/jira/browse/SENTRY-2159 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add e2e test cases for granting implicit owner privileges for newly > created table > The test case can > 1) Set db owner.privileges to all > 2) grant user_role create table privilege to create table on db1 > 3) user with user_role create a table table1 in db1 > 4) verify user with user_role can insert row in table1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2159) Add e2e tests for granting owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2159: -- Summary: Add e2e tests for granting owner privileges (was: Add e2e tests for granting implicit owner privileges) > Add e2e tests for granting owner privileges > --- > > Key: SENTRY-2159 > URL: https://issues.apache.org/jira/browse/SENTRY-2159 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > > Need to add e2e test cases for granting implicit owner privileges for newly > created table > The test case can > 1) Set db owner.privileges to all > 2) grant user_role create table privilege to create table on db1 > 3) user with user_role create a table table1 in db1 > 4) verify user with user_role can insert row in table1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SENTRY-2291) Notification Processor should update user privileges for path notification events
[ https://issues.apache.org/jira/browse/SENTRY-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538929#comment-16538929 ] Na Li edited comment on SENTRY-2291 at 7/10/18 4:50 PM: This issue is fixed by [SENTRY-2251|https://issues.apache.org/jira/browse/SENTRY-2251] and [SENTRY-2247|https://issues.apache.org/jira/browse/SENTRY-2247] was (Author: linaataustin): This issue is fixed by [SENTRY-2251|https://issues.apache.org/jira/browse/SENTRY-2251] > Notification Processor should update user privileges for path notification > events > - > > Key: SENTRY-2291 > URL: https://issues.apache.org/jira/browse/SENTRY-2291 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > > Right now, notification processor only update role privileges for path > notification events. We need to update user privileges as well. > For example, when a table is dropped, its owner privilege should be dropped > regardless the owner is user or role. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SENTRY-2291) Notification Processor should update user privileges for path notification events
[ https://issues.apache.org/jira/browse/SENTRY-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li resolved SENTRY-2291. --- Resolution: Fixed This issue is fixed by [SENTRY-2251|https://issues.apache.org/jira/browse/SENTRY-2251] > Notification Processor should update user privileges for path notification > events > - > > Key: SENTRY-2291 > URL: https://issues.apache.org/jira/browse/SENTRY-2291 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > > Right now, notification processor only update role privileges for path > notification events. We need to update user privileges as well. > For example, when a table is dropped, its owner privilege should be dropped > regardless the owner is user or role. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2305) Optimize time taken for persistence HMS snapshot
[ https://issues.apache.org/jira/browse/SENTRY-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538920#comment-16538920 ] Na Li commented on SENTRY-2305: --- [~kkalyan] Do we have data on how bad the performance is for persisting HMS snapshot if we have SENTRY-2306 fix. > Optimize time taken for persistence HMS snapshot > - > > Key: SENTRY-2305 > URL: https://issues.apache.org/jira/browse/SENTRY-2305 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > > There are couple of options > # Break the total snapshot into to batches and persist all of them in > parallel in different transactions. As sentry uses repeatable_read isolation > level we should be able to have parallel writes on the same table. This bring > an issue if there is a failure in persisting any of the batches. This > approach needs additional logic of cleaning the partially persisted snapshot. > I’m evaluating this option. > ** *Result:* Initial results are promising. Time to persist the snapshot came > down by 60%. > # Try disabling L1 Cache for persisting the snapshot. > # Try persisting the snapshot entries sequentially in separate transactions. > As transactions which commit huge data might take longer as they take a lot > of CPU cycles to keep the rollback log up to date. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2304) Optimize the time taken for HMS snapshot creation.
[ https://issues.apache.org/jira/browse/SENTRY-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538913#comment-16538913 ] Na Li commented on SENTRY-2304: --- Instead of asking table info per db, it is much faster for HMS to provide all table info for all DB. It is fetching with filter versus bulk fetching. If we are serious about improving performance, we should ask HMS to provide API to return result via bulk fetching. And HMS client can organize the result using DB names. The same approach applies to partition. The HMS client can pass filter for partition to HMS server to reduce the amount of data sending back to HMS client. Right now, the filtering is done at HMS client, and could result in large network traffic. > Optimize the time taken for HMS snapshot creation. > -- > > Key: SENTRY-2304 > URL: https://issues.apache.org/jira/browse/SENTRY-2304 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Priority: Major > > First get all the database names and their locations. > # Create Table Task for each database by providing the db name > # DB Task will get the names and locations for all the tables in that > database. > # Create Partition task for each table in that database by providing the > database and table names. > ## Table task will get the locations of all the partitions in that table. > This approach needs new API’s implemented in HMS to get the names and > locations of the databases/tables/partitions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537628#comment-16537628
]
Na Li commented on SENTRY-2299:
---
[~Tagar] CDH5.14.2 contains many fixes from sentry 2.0, so it is not
corresponding to Sentry 1.5.1.
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537624#comment-16537624
]
Na Li commented on SENTRY-2299:
---
[~Tagar] are you having this issue now? How often do you have it?
The work around is to restart NameNode service. It will force a full snapshot
and then the issue will be gone for a while.
I am looking for ways to identify where the "create table" delta update is
lost. It would be great if you can provide more insight.
I do think this is related to HA because how sentry gets path update is
changed.
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message
[jira] [Assigned] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
[
https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li reassigned SENTRY-2299:
-
Assignee: Na Li
> NPE In Sentry HDFS Sync Plug
>
>
> Key: SENTRY-2299
> URL: https://issues.apache.org/jira/browse/SENTRY-2299
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> Sentry HDFS ACL synchronization stopped working and throws
> NullPointerException. The HDFS logs showed repeating errors like the
> following:
> {code}
> 11:16:15.743 AM WARNSentryAuthorizationInfo
> Failed to update, will retry in [3]ms, error:
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at
> org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at
> org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors.
> The issue stopped occurring, apparently not through any specific user
> intervention. (The customer tried manually failing over the active NameNode,
> with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was
> lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
> exception.
> {code}
> void renameAuthzObject(String oldName, List> oldPathElems,
> String newName, List> newPathElems) {
> if (LOG.isDebugEnabled()) {
> LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> }
> if (oldPathElems == null || oldPathElems.isEmpty() ||
> newPathElems == null || newPathElems.isEmpty() ||
> newName == null || newName.equals(oldName)) {
> LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
> ": invalid inputs, skipping",
> this, oldName, assemblePaths(oldPathElems), newName,
> assemblePaths(newPathElems)));
> return;
> }
> // if oldPath == newPath, that is path has not changed as part of rename
> and hence new table
> // needs to have old paths => new_table.add(old_table_partition_paths)
> List oldPathElements = oldPathElems.get(0);
> List newPathElements = newPathElems.get(0);
> if (!oldPathElements.equals(newPathElements)) {
> Entry oldEntry = root.find(oldPathElements.toArray(new String[0]),
> false);
> Entry newParent = root.createParent(newPathElements);
> oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
> 1)); -> oldEntry is null
> }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance
> is really low
> 2. The delta change is lost from sentry server to name node plugin. The
> chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead,
> throw exception, which will be caught by its caller and causes sentry client
> at name node plugin gets path full snapshot from sentry server.
> 2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537530#comment-16537530 ] Na Li commented on SENTRY-2247: --- This update no only adds e2e tests for owner privilege, it also fixes the following issues, so those tests can be enabled and pass * The server name of owner privilege was not set at server side * When dropping privilege, if a privilege has no role, it is not dropped * When dropping privilege, if it has grant option being true, it is not dropped. * When renaming table, if its privilege has grant option being true, it is not transferred to new table > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch, SENTRY-2247.002.patch, > SENTRY-2247.003.patch, SENTRY-2247.004.patch, SENTRY-2247.005.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2300) Move Permission Update due to DDL to HMS Post Event Listener
Na Li created SENTRY-2300:
-
Summary: Move Permission Update due to DDL to HMS Post Event
Listener
Key: SENTRY-2300
URL: https://issues.apache.org/jira/browse/SENTRY-2300
Project: Sentry
Issue Type: Improvement
Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
Assignee: Na Li
There was a code in MetastorePlugin that modified Sentry privileges on table
Create/Drop and database Create/Drop. As part of Sentry HA work we moved all
this logic from Sentry plugin to be driven by notifications which required the
extra synchronization between HMS and Sentry.
It should be possible to do permission changes in the post event listener
itself to avoid blocking for Sentry. This requires some experiments though
because it may cause strange artifacts since at the time these DDL operations
are done Sentry may not be aware of the current state - for example you may try
to change permissions of a table that Sentry doesn’t know about, which seems to
be OK.
This update will have the following benefits:
{code}
* HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update
will remove this delay
* Sentry knows every DDL update, and therefore can update permission correctly.
In current approach using notification processing, Sentry could miss updates if
full snapshot is fetched from HMS, and permission is not updated correctly.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (SENTRY-2299) NPE In Sentry HDFS Sync Plug
Na Li created SENTRY-2299:
-
Summary: NPE In Sentry HDFS Sync Plug
Key: SENTRY-2299
URL: https://issues.apache.org/jira/browse/SENTRY-2299
Project: Sentry
Issue Type: Bug
Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
Sentry HDFS ACL synchronization stopped working and throws
NullPointerException. The HDFS logs showed repeating errors like the following:
{code}
11:16:15.743 AM WARNSentryAuthorizationInfo
Failed to update, will retry in [3]ms, error:
java.lang.NullPointerException
at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
at
org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
at
org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
at
org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
at
org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
at
org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
The customer checked the Sentry logs and didn't see any corresponding errors.
The issue stopped occurring, apparently not through any specific user
intervention. (The customer tried manually failing over the active NameNode,
with no change.)
{code}
Arjun mentioned the reason is that some delta update from sentry server was
lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null
exception.
{code}
void renameAuthzObject(String oldName, List> oldPathElems,
String newName, List> newPathElems) {
if (LOG.isDebugEnabled()) {
LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
this, oldName, assemblePaths(oldPathElems), newName,
assemblePaths(newPathElems)));
}
if (oldPathElems == null || oldPathElems.isEmpty() ||
newPathElems == null || newPathElems.isEmpty() ||
newName == null || newName.equals(oldName)) {
LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
": invalid inputs, skipping",
this, oldName, assemblePaths(oldPathElems), newName,
assemblePaths(newPathElems)));
return;
}
// if oldPath == newPath, that is path has not changed as part of rename
and hence new table
// needs to have old paths => new_table.add(old_table_partition_paths)
List oldPathElements = oldPathElems.get(0);
List newPathElements = newPathElems.get(0);
if (!oldPathElements.equals(newPathElements)) {
Entry oldEntry = root.find(oldPathElements.toArray(new String[0]), false);
Entry newParent = root.createParent(newPathElements);
oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() -
1)); -> oldEntry is null
}
{code}
There are several possible reasons why some delta changes are lost.
{code}
1. Sentry server does not save the rename update as delta update. The chance is
really low
2. The delta change is lost from sentry server to name node plugin. The chance
is also low
3. When applying delta change about old entry, it is lost
{code}
The fix for this issue
1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead, throw
exception, which will be caught by its caller and causes sentry client at name
node plugin gets path full snapshot from sentry server.
2. Find out why the oldEntry is null and fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch, SENTRY-2247.002.patch, > SENTRY-2247.003.patch, SENTRY-2247.004.patch, SENTRY-2247.005.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Attachment: SENTRY-2247.005.patch > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch, SENTRY-2247.002.patch, > SENTRY-2247.003.patch, SENTRY-2247.004.patch, SENTRY-2247.005.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Attachment: SENTRY-2247.004.patch > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch, SENTRY-2247.002.patch, > SENTRY-2247.003.patch, SENTRY-2247.004.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Attachment: SENTRY-2247.003.patch > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch, SENTRY-2247.002.patch, > SENTRY-2247.003.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2295) Owner privileges should not be granted to sentry admin users
[ https://issues.apache.org/jira/browse/SENTRY-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531923#comment-16531923 ] Na Li commented on SENTRY-2295: --- [~kkalyan] We need to document this behavior for user and testing. > Owner privileges should not be granted to sentry admin users > > > Key: SENTRY-2295 > URL: https://issues.apache.org/jira/browse/SENTRY-2295 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2295.001.patch, SENTRY-2295.002.patch > > > There are two cases that should be considered. > # When an admin user is created a database or table, owner privilege should > not be added. > # When the ownership is transferred to an admin user, existing owner > privilege should be revoked but new one should not be added. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (SENTRY-2274) Grant and revoke owner privileges based on HMS updates(server-side)
[ https://issues.apache.org/jira/browse/SENTRY-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li resolved SENTRY-2274. --- Resolution: Fixed [~kkalyan] Thanks for your explanation. There is no issues by looking into the implementation of those functions. I saw those issues by looking at code, not by testing > Grant and revoke owner privileges based on HMS updates(server-side) > --- > > Key: SENTRY-2274 > URL: https://issues.apache.org/jira/browse/SENTRY-2274 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2274.001.patch, SENTRY-2274.002.patch > > > Sentry has SentrySyncHMSNotificationsPostEventListener which is added as a > post listener in HMS. This listener should be extended to get the owner > information of tables and databases. > Based on these notifications owner privileges are granted/revoked. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (SENTRY-2274) Grant and revoke owner privileges based on HMS updates(server-side)
[ https://issues.apache.org/jira/browse/SENTRY-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li reopened SENTRY-2274: --- > Grant and revoke owner privileges based on HMS updates(server-side) > --- > > Key: SENTRY-2274 > URL: https://issues.apache.org/jira/browse/SENTRY-2274 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2274.001.patch, SENTRY-2274.002.patch > > > Sentry has SentrySyncHMSNotificationsPostEventListener which is added as a > post listener in HMS. This listener should be extended to get the owner > information of tables and databases. > Based on these notifications owner privileges are granted/revoked. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2274) Grant and revoke owner privileges based on HMS updates(server-side)
[
https://issues.apache.org/jira/browse/SENTRY-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531517#comment-16531517
]
Na Li commented on SENTRY-2274:
---
[~kkalyan] There are multiple issues in the following function
* It does not find owner privilege that has grant option. It is possible that
right now the grant option is disabled, but existing owner privileges have that
enabled. Need to find owner privilege that has grant option and without grant
option
* If user is stale (no associated privileges), it should be removed, not
persisted.
* If the previous owner is a role, need to update role as well, not just user.
{code}
public void revokeOwnerPrivilegesCore(PersistenceManager pm, final
TSentryAuthorizable tAuthorizable)
throws Exception{
TSentryPrivilege tOwnerPrivilege = toSentryPrivilege(tAuthorizable);
tOwnerPrivilege.setAction(AccessConstants.OWNER);
// Finding owner privileges and removing them.
List mOwnerPrivileges =
getMSentryPrivileges(tOwnerPrivilege, pm);
for(MSentryPrivilege mOwnerPriv : mOwnerPrivileges) {
Set users;
users = mOwnerPriv.getUsers();
// Making sure of removing stale users.
for (MSentryUser user : users) {
user.removePrivilege(mOwnerPriv);
persistEntity(pm, SentryEntityType.USER, user);
}
}
pm.deletePersistentAll(mOwnerPrivileges);
}
{code}
Can you fix the above issues?
> Grant and revoke owner privileges based on HMS updates(server-side)
> ---
>
> Key: SENTRY-2274
> URL: https://issues.apache.org/jira/browse/SENTRY-2274
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-2274.001.patch, SENTRY-2274.002.patch
>
>
> Sentry has SentrySyncHMSNotificationsPostEventListener which is added as a
> post listener in HMS. This listener should be extended to get the owner
> information of tables and databases.
> Based on these notifications owner privileges are granted/revoked.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Attachment: SENTRY-2247.002.patch > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch, SENTRY-2247.002.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2293) Fix logging parameters on SentryHDFSServiceProcessor
[
https://issues.apache.org/jira/browse/SENTRY-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2293:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
[~arjunmishra13] Thanks for your contribution
> Fix logging parameters on SentryHDFSServiceProcessor
>
>
> Key: SENTRY-2293
> URL: https://issues.apache.org/jira/browse/SENTRY-2293
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.0.1
>Reporter: Arjun Mishra
>Assignee: Arjun Mishra
>Priority: Major
> Attachments: SENTRY-2293.01.patch
>
>
> SentryHDFSServiceProcessor method getPathsUpdatesFrom has a log message where
> parameters are swapped. See below and fix it
> {noformat}
> LOGGER.debug("PATHS updates requested from HDFS [ImgNum={}, SeqNum={}]",
> request.getPathSeqNum(), request.getPathImgNum());
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Attachment: SENTRY-2247.001.patch > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2247) Add e2e tests to verify owner privileges
[ https://issues.apache.org/jira/browse/SENTRY-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2247: -- Status: Patch Available (was: Open) > Add e2e tests to verify owner privileges > > > Key: SENTRY-2247 > URL: https://issues.apache.org/jira/browse/SENTRY-2247 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2247.001.patch > > > Add e2e tests to verify owner privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SENTRY-2292) Separate Processing of Notifications to a separate thread
[ https://issues.apache.org/jira/browse/SENTRY-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528161#comment-16528161 ] Na Li edited comment on SENTRY-2292 at 6/29/18 8:29 PM: [~arjunmishra13] This may bring sentry to unknown state for path info and cause performance issue. 1) For example, when sentry is waiting for full snapshot which is starting with Notification_ID = 3, there are more events with Notification_ID = 4 of creating table table_1, and its associated permission update If sentry uses one thread for full snapshot and another thread for getting notification events and process them (as you proposed), the following could happen depending on timing of these two threads * sentry may save updated from notification 4 and its associated permission first with thread_2, * then persist the full snapshot with notification 3 with thread_1. The snapshot ID increases * Then next pulling interval, it won't get notification 4 as max notification ID is 4 * For the new snapshot ID, there is no table_1, but its permission exists 2) Sentry had assumption that there is one thread for getting and saving path info from HMS. Two thread would cause concurrency issue For multiple sentry threads to consume those notifications Sentry server need to add code to detect the same notification is already processed and saved in sentry DB, so it won't process and save the same notifications in sentry DB. Then, it is fine for multiple threads to consume those notifications. Without that code change, exception will occur when multiple sentry threads get same notifications and save them in sentry DB. The table SENTRY_PATH_CHANGE contains unique index for the field NOTIFICATION_HASH. Then transaction retry will happen for long time. was (Author: linaataustin): [~arjunmishra13] This may bring sentry to unknown state for path info and cause performance issue. 1) For example, when sentry is waiting for full snapshot which is starting with Notification_ID = 3, there are more events with Notification_ID = 4 of creating table table_1, and its associated permission update If sentry uses one thread for full snapshot and another thread for getting notification events and process them (as you proposed), the following could happen depending on timing of these two threads * sentry may save updated from notification 4 and its associated permission first with thread_2, * then persist the full snapshot with notification 3 with thread_1. * Then next pulling interval, it gets notification 4, and the permission is removed for creating table if syncStoreOnCreate = true (thinking that permission is associated with non-exist table) 2) Sentry had assumption that there is one thread for getting and saving path info from HMS. Two thread would cause concurrency issue For multiple sentry threads to consume those notifications Sentry server need to add code to detect the same notification is already processed and saved in sentry DB, so it won't process and save the same notifications in sentry DB. Then, it is fine for multiple threads to consume those notifications. Without that code change, exception will occur when multiple sentry threads get same notifications and save them in sentry DB. The table SENTRY_PATH_CHANGE contains unique index for the field NOTIFICATION_HASH. Then transaction retry will happen for long time. > Separate Processing of Notifications to a separate thread > - > > Key: SENTRY-2292 > URL: https://issues.apache.org/jira/browse/SENTRY-2292 > Project: Sentry > Issue Type: Improvement > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > > When fetching full update, processing notifications are held which blocks HMS > threads. Separating them to different threads will relieve HMS threads -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2292) Separate Processing of Notifications to a separate thread
[ https://issues.apache.org/jira/browse/SENTRY-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528161#comment-16528161 ] Na Li commented on SENTRY-2292: --- [~arjunmishra13] This may bring sentry to unknown state for path info and cause performance issue. 1) For example, when sentry is waiting for full snapshot which is starting with Notification_ID = 3, there are more events with Notification_ID = 4 of creating table table_1, and its associated permission update If sentry uses one thread for full snapshot and another thread for getting notification events and process them (as you proposed), the following could happen depending on timing of these two threads * sentry may save updated from notification 4 and its associated permission first with thread_2, * then persist the full snapshot with notification 3 with thread_1. * Then next pulling interval, it gets notification 4, and the permission is removed for creating table if syncStoreOnCreate = true (thinking that permission is associated with non-exist table) 2) Sentry had assumption that there is one thread for getting and saving path info from HMS. Two thread would cause concurrency issue For multiple sentry threads to consume those notifications Sentry server need to add code to detect the same notification is already processed and saved in sentry DB, so it won't process and save the same notifications in sentry DB. Then, it is fine for multiple threads to consume those notifications. Without that code change, exception will occur when multiple sentry threads get same notifications and save them in sentry DB. The table SENTRY_PATH_CHANGE contains unique index for the field NOTIFICATION_HASH. Then transaction retry will happen for long time. > Separate Processing of Notifications to a separate thread > - > > Key: SENTRY-2292 > URL: https://issues.apache.org/jira/browse/SENTRY-2292 > Project: Sentry > Issue Type: Improvement > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > > When fetching full update, processing notifications are held which blocks HMS > threads. Separating them to different threads will relieve HMS threads -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2291) Notification Processor should update user privileges for path notification events
[ https://issues.apache.org/jira/browse/SENTRY-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2291: -- Description: Right now, notification processor only update role privileges for path notification events. We need to update user privileges as well. For example, when a table is dropped, its owner privilege should be dropped regardless the owner is user or role. was: When a table is renamed, its associated user privileges should be moved to be associated with the renamed table. This is related to owner privilege as owner could be a user. > Notification Processor should update user privileges for path notification > events > - > > Key: SENTRY-2291 > URL: https://issues.apache.org/jira/browse/SENTRY-2291 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > > Right now, notification processor only update role privileges for path > notification events. We need to update user privileges as well. > For example, when a table is dropped, its owner privilege should be dropped > regardless the owner is user or role. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2291) Notification Processor should update user privileges for path notification events
[ https://issues.apache.org/jira/browse/SENTRY-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2291: -- Summary: Notification Processor should update user privileges for path notification events (was: Alter Table Rename should update user privilege to renamed table) > Notification Processor should update user privileges for path notification > events > - > > Key: SENTRY-2291 > URL: https://issues.apache.org/jira/browse/SENTRY-2291 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > > When a table is renamed, its associated user privileges should be moved to be > associated with the renamed table. > This is related to owner privilege as owner could be a user. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2291) Alter Table Rename should update user privilege to renamed table
Na Li created SENTRY-2291: - Summary: Alter Table Rename should update user privilege to renamed table Key: SENTRY-2291 URL: https://issues.apache.org/jira/browse/SENTRY-2291 Project: Sentry Issue Type: Sub-task Components: Sentry Affects Versions: 2.1.0 Reporter: Na Li Assignee: Na Li When a table is renamed, its associated user privileges should be moved to be associated with the renamed table. This is related to owner privilege as owner could be a user. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2268) Review the required privileges for DDL commands
[
https://issues.apache.org/jira/browse/SENTRY-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li reassigned SENTRY-2268:
-
Assignee: Na Li
> Review the required privileges for DDL commands
> ---
>
> Key: SENTRY-2268
> URL: https://issues.apache.org/jira/browse/SENTRY-2268
> Project: Sentry
> Issue Type: Task
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> The privileges required for DDL commands are listed in
> HiveAuthzPrivilegesMap.
> {code}
> addOutputObjectPriviledge(AuthorizableType.Table,
> EnumSet.of(DBModelAction.INSERT, DBModelAction.ALTER))
> {code}
> means the required output privileges is table level insert OR alter.
> {code}
> addOutputObjectPriviledge(AuthorizableType.Table,
> EnumSet.of(DBModelAction.INSERT)).
> addOutputObjectPriviledge(AuthorizableType.Table,
> EnumSet.of(DBModelAction.ALTER))
> {code}
> means the required output privileges is table level insert AND alter.
> We need to review the privileges to see if they are defined correctly. I
> suspect multiple definitions want to have privileges with AND, but end up
> getting privileges with OR.
> We should also check if the privilege level is correct. for example, "insert"
> is table level privilege. It does not make sense to require database level
> "insert".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2259) SQL CONSTRAINT name for user privilege is too long for Oracle 11.2
[
https://issues.apache.org/jira/browse/SENTRY-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2259:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> SQL CONSTRAINT name for user privilege is too long for Oracle 11.2
> --
>
> Key: SENTRY-2259
> URL: https://issues.apache.org/jira/browse/SENTRY-2259
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2259.001.patch
>
>
> Prior to Oracle version 12.2, identifiers are not allowed to exceed 30
> characters in length
> Names must be from 1 to 30 bytes long with these exceptions:
> Names of databases are limited to 8 bytes.
> Names of database links can be as long as 128 bytes.
> The following index name is 32 bytes, and cause exception "ORA-00972:
> identifier is too long" on oracle DB 11.2.
> SENTRY_USER_DB_PRIVILEGE_MAP_PK
> SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
> {code}
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
> ADD CONSTRAINT SENTRY_USER_DB_PRIVILEGE_MAP_PK PRIMARY KEY
> (USER_ID,DB_PRIVILEGE_ID);
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
>ADD CONSTRAINT SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
>FOREIGN KEY (DB_PRIVILEGE_ID) REFERENCES
> SENTRY_DB_PRIVILEGE(DB_PRIVILEGE_ID);
> {code}
> Need to change the constrain name to be smaller than 30 characters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2242) Add schema changes to limit one one user as owner privilege per object
[
https://issues.apache.org/jira/browse/SENTRY-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527998#comment-16527998
]
Na Li commented on SENTRY-2242:
---
[~kkalyan][~spena] Should we close this issue as not to fix? when owner
changes, we limit in application that there is only one owner for an object?
> Add schema changes to limit one one user as owner privilege per object
> --
>
> Key: SENTRY-2242
> URL: https://issues.apache.org/jira/browse/SENTRY-2242
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
>
> Currently user<-> privileges is implemented to handle privileges to owners of
> the respective objects. There can not be more than one owner to single object
> (database/table) so restriction should be added either at the application or
> the database schema to prevent it from happening.
> I feel schema change is the best way to do it for now as there is no plan to
> implement user privileges in near future. In future when user privileges
> feature is implemented this change could be reverted and restriction can be
> added at application.
> *Current Schema:*
> {noformat}
> ALTER TABLE `SENTRY_USER_DB_PRIVILEGE_MAP`
> ADD CONSTRAINT `SENTRY_USER_DB_PRIVILEGE_MAP_PK` PRIMARY KEY
> (`USER_ID`,`DB_PRIVILEGE_ID`);
> {noformat}
> *Proposed Solution:*
> {noformat}
> ALTER TABLE `SENTRY_USER_DB_PRIVILEGE_MAP`
> ADD CONSTRAINT `SENTRY_USER_DB_PRIVILEGE_MAP_PK` PRIMARY KEY
> (`DB_PRIVILEGE_ID`);
> {noformat}
> With this change DB_PRIVILEGE_ID would be the primary key and this restrict
> same privilege to be granted to another user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2289) Tests in TestHDFSIntegrationAdvanced fail from time to time
[ https://issues.apache.org/jira/browse/SENTRY-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2289: -- Status: Patch Available (was: Open) > Tests in TestHDFSIntegrationAdvanced fail from time to time > --- > > Key: SENTRY-2289 > URL: https://issues.apache.org/jira/browse/SENTRY-2289 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2289.001.patch > > > The test cases fail from time to time, but works if run individually. > It is likely due to the async nature of HDFS sync from sentry, so when next > test is running, old test clean up removes the added DB, table, role or > privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2289) Tests in TestHDFSIntegrationAdvanced fail from time to time
[ https://issues.apache.org/jira/browse/SENTRY-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2289: -- Attachment: SENTRY-2289.001.patch > Tests in TestHDFSIntegrationAdvanced fail from time to time > --- > > Key: SENTRY-2289 > URL: https://issues.apache.org/jira/browse/SENTRY-2289 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2289.001.patch > > > The test cases fail from time to time, but works if run individually. > It is likely due to the async nature of HDFS sync from sentry, so when next > test is running, old test clean up removes the added DB, table, role or > privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2289) Tests in TestHDFSIntegrationAdvanced fail from time to time
Na Li created SENTRY-2289: - Summary: Tests in TestHDFSIntegrationAdvanced fail from time to time Key: SENTRY-2289 URL: https://issues.apache.org/jira/browse/SENTRY-2289 Project: Sentry Issue Type: Bug Components: Sentry Affects Versions: 2.1.0 Reporter: Na Li Assignee: Na Li The test cases fail from time to time, but works if run individually. It is likely due to the async nature of HDFS sync from sentry, so when next test is running, old test clean up removes the added DB, table, role or privileges. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) [~spena][~kk.nit...@gmail.com] Thanks for review > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Fix For: 2.1.0 > > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch, > SENTRY-2280.003.patch, SENTRY-2280.004.patch, SENTRY-2280.005.patch, > SENTRY-2280.006.patch, SENTRY-2280.006.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.006.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch, > SENTRY-2280.003.patch, SENTRY-2280.004.patch, SENTRY-2280.005.patch, > SENTRY-2280.006.patch, SENTRY-2280.006.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.006.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch, > SENTRY-2280.003.patch, SENTRY-2280.004.patch, SENTRY-2280.005.patch, > SENTRY-2280.006.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.005.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch, > SENTRY-2280.003.patch, SENTRY-2280.004.patch, SENTRY-2280.005.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.004.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch, > SENTRY-2280.003.patch, SENTRY-2280.004.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.003.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch, > SENTRY-2280.003.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.002.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch, SENTRY-2280.002.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Issue Type: Sub-task (was: Bug) Parent: SENTRY-2151 > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Issue Type: Sub-task (was: Bug) Parent: SENTRY-2151 > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Fix Version/s: 2.1.0 > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Status: Patch Available (was: Open) > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
[ https://issues.apache.org/jira/browse/SENTRY-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2280: -- Attachment: SENTRY-2280.001.patch > The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > --- > > Key: SENTRY-2280 > URL: https://issues.apache.org/jira/browse/SENTRY-2280 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Critical > Attachments: SENTRY-2280.001.patch > > > When running e2e test > TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found > the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is > null > There are multiple issues in fix from "SENTRY-2243: Extend the thrift > definition for policy service to learn owner information" > 1. The exception was thrown because the protocol_version was not set > 2. TSentryAuthorizable.server was not set > 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table > rename event > As a result, the request received by server is null since thrift at server > side cannot re-construct a valid request of type TSentryHmsEventNotification > Once we fix the above issues, we need to make sure > SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in > "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Resolution: Fixed Status: Resolved (was: Patch Available) > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523199#comment-16523199 ] Na Li commented on SENTRY-2264: --- [~spena] owner privilege related issues are tracked in SENTRY-2280 > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2280) The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null
Na Li created SENTRY-2280: - Summary: The request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null Key: SENTRY-2280 URL: https://issues.apache.org/jira/browse/SENTRY-2280 Project: Sentry Issue Type: Bug Components: Sentry Affects Versions: 2.1.0 Reporter: Na Li Assignee: Na Li When running e2e test TestDbPrivilegeCleanupOnDrop.testRenameTablesWithinDBSinglePrivilege, I found the request received in SentryPolicyStoreProcessor.sentry_notify_hms_event is null There are multiple issues in fix from "SENTRY-2243: Extend the thrift definition for policy service to learn owner information" 1. The exception was thrown because the protocol_version was not set 2. TSentryAuthorizable.server was not set 3. TSentryHmsEventNotification.ownerType and ownerName are not set for table rename event As a result, the request received by server is null since thrift at server side cannot re-construct a valid request of type TSentryHmsEventNotification Once we fix the above issues, we need to make sure SentryPolicyStoreProcessor.sentry_notify_hms_event does not transfer owner in "alter table rename" event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523176#comment-16523176 ] Na Li commented on SENTRY-2264: --- The failed test is not related to my fix. It is more likely caused by timing of the test. > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Attachment: SENTRY-2264.004.patch > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523163#comment-16523163 ] Na Li commented on SENTRY-2264: --- [~spena] The ALL privilege will be required for source table to avoid elevating privilege when renaming table cross databases. Only create privilege is required on destination database. Can we handle owner privilege related issue in another jira? This one focuses on tighten privilege requirement for source table for table rename. > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Attachment: SENTRY-2264.004.patch > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch, SENTRY-2264.004.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Description: After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "ALL" at source database level "CREATE" at destination. The reason we don't require "alter, insert" for destination DB is that "alter" and "insert" is table level privileges and when "alter table rename" command is executed, there is no table in destination DB. So we cannot enforce these table level privileges. Therefore the only change is add table-level "ALL" privilege in required input privileges to avoid elevate privilege by moving table cross DB was: After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "ALL" at source database level "CREATE" at destination. The reason we don't require "alter, insert" for destination DB is that "alter" and "insert" is table level privileges and when "alter table rename" command is executed, there is no table in destination DB. So we cannot enforce these table level privileges. Therefore the only change is add table-level "select" privilege in required input privileges > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "ALL" privilege in required input privileges to avoid elevate > privilege by moving table cross DB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Description: After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "ALL" at source database level "CREATE" at destination. The reason we don't require "alter, insert" for destination DB is that "alter" and "insert" is table level privileges and when "alter table rename" command is executed, there is no table in destination DB. So we cannot enforce these table level privileges. Therefore the only change is add table-level "select" privilege in required input privileges was: After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "SELECT" and database level "DELECT" at source database level "CREATE" at destination. The reason we don't require "alter, insert" for destination DB is that "alter" and "insert" is table level privileges and when "alter table rename" command is executed, there is no table in destination DB. So we cannot enforce these table level privileges. Therefore the only change is add table-level "select" privilege in required input privileges > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "ALL" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "select" privilege in required input privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Attachment: SENTRY-2264.003.patch > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch, > SENTRY-2264.003.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "SELECT" and database level "DELECT" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "select" privilege in required input privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2268) Review the required privileges for DDL commands
[
https://issues.apache.org/jira/browse/SENTRY-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2268:
--
Description:
The privileges required for DDL commands are listed in HiveAuthzPrivilegesMap.
{code}
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.INSERT, DBModelAction.ALTER))
{code}
means the required output privileges is table level insert OR alter.
{code}
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.INSERT)).
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.ALTER))
{code}
means the required output privileges is table level insert AND alter.
We need to review the privileges to see if they are defined correctly. I
suspect multiple definitions want to have privileges with AND, but end up
getting privileges with OR.
We should also check if the privilege level is correct. for example, "insert"
is table level privilege. It does not make sense to require database level
"insert".
was:
The privileges required for DDL commands are listed in HiveAuthzPrivilegesMap.
{code}
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.INSERT, DBModelAction.ALTER))
{code}
means the required output privileges is table level insert OR alter.
{code}
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.INSERT)).
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.ALTER))
{code}
means the required output privileges is table level insert AND alter.
We need to review the privileges to see if they are defined correctly. I
suspect multiple definitions want to have privileges with AND, but end up
getting privileges with OR.
> Review the required privileges for DDL commands
> ---
>
> Key: SENTRY-2268
> URL: https://issues.apache.org/jira/browse/SENTRY-2268
> Project: Sentry
> Issue Type: Task
>Reporter: Na Li
>Priority: Major
>
> The privileges required for DDL commands are listed in
> HiveAuthzPrivilegesMap.
> {code}
> addOutputObjectPriviledge(AuthorizableType.Table,
> EnumSet.of(DBModelAction.INSERT, DBModelAction.ALTER))
> {code}
> means the required output privileges is table level insert OR alter.
> {code}
> addOutputObjectPriviledge(AuthorizableType.Table,
> EnumSet.of(DBModelAction.INSERT)).
> addOutputObjectPriviledge(AuthorizableType.Table,
> EnumSet.of(DBModelAction.ALTER))
> {code}
> means the required output privileges is table level insert AND alter.
> We need to review the privileges to see if they are defined correctly. I
> suspect multiple definitions want to have privileges with AND, but end up
> getting privileges with OR.
> We should also check if the privilege level is correct. for example, "insert"
> is table level privilege. It does not make sense to require database level
> "insert".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Description: After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "SELECT" and database level "DELECT" at source database level "CREATE" at destination. The reason we don't require "alter, insert" for destination DB is that "alter" and "insert" is table level privileges and when "alter table rename" command is executed, there is no table in destination DB. So we cannot enforce these table level privileges. Therefore the only change is add table-level "select" privilege in required input privileges was: After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "SELECT" and database level "DELECT" at source database level "CREATE, INSERT, ALTER" at destination. > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "SELECT" and database level "DELECT" at source > database level "CREATE" at destination. > The reason we don't require "alter, insert" for destination DB is that > "alter" and "insert" is table level privileges and when "alter table rename" > command is executed, there is no table in destination DB. So we cannot > enforce these table level privileges. Therefore the only change is add > table-level "select" privilege in required input privileges -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Attachment: SENTRY-2264.002.patch > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch, SENTRY-2264.002.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "SELECT" and database level "DELECT" at source > database level "CREATE, INSERT, ALTER" at destination. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2268) Review the required privileges for DDL commands
Na Li created SENTRY-2268:
-
Summary: Review the required privileges for DDL commands
Key: SENTRY-2268
URL: https://issues.apache.org/jira/browse/SENTRY-2268
Project: Sentry
Issue Type: Bug
Reporter: Na Li
The privileges required for DDL commands are listed in HiveAuthzPrivilegesMap.
{code}
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.INSERT, DBModelAction.ALTER))
{code}
means the required output privileges is table level insert OR alter.
{code}
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.INSERT)).
addOutputObjectPriviledge(AuthorizableType.Table,
EnumSet.of(DBModelAction.ALTER))
{code}
means the required output privileges is table level insert AND alter.
We need to review the privileges to see if they are defined correctly. I
suspect multiple definitions want to have privileges with AND, but end up
getting privileges with OR.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2265) Translate owner privilege in sentry binding for authorization
[
https://issues.apache.org/jira/browse/SENTRY-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2265:
--
Description:
When authorization is done, sentry binding should translate owner privilege to
"ALL" before using it.
In CommonPrivilege, need to add Owner privilege and treat it as ALL privilege
for authorization
{code}
private boolean impliesResource(ImplyMethodType implyMethodType, String
policyValue, String requestValue) {
// wildcard support, "*", "+", "all"("+" and "all" are for backward
compatibility) are represented as wildcard
// if requestValue is wildcard, means privilege request is to match with
any value of given resource
if (SentryConstants.RESOURCE_WILDCARD_VALUE.equals(policyValue)
|| SentryConstants.RESOURCE_WILDCARD_VALUE.equals(requestValue)
||
SentryConstants.RESOURCE_WILDCARD_VALUE_ALL.equalsIgnoreCase(policyValue)
||
SentryConstants.RESOURCE_WILDCARD_VALUE_ALL.equalsIgnoreCase(requestValue)
||
SentryConstants.RESOURCE_WILDCARD_VALUE_SOME.equals(requestValue)) {
return true;
}
{code}
was:When authorization is done, sentry binding should translate owner
privilege to "ALL" before using it.
> Translate owner privilege in sentry binding for authorization
> -
>
> Key: SENTRY-2265
> URL: https://issues.apache.org/jira/browse/SENTRY-2265
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Major
>
> When authorization is done, sentry binding should translate owner privilege
> to "ALL" before using it.
> In CommonPrivilege, need to add Owner privilege and treat it as ALL privilege
> for authorization
> {code}
> private boolean impliesResource(ImplyMethodType implyMethodType, String
> policyValue, String requestValue) {
> // wildcard support, "*", "+", "all"("+" and "all" are for backward
> compatibility) are represented as wildcard
> // if requestValue is wildcard, means privilege request is to match with
> any value of given resource
> if (SentryConstants.RESOURCE_WILDCARD_VALUE.equals(policyValue)
> || SentryConstants.RESOURCE_WILDCARD_VALUE.equals(requestValue)
> ||
> SentryConstants.RESOURCE_WILDCARD_VALUE_ALL.equalsIgnoreCase(policyValue)
> ||
> SentryConstants.RESOURCE_WILDCARD_VALUE_ALL.equalsIgnoreCase(requestValue)
> ||
> SentryConstants.RESOURCE_WILDCARD_VALUE_SOME.equals(requestValue)) {
> return true;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Attachment: SENTRY-2264.001.patch > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "SELECT" and database level "DELECT" at source > database level "CREATE, INSERT, ALTER" at destination. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
[ https://issues.apache.org/jira/browse/SENTRY-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2264: -- Status: Patch Available (was: Open) > It is possible to elevate privileges from DROP using alter table rename > --- > > Key: SENTRY-2264 > URL: https://issues.apache.org/jira/browse/SENTRY-2264 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2264.001.patch > > > After introducing FGP, a user with only DROP on a database db1 and at least > CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus > elevate their privileges. > To reproduce: > As admin (e.g. hive): > 1. Create db1, db1.table1, db2, role r1. > 2. Grant DROP on db1 to role r1. > 3. Grant ALL on db2 to role r1 > 4. Grant role r1 to user testuser1. > As testuser1: > 1. use db1; alter table db1.table1 rename to db2.table1 > 2. select * from db2. table1 > Result: the select command succeeds. > Desired behavior: > we should at least require following privileges to execute the table rename > command: > table level "SELECT" and database level "DELECT" at source > database level "CREATE, INSERT, ALTER" at destination. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2264) It is possible to elevate privileges from DROP using alter table rename
Na Li created SENTRY-2264: - Summary: It is possible to elevate privileges from DROP using alter table rename Key: SENTRY-2264 URL: https://issues.apache.org/jira/browse/SENTRY-2264 Project: Sentry Issue Type: Bug Components: Sentry Affects Versions: 2.1.0 Reporter: Na Li Assignee: Na Li After introducing FGP, a user with only DROP on a database db1 and at least CREATE on db2 can run ALTER TABLE RENAME db1.table1 db2.table2, and thus elevate their privileges. To reproduce: As admin (e.g. hive): 1. Create db1, db1.table1, db2, role r1. 2. Grant DROP on db1 to role r1. 3. Grant ALL on db2 to role r1 4. Grant role r1 to user testuser1. As testuser1: 1. use db1; alter table db1.table1 rename to db2.table1 2. select * from db2. table1 Result: the select command succeeds. Desired behavior: we should at least require following privileges to execute the table rename command: table level "SELECT" and database level "DELECT" at source database level "CREATE, INSERT, ALTER" at destination. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2259) SQL CONSTRAINT name for user privilege is too long for Oracle 11.2
[
https://issues.apache.org/jira/browse/SENTRY-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2259:
--
Summary: SQL CONSTRAINT name for user privilege is too long for Oracle 11.2
(was: SQL CONSTRAINT name is too long for Oracle 11.2)
> SQL CONSTRAINT name for user privilege is too long for Oracle 11.2
> --
>
> Key: SENTRY-2259
> URL: https://issues.apache.org/jira/browse/SENTRY-2259
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2259.001.patch
>
>
> Prior to Oracle version 12.2, identifiers are not allowed to exceed 30
> characters in length
> Names must be from 1 to 30 bytes long with these exceptions:
> Names of databases are limited to 8 bytes.
> Names of database links can be as long as 128 bytes.
> The following index name is 32 bytes, and cause exception "ORA-00972:
> identifier is too long" on oracle DB 11.2.
> SENTRY_USER_DB_PRIVILEGE_MAP_PK
> SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
> {code}
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
> ADD CONSTRAINT SENTRY_USER_DB_PRIVILEGE_MAP_PK PRIMARY KEY
> (USER_ID,DB_PRIVILEGE_ID);
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
>ADD CONSTRAINT SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
>FOREIGN KEY (DB_PRIVILEGE_ID) REFERENCES
> SENTRY_DB_PRIVILEGE(DB_PRIVILEGE_ID);
> {code}
> Need to change the constrain name to be smaller than 30 characters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2259) SQL CONSTRAINT name is too long for Oracle 11.2
[
https://issues.apache.org/jira/browse/SENTRY-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2259:
--
Status: Patch Available (was: Open)
> SQL CONSTRAINT name is too long for Oracle 11.2
> ---
>
> Key: SENTRY-2259
> URL: https://issues.apache.org/jira/browse/SENTRY-2259
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2259.001.patch
>
>
> Prior to Oracle version 12.2, identifiers are not allowed to exceed 30
> characters in length
> Names must be from 1 to 30 bytes long with these exceptions:
> Names of databases are limited to 8 bytes.
> Names of database links can be as long as 128 bytes.
> The following index name is 32 bytes, and cause exception "ORA-00972:
> identifier is too long" on oracle DB 11.2.
> SENTRY_USER_DB_PRIVILEGE_MAP_PK
> SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
> {code}
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
> ADD CONSTRAINT SENTRY_USER_DB_PRIVILEGE_MAP_PK PRIMARY KEY
> (USER_ID,DB_PRIVILEGE_ID);
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
>ADD CONSTRAINT SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
>FOREIGN KEY (DB_PRIVILEGE_ID) REFERENCES
> SENTRY_DB_PRIVILEGE(DB_PRIVILEGE_ID);
> {code}
> Need to change the constrain name to be smaller than 30 characters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2259) SQL CONSTRAINT name is too long for Oracle 11.2
[
https://issues.apache.org/jira/browse/SENTRY-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2259:
--
Attachment: SENTRY-2259.001.patch
> SQL CONSTRAINT name is too long for Oracle 11.2
> ---
>
> Key: SENTRY-2259
> URL: https://issues.apache.org/jira/browse/SENTRY-2259
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2259.001.patch
>
>
> Prior to Oracle version 12.2, identifiers are not allowed to exceed 30
> characters in length
> Names must be from 1 to 30 bytes long with these exceptions:
> Names of databases are limited to 8 bytes.
> Names of database links can be as long as 128 bytes.
> The following index name is 32 bytes, and cause exception "ORA-00972:
> identifier is too long" on oracle DB 11.2.
> SENTRY_USER_DB_PRIVILEGE_MAP_PK
> SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
> {code}
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
> ADD CONSTRAINT SENTRY_USER_DB_PRIVILEGE_MAP_PK PRIMARY KEY
> (USER_ID,DB_PRIVILEGE_ID);
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
>ADD CONSTRAINT SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
>FOREIGN KEY (DB_PRIVILEGE_ID) REFERENCES
> SENTRY_DB_PRIVILEGE(DB_PRIVILEGE_ID);
> {code}
> Need to change the constrain name to be smaller than 30 characters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2245) Remove privileges that do not associate with a role or a user
[ https://issues.apache.org/jira/browse/SENTRY-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2245: -- Resolution: Fixed Status: Resolved (was: Patch Available) [~kkalyan][~spena] Thanks for reviewing the change > Remove privileges that do not associate with a role or a user > - > > Key: SENTRY-2245 > URL: https://issues.apache.org/jira/browse/SENTRY-2245 > Project: Sentry > Issue Type: Sub-task >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2245.003.patch, SENTRY-2245.004.patch, > SENTRY-2245.005.patch, SENTRY-2245.006.patch, SENTRY-2245.006.patch > > > After a privilege is detached from a role or a user, we should check if this > privilege is not associated with any role nor any user. Then, we should > remove this privilege from DB. Otherwise, the orphan privileges will grow > overtime. > One example place is in alterSentryRoleGrantPrivilegeCore. After removing > role from mInsert, this privilege may not be associated with any role or > user, and should be removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2244) Alter sentry role or user at granting privilege can avoid extra query to database
[ https://issues.apache.org/jira/browse/SENTRY-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2244: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Alter sentry role or user at granting privilege can avoid extra query to > database > - > > Key: SENTRY-2244 > URL: https://issues.apache.org/jira/browse/SENTRY-2244 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2244.001.patch, SENTRY-2244.002.patch, > SENTRY-2244.002.patch > > > In alterSentryRoleGrantPrivilegeCore and alterSentryUserGrantPrivilegeCore , > all of the privileges of a role or a user are already fetched from DB. In > following processing, there is no need to query DB to get privilege INSERT, > SELECT, ALL of this role again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2258) Remove user when it is not associated with other objects
[ https://issues.apache.org/jira/browse/SENTRY-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2258: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Remove user when it is not associated with other objects > > > Key: SENTRY-2258 > URL: https://issues.apache.org/jira/browse/SENTRY-2258 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2258.001.patch, SENTRY-2258.002.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > When a privilege is created and assigned to a user, this user is > automatically created if it does not exist. > So when a privilege is removed from a user, or a role is removed from a user, > this user should be removed when it is not associated with other objects > (like privileges and roles) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2259) SQL CONSTRAINT name is too long for Oracle 11.2
[
https://issues.apache.org/jira/browse/SENTRY-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-2259:
--
Summary: SQL CONSTRAINT name is too long for Oracle 11.2 (was: SQL
Constrain name is too long for Oracle 11.2)
> SQL CONSTRAINT name is too long for Oracle 11.2
> ---
>
> Key: SENTRY-2259
> URL: https://issues.apache.org/jira/browse/SENTRY-2259
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
>
> Prior to Oracle version 12.2, identifiers are not allowed to exceed 30
> characters in length
> Names must be from 1 to 30 bytes long with these exceptions:
> Names of databases are limited to 8 bytes.
> Names of database links can be as long as 128 bytes.
> The following index name is 32 bytes, and cause exception "ORA-00972:
> identifier is too long" on oracle DB 11.2.
> SENTRY_USER_DB_PRIVILEGE_MAP_PK
> SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
> {code}
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
> ADD CONSTRAINT SENTRY_USER_DB_PRIVILEGE_MAP_PK PRIMARY KEY
> (USER_ID,DB_PRIVILEGE_ID);
> ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
>ADD CONSTRAINT SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
>FOREIGN KEY (DB_PRIVILEGE_ID) REFERENCES
> SENTRY_DB_PRIVILEGE(DB_PRIVILEGE_ID);
> {code}
> Need to change the constrain name to be smaller than 30 characters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2245) Remove privileges that do not associate with a role or a user
[
https://issues.apache.org/jira/browse/SENTRY-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503581#comment-16503581
]
Na Li commented on SENTRY-2245:
---
The failure for patch 006 seems not caused by code change, but by "[ERROR]
Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
project sentry-tests-hive: There are test failures." reattach the patch again
to run test
https://builds.apache.org/job/PreCommit-SENTRY-Build/3859/artifact/test-output/test.txt
{code}
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0
approved: 1 licence.
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Sentry SUCCESS [2.627s]
[INFO] Sentry Core ... SUCCESS [0.169s]
[INFO] Sentry Core Common SUCCESS [6.255s]
[INFO] Sentry Core Model DB .. SUCCESS [1.911s]
[INFO] Sentry Core Model Indexer . SUCCESS [1.126s]
[INFO] Sentry Core Model Solr SUCCESS [2.213s]
[INFO] Sentry Core Model Sqoop ... SUCCESS [1.233s]
[INFO] Sentry Core Model Kafka ... SUCCESS [1.121s]
[INFO] Sentry Policies ... SUCCESS [0.085s]
[INFO] Sentry Policy Common .. SUCCESS [1.446s]
[INFO] Sentry Providers .. SUCCESS [0.155s]
[INFO] Sentry Provider Common SUCCESS [1.548s]
[INFO] Sentry Provider File .. SUCCESS [1.774s]
[INFO] Sentry Policy Engine .. SUCCESS [0.299s]
[INFO] Sentry Bindings ... SUCCESS [0.118s]
[INFO] Sentry Hive Binding Configuration . SUCCESS [2.701s]
[INFO] Hive follower for Sentry .. SUCCESS [2.991s]
[INFO] Sentry HDFS ... SUCCESS [0.080s]
[INFO] Sentry HDFS Common SUCCESS [22.078s]
[INFO] Sentry Service SUCCESS [0.103s]
[INFO] Sentry Service API SUCCESS [1.651s]
[INFO] Sentry Service Client . SUCCESS [0.545s]
[INFO] Sentry Provider DB SUCCESS [6.681s]
[INFO] Sentry Binding for Kafka .. SUCCESS [12.247s]
[INFO] Sentry Provider Cache . SUCCESS [1.584s]
[INFO] Sentry Hive Binding Common SUCCESS [1.897s]
[INFO] Sentry Binding for Solr ... SUCCESS [17.711s]
[INFO] Sentry Binding for Sqoop .. SUCCESS [10.208s]
[INFO] Sentry Binding for HBase Indexer .. SUCCESS [4.103s]
[INFO] Sentry Binding for Hive ... SUCCESS [25.373s]
[INFO] Sentry Solr ... SUCCESS [0.074s]
[INFO] Solr Sentry handler ... SUCCESS [2.814s]
[INFO] Sentry Service Server . SUCCESS [10:21.329s]
[INFO] Sentry Tests .. SUCCESS [0.118s]
[INFO] Sentry Solr Tests . SUCCESS [2:16.965s]
[INFO] Sentry Sqoop Tests SUCCESS [2:34.408s]
[INFO] Sentry Kafka Tests SUCCESS [3:03.556s]
[INFO] Sentry HDFS Service ... SUCCESS [6:56.989s]
[INFO] Sentry HDFS Namenode Plugin ... SUCCESS [8.056s]
[INFO] Sentry Hive Tests . FAILURE
[1:29:06.506s]
[INFO] Sentry HDFS Dist .. SUCCESS [2.512s]
[INFO] sentry-tools .. SUCCESS [2:35.910s]
[INFO] Sentry Distribution ... SUCCESS [39.906s]
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 2:00:02.134s
[INFO] Finished at: Wed Jun 06 00:33:49 UTC 2018
[INFO] Final Memory: 221M/1695M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
project sentry-tests-hive: There are test failures.
[ERROR]
[ERROR] Please refer to
/home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build/sentry-tests/sentry-tests-hive/target/surefire-reports
for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump,
[date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] ExecutionException The forked V
[jira] [Updated] (SENTRY-2245) Remove privileges that do not associate with a role or a user
[ https://issues.apache.org/jira/browse/SENTRY-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2245: -- Attachment: SENTRY-2245.006.patch > Remove privileges that do not associate with a role or a user > - > > Key: SENTRY-2245 > URL: https://issues.apache.org/jira/browse/SENTRY-2245 > Project: Sentry > Issue Type: Sub-task >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2245.003.patch, SENTRY-2245.004.patch, > SENTRY-2245.005.patch, SENTRY-2245.006.patch, SENTRY-2245.006.patch > > > After a privilege is detached from a role or a user, we should check if this > privilege is not associated with any role nor any user. Then, we should > remove this privilege from DB. Otherwise, the orphan privileges will grow > overtime. > One example place is in alterSentryRoleGrantPrivilegeCore. After removing > role from mInsert, this privilege may not be associated with any role or > user, and should be removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2259) SQL Constrain name is too long for Oracle 11.2
Na Li created SENTRY-2259:
-
Summary: SQL Constrain name is too long for Oracle 11.2
Key: SENTRY-2259
URL: https://issues.apache.org/jira/browse/SENTRY-2259
Project: Sentry
Issue Type: Bug
Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
Assignee: Na Li
Prior to Oracle version 12.2, identifiers are not allowed to exceed 30
characters in length
Names must be from 1 to 30 bytes long with these exceptions:
Names of databases are limited to 8 bytes.
Names of database links can be as long as 128 bytes.
The following index name is 32 bytes, and cause exception "ORA-00972:
identifier is too long" on oracle DB 11.2.
SENTRY_USER_DB_PRIVILEGE_MAP_PK
SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
{code}
ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
ADD CONSTRAINT SENTRY_USER_DB_PRIVILEGE_MAP_PK PRIMARY KEY
(USER_ID,DB_PRIVILEGE_ID);
ALTER TABLE SENTRY_USER_DB_PRIVILEGE_MAP
ADD CONSTRAINT SEN_USR_DB_PRV_MAP_SN_DB_PRV_FK
FOREIGN KEY (DB_PRIVILEGE_ID) REFERENCES
SENTRY_DB_PRIVILEGE(DB_PRIVILEGE_ID);
{code}
Need to change the constrain name to be smaller than 30 characters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2245) Remove privileges that do not associate with a role or a user
[ https://issues.apache.org/jira/browse/SENTRY-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2245: -- Attachment: SENTRY-2245.006.patch > Remove privileges that do not associate with a role or a user > - > > Key: SENTRY-2245 > URL: https://issues.apache.org/jira/browse/SENTRY-2245 > Project: Sentry > Issue Type: Sub-task >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2245.003.patch, SENTRY-2245.004.patch, > SENTRY-2245.005.patch, SENTRY-2245.006.patch > > > After a privilege is detached from a role or a user, we should check if this > privilege is not associated with any role nor any user. Then, we should > remove this privilege from DB. Otherwise, the orphan privileges will grow > overtime. > One example place is in alterSentryRoleGrantPrivilegeCore. After removing > role from mInsert, this privilege may not be associated with any role or > user, and should be removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2258) Remove user when it is not associated with other objects
[ https://issues.apache.org/jira/browse/SENTRY-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2258: -- Status: Patch Available (was: Open) > Remove user when it is not associated with other objects > > > Key: SENTRY-2258 > URL: https://issues.apache.org/jira/browse/SENTRY-2258 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2258.001.patch, SENTRY-2258.002.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > When a privilege is created and assigned to a user, this user is > automatically created if it does not exist. > So when a privilege is removed from a user, or a role is removed from a user, > this user should be removed when it is not associated with other objects > (like privileges and roles) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2258) Remove user when it is not associated with other objects
[ https://issues.apache.org/jira/browse/SENTRY-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2258: -- Attachment: SENTRY-2258.002.patch > Remove user when it is not associated with other objects > > > Key: SENTRY-2258 > URL: https://issues.apache.org/jira/browse/SENTRY-2258 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2258.001.patch, SENTRY-2258.002.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > When a privilege is created and assigned to a user, this user is > automatically created if it does not exist. > So when a privilege is removed from a user, or a role is removed from a user, > this user should be removed when it is not associated with other objects > (like privileges and roles) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2258) Remove user when it is not associated with other objects
[ https://issues.apache.org/jira/browse/SENTRY-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2258: -- Attachment: SENTRY-2258.001.patch > Remove user when it is not associated with other objects > > > Key: SENTRY-2258 > URL: https://issues.apache.org/jira/browse/SENTRY-2258 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2258.001.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > When a privilege is created and assigned to a user, this user is > automatically created if it does not exist. > So when a privilege is removed from a user, or a role is removed from a user, > this user should be removed when it is not associated with other objects > (like privileges and roles) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2158) Update notification handler to update privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2158: -- Status: Patch Available (was: Open) > Update notification handler to update privileges to user > > > Key: SENTRY-2158 > URL: https://issues.apache.org/jira/browse/SENTRY-2158 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2158.001.patch > > > SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing > permission related commands. We should update notification handler in the > following files when granting privileges to user. > When authorizable changes, need to change user privileges too > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2158) Update notification handler to update privileges to user
[ https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2158: -- Attachment: SENTRY-2158.001.patch > Update notification handler to update privileges to user > > > Key: SENTRY-2158 > URL: https://issues.apache.org/jira/browse/SENTRY-2158 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2158.001.patch > > > SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing > permission related commands. We should update notification handler in the > following files when granting privileges to user. > When authorizable changes, need to change user privileges too > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
