[jira] [Commented] (SENTRY-2370) Create a cache of PathUpdates to send to NN
[ https://issues.apache.org/jira/browse/SENTRY-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597061#comment-16597061 ] Arjun Mishra commented on SENTRY-2370: -- [~akolb] Yes the motivation was to not have to fetch from Sentry database when NN failed to get a full update (path or perm) which can sometimes take up to 30 mins. The caching strategy is * Store Path/Perm Updates when they are created * On consecutive requests, we check image number and sequence number to determine if the cache is current. If yes, return the cache, else re-build it Cache invalidation will be a part of another ticket since we want to invalidate cache once NN acknowledges receipt of full updates. But that is a part of SENTRY-2287 (https://issues.apache.org/jira/browse/SENTRY-2287) which is pending. > Create a cache of PathUpdates to send to NN > --- > > Key: SENTRY-2370 > URL: https://issues.apache.org/jira/browse/SENTRY-2370 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Attachments: SENTRY-2370.01.patch, SENTRY-2370.02.patch > > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2371) Add a new thrift API for getting all privileges a user has
[ https://issues.apache.org/jira/browse/SENTRY-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596818#comment-16596818 ] Dan Burkert commented on SENTRY-2371: - Clarification: our primary use-case will be to gather all of the privileges for a specific user and table, and only occasionally will we need to gather all privileges for a user across all tables. We should never need to gather all privileges for all users across all objects, or gather all privileges for all users across a specific object. > Add a new thrift API for getting all privileges a user has > --- > > Key: SENTRY-2371 > URL: https://issues.apache.org/jira/browse/SENTRY-2371 > Project: Sentry > Issue Type: Improvement >Reporter: Hao Hao >Priority: Major > > SENTRY-2256 introduces an API to list the privileges directly assigned to a > user. But there isn't an API to list all privileges a user has including the > ones granted to the group the user is in. Such an API is important when > clients need to find out all privilege a user has. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2371) Add a new thrift API for getting all privileges a user has
[ https://issues.apache.org/jira/browse/SENTRY-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596813#comment-16596813 ] Dan Burkert commented on SENTRY-2371: - [~LinaAtAustin] - Hao is out for a few days, but I have some context on this feature request. For Kudu/Sentry integration (KUDU-428), we'd like to have a single thrift API to call to gather all the privileges for a user. The existing APIs require sending multiple sequential RPCs to Sentry to get all associated groups, then associated roles, then associated privileges. It would be a lot more efficient if Sentry could perform this join for us on the server (perhaps even in the database?), and would save us from having to re-implement the join in C++. > Add a new thrift API for getting all privileges a user has > --- > > Key: SENTRY-2371 > URL: https://issues.apache.org/jira/browse/SENTRY-2371 > Project: Sentry > Issue Type: Improvement >Reporter: Hao Hao >Priority: Major > > SENTRY-2256 introduces an API to list the privileges directly assigned to a > user. But there isn't an API to list all privileges a user has including the > ones granted to the group the user is in. Such an API is important when > clients need to find out all privilege a user has. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2371) Add a new thrift API for getting all privileges a user has
[ https://issues.apache.org/jira/browse/SENTRY-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596810#comment-16596810 ] Alexander Kolbasov commented on SENTRY-2371: I think Sentry can do these kind of queries more efficiently since it can get the result from a single transaction. Otherwise there is always a chance of inconsistency if something changed in between. > Add a new thrift API for getting all privileges a user has > --- > > Key: SENTRY-2371 > URL: https://issues.apache.org/jira/browse/SENTRY-2371 > Project: Sentry > Issue Type: Improvement >Reporter: Hao Hao >Priority: Major > > SENTRY-2256 introduces an API to list the privileges directly assigned to a > user. But there isn't an API to list all privileges a user has including the > ones granted to the group the user is in. Such an API is important when > clients need to find out all privilege a user has. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2370) Create a cache of PathUpdates to send to NN
[ https://issues.apache.org/jira/browse/SENTRY-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596808#comment-16596808 ] Alexander Kolbasov commented on SENTRY-2370: Can you elaborate on the caching strategy? Is it only for the case when NN failed to get a full snapshot? How do you invalidate the cache if something changed since then? > Create a cache of PathUpdates to send to NN > --- > > Key: SENTRY-2370 > URL: https://issues.apache.org/jira/browse/SENTRY-2370 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Attachments: SENTRY-2370.01.patch, SENTRY-2370.02.patch > > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2371) Add a new thrift API for getting all privileges a user has
[ https://issues.apache.org/jira/browse/SENTRY-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596807#comment-16596807 ] Na Li commented on SENTRY-2371: --- [~hahao] Could "SENTRY-2284: Add two client API to get all roles or users privileges mapping (Sergio Pena, reviewed by Na Li, Arjun Mishra)" satisfy your need? You can find the group a user belongs to, then roles of those groups, so getting privileges for user and roles should work. > Add a new thrift API for getting all privileges a user has > --- > > Key: SENTRY-2371 > URL: https://issues.apache.org/jira/browse/SENTRY-2371 > Project: Sentry > Issue Type: Improvement >Reporter: Hao Hao >Priority: Major > > SENTRY-2256 introduces an API to list the privileges directly assigned to a > user. But there isn't an API to list all privileges a user has including the > ones granted to the group the user is in. Such an API is important when > clients need to find out all privilege a user has. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2320) CLI: Refactor existing Sentry CLIs to use jcommander instead of commons-cli
[ https://issues.apache.org/jira/browse/SENTRY-2320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596782#comment-16596782 ] Steve Moist commented on SENTRY-2320: - Please link to the review. > CLI: Refactor existing Sentry CLIs to use jcommander instead of commons-cli > --- > > Key: SENTRY-2320 > URL: https://issues.apache.org/jira/browse/SENTRY-2320 > Project: Sentry > Issue Type: Sub-task >Affects Versions: 2.1.0 >Reporter: Anthony Young-Garner >Assignee: Anthony Young-Garner >Priority: Major > Labels: ABAC > Attachments: SENTRY-2320-1.patch > > > In preparation for ABCM CLI changes discussed in SENTRY-2227 and as discussed > on the mailing list, the existing CLIs will be refactored to use the > jcommander library instead of commons-cli -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2370) Create a cache of PathUpdates to send to NN
[ https://issues.apache.org/jira/browse/SENTRY-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2370: - Attachment: SENTRY-2370.02.patch > Create a cache of PathUpdates to send to NN > --- > > Key: SENTRY-2370 > URL: https://issues.apache.org/jira/browse/SENTRY-2370 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Attachments: SENTRY-2370.01.patch, SENTRY-2370.02.patch > > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2357) Revert SENTRY-2309 as it causes an HMS issue for pig
[ https://issues.apache.org/jira/browse/SENTRY-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596702#comment-16596702 ] Steve Moist commented on SENTRY-2357: - No idea, we have some automated integration tests that run a cluster. When kerberos was enabled, the test failed. When it was not it didn't fail. It's not something I want to dig into as commenting out all the println's fixed it and having it debug to it's own log for HMS sync info is a better solution. > Revert SENTRY-2309 as it causes an HMS issue for pig > - > > Key: SENTRY-2357 > URL: https://issues.apache.org/jira/browse/SENTRY-2357 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Steve Moist >Assignee: Arjun Mishra >Priority: Major > Fix For: 2.1.0 > > > When running create table on pig, it has an issue with HMS > > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: > org.apache.hadoop.security.AccessControlException Permission denied: > user=systest, access=WRITE, inode="/user/hive/ > warehouse":hive:hive:drwxrwx--x > at > org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:400) > at > org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:256) > at > org.apache.sentry.hdfs.SentryINodeAttributesProvider$SentryPermissionEnforcer.checkPermission(SentryINodeAttributesProvider.java:86) > at > org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:194) > at > org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1846) > at > org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1830) > at > org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1789) > at > org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:60) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3142) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1123) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:696) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1726) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2318) Sentry listener should log the failure if grant/revoke of owner privilege fails
[ https://issues.apache.org/jira/browse/SENTRY-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596600#comment-16596600 ] Na Li commented on SENTRY-2318: --- [~kkalyan] In our discussion, it seems we tend to agree that post event listener should not throw exception and cause HMS to retry. Should we resolve this jira as not to fix or not a problem? > Sentry listener should log the failure if grant/revoke of owner privilege > fails > --- > > Key: SENTRY-2318 > URL: https://issues.apache.org/jira/browse/SENTRY-2318 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Attachments: SENTRY-2318.001.patch, SENTRY-2318.002.patch > > > If the grant/revoke of owner privilege fails for any reason it should logged > be as an error in the hive logs otherwise this will go un-noticed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2370) Create a cache of PathUpdates to send to NN
[ https://issues.apache.org/jira/browse/SENTRY-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2370: - Description: When NN requests path updates from sentry and if it exceeds the time threshold, on consecutive attempts sentry will attempt to fetch the full update from scratch. Instead it should cache it and update the cache before sending it to NN. was: When NN requests path updates from sentry and if it exceeds the time threshold, on consecutive attempts sentry will attempt to fetch the full update from scratch. Instead it should cache it and update the cache before sending it to NN. Also once sent to NN client it should be reset > Create a cache of PathUpdates to send to NN > --- > > Key: SENTRY-2370 > URL: https://issues.apache.org/jira/browse/SENTRY-2370 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Attachments: SENTRY-2370.01.patch > > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2370) Create a cache of PathUpdates to send to NN
[ https://issues.apache.org/jira/browse/SENTRY-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra updated SENTRY-2370: - Description: When NN requests path updates from sentry and if it exceeds the time threshold, on consecutive attempts sentry will attempt to fetch the full update from scratch. Instead it should cache it and update the cache before sending it to NN. Also once sent to NN client it should be reset was:When NN requests path updates from sentry and if it exceeds the time threshold, on consecutive attempts sentry will attempt to fetch the full update from scratch. Instead it should cache it and update the cache before sending it to NN > Create a cache of PathUpdates to send to NN > --- > > Key: SENTRY-2370 > URL: https://issues.apache.org/jira/browse/SENTRY-2370 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > Attachments: SENTRY-2370.01.patch > > > When NN requests path updates from sentry and if it exceeds the time > threshold, on consecutive attempts sentry will attempt to fetch the full > update from scratch. Instead it should cache it and update the cache before > sending it to NN. > Also once sent to NN client it should be reset -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2372) SentryStore should not implement grantOptionCheck
Sergio Peña created SENTRY-2372:
---
Summary: SentryStore should not implement grantOptionCheck
Key: SENTRY-2372
URL: https://issues.apache.org/jira/browse/SENTRY-2372
Project: Sentry
Issue Type: Improvement
Components: Sentry, sentrystore
Affects Versions: 2.1.0
Reporter: Sergio Peña
During functional testing it was found that SentryStore implementation contains
logic that enforces sentry rights and depends on cluster-specific context.
Specifically grantOptionCheck needs to be able to resolve hadoop user's groups
and sentry admin groups configured on the cluster.
There are two problems with this:
# Some backends use SentryStore in a multi-tenant way and does have the
context that SentryStore expects when it is used in cluster.
# Security enforcement logic shouldn't be in SentryStore if it is to be
trusted. Since the backends Sentry API may be stateless the caller has to pass
request context to such implementation backend together with the explicit
SentryStore arguments. If the context (e.g. groups) is passed with the request
the checks become unenforceable since caller controls variables on both sides
of the comparison.
The recommendation is to remove {{grantOptionCheck}} and {{SentryStore}} and to
implement equivalent logic in {{SentryPolicyStoreProcessor}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (SENTRY-2341) With External Tables Sentry does not handle alter table rename correctly
[ https://issues.apache.org/jira/browse/SENTRY-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Mishra resolved SENTRY-2341. -- Resolution: Not A Bug > With External Tables Sentry does not handle alter table rename correctly > > > Key: SENTRY-2341 > URL: https://issues.apache.org/jira/browse/SENTRY-2341 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: Arjun Mishra >Assignee: Arjun Mishra >Priority: Major > > For external tables, running ALTER TABLE RENAME will not change the location > of that table, but will drop the ACL. > Sentry drops privileges every time we delete an object, or rename an object. > In case of rename Sentry drops the privilege from old object and adds to new > one > However with external tables, on renaming the table no new location is > created. So even though Sentry has in its SENTRY_PATH_CHANGE the ACL for this > new location, it never gets applied because it doesn't actually exist -- This message was sent by Atlassian JIRA (v7.6.3#76005)
