[jira] [Commented] (SENTRY-1904) TransactionManager should limit the max time spent by transaction retry
[
https://issues.apache.org/jira/browse/SENTRY-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351226#comment-16351226
]
Hadoop QA commented on SENTRY-1904:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12909068/SENTRY-1904.006.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3653/console
This message is automatically generated.
> TransactionManager should limit the max time spent by transaction retry
> ---
>
> Key: SENTRY-1904
> URL: https://issues.apache.org/jira/browse/SENTRY-1904
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.0.0
>Reporter: Alexander Kolbasov
>Assignee: kalyan kumar kalvagadda
>Priority: Major
> Attachments: SENTRY-1904.001.patch, SENTRY-1904.002.patch,
> SENTRY-1904.003.patch, SENTRY-1904.004.patch, SENTRY-1904.005.patch,
> SENTRY-1904.006.patch
>
>
> The TransactionManager uses exponential backoff strategy for transaction
> retries. This may cause some transactions to be delayed by a very long time.
> We should also have a constraint on the max time for a transaction so that we
> do not retry for too long.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2136) Bump metrics dependency to new namespace and version used by the rest of Hadoop
[
https://issues.apache.org/jira/browse/SENTRY-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351191#comment-16351191
]
Hadoop QA commented on SENTRY-2136:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12909032/SENTRY-2136.001.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3652/console
This message is automatically generated.
> Bump metrics dependency to new namespace and version used by the rest of
> Hadoop
> ---
>
> Key: SENTRY-2136
> URL: https://issues.apache.org/jira/browse/SENTRY-2136
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.0.0
> Environment: Sentry 2.1.0-SNAPSHOT
> building on Mac OS X 10.13.1, Java 1.8u161
>Reporter: Liam Sargent
>Assignee: Liam Sargent
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: SENTRY-2136.001.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Sentry currently specifies the _com.codahale.metrics_ package, version 3.0.2,
> in the top level pom.xml. When building Sentry, the Hadoop dependencies
> specifiy version 3.2.2, which changed the maven repo namespace to
> _io.dropwizard.metrics._ This conflict commonly causes test failures and
> classpath issues.
> After reaching out on the mailing list, there appears to be no legitimate
> reason to be using the outdated (3.0.2) version of metrics, and we should be
> able to safely bump to _io.dropwizard.metrics_ 3.2.2 to resolve the
> conflicting dependencies with the greater Hadoop ecosystem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-1904) TransactionManager should limit the max time spent by transaction retry
[ https://issues.apache.org/jira/browse/SENTRY-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kalyan kumar kalvagadda updated SENTRY-1904: Attachment: SENTRY-1904.006.patch > TransactionManager should limit the max time spent by transaction retry > --- > > Key: SENTRY-1904 > URL: https://issues.apache.org/jira/browse/SENTRY-1904 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: kalyan kumar kalvagadda >Priority: Major > Attachments: SENTRY-1904.001.patch, SENTRY-1904.002.patch, > SENTRY-1904.003.patch, SENTRY-1904.004.patch, SENTRY-1904.005.patch, > SENTRY-1904.006.patch > > > The TransactionManager uses exponential backoff strategy for transaction > retries. This may cause some transactions to be delayed by a very long time. > We should also have a constraint on the max time for a transaction so that we > do not retry for too long. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[ https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351014#comment-16351014 ] kalyan kumar kalvagadda commented on SENTRY-2109: - I have reverted the patch. > Fix the logic of identifying HMS out of Sync and handle gaps and > out-of-sequence notifications. > --- > > Key: SENTRY-2109 > URL: https://issues.apache.org/jira/browse/SENTRY-2109 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2109.001.patch, SENTRY-2109.002.patch, > SENTRY-2109.003.patch, SENTRY-2109.004.patch, SENTRY-2109.005.patch, > SENTRY-2109.006.patch, SENTRY-2109.007.patch, SENTRY-2109.008.patch, > SENTRY-2109.009.patch, SENTRY-2109.010.patch, SENTRY-2109.010.patch, > SENTRY-2109.011.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch, > SENTRY-2109.012.patch, Screenshot_HMS_NOTIFICATION_LOG.png > > > Currently HMSFollower proactively checks if sentry is out of sync with HMS > and initiates full snapshot, if needed. > There will be false positives with the current logic if there are gaps in the > event-id in the notification log sequence. > This jira is aimed at making that logic robust. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2135) Running show grant throws a NullPointerException
[ https://issues.apache.org/jira/browse/SENTRY-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350992#comment-16350992 ] Steve Moist commented on SENTRY-2135: - [HiveServer2-Handler-Pool: Thread-143]: Error executing statement: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: NullPointerException null at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:330) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:203) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:286) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.operation.Operation.run(Operation.java:262) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:503) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:490) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) [hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) [hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) [hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) [hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) [hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:605) [hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) [hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144] Caused by: java.lang.NullPointerException at org.apache.sentry.binding.hive.SentryHiveAuthorizationTaskFactoryImpl.createShowGrantTask(SentryHiveAuthorizationTaskFactoryImpl.java:193) ~[sentry-binding-hive-2.0.0-cdh6.x-SNAPSHOT.jar:?] at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeShowGrant(DDLSemanticAnalyzer.java:616) ~[hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:511) ~[hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250) ~[hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:538) ~[hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1330) ~[hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1308) ~[hive-exec-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:201) ~[hive-service-2.1.1-cdh6.x-SNAPSHOT.jar:2.1.1-cdh6.x-SNAPSHOT] ... 15 more > Running show grant throws a NullPointerException > > > Key: SENTRY-2135 > URL: https://issues.apache.org/jira/browse/SENTRY-2135 > Project: Sentry > Issue Type: Bug > Components: Hive Plugin >Affects Versions: 2.0.0 >Reporter: Steve Moist >Assignee: Steve Moist >Priority: Minor > > When running in beeline "show grant" causes a: > Error: Error while compiling statement: FAILED: NullPointerException null > (state=42000,code=4) > > User would expect some different error about incorrect syntax or command > rather than a NPE. -- This message was sent by Atlassian J
[jira] [Commented] (SENTRY-2140) Tag based access control
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350956#comment-16350956 ] Steve Moist commented on SENTRY-2140: - This is just Sentry for now, I'd imagine we'd want this functionality in impala later and I'm not sure of the scope for it now. As for the syntax, since these tags are not related to Hive and are sourced from some outside source, I'd imagine that either an api or cli to add tags to Hive information in Sentry. We most certainly would need a new data model to properly store these new attribute privileges. I'm currently unsure if this requires user-level privileges. Right now, I'm focusing on just tagging Hive columns, but I don't see why it could be expanded to tables or databases. > Tag based access control > > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "tags" that prevent users/roles from not accessing or seeing the data. For > users/roles that have that tag, they should be able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[ https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350906#comment-16350906 ] Alexander Kolbasov commented on SENTRY-2109: [~kkalyan] I would appreciate if you revert the patch and we complete the discussion review and then you commit (or not commit) whatever comes out of it. This patch had many iterations. There were several reviewers who had comments about earlier versions of the patch and some reviewers posted unanswered comments in this Jira so it is pretty clear that the review wasn't complete even though you got one +1 from a committer. I think that the best course of action is to revert the patch and complete the review. > Fix the logic of identifying HMS out of Sync and handle gaps and > out-of-sequence notifications. > --- > > Key: SENTRY-2109 > URL: https://issues.apache.org/jira/browse/SENTRY-2109 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2109.001.patch, SENTRY-2109.002.patch, > SENTRY-2109.003.patch, SENTRY-2109.004.patch, SENTRY-2109.005.patch, > SENTRY-2109.006.patch, SENTRY-2109.007.patch, SENTRY-2109.008.patch, > SENTRY-2109.009.patch, SENTRY-2109.010.patch, SENTRY-2109.010.patch, > SENTRY-2109.011.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch, > SENTRY-2109.012.patch, Screenshot_HMS_NOTIFICATION_LOG.png > > > Currently HMSFollower proactively checks if sentry is out of sync with HMS > and initiates full snapshot, if needed. > There will be false positives with the current logic if there are gaps in the > event-id in the notification log sequence. > This jira is aimed at making that logic robust. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2140) Tag based access control
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350882#comment-16350882 ] Alexander Kolbasov commented on SENTRY-2140: Is it a Sentry request or Hive request or both? Do you propose to tag columns using some kind of new Hive syntax or something else? What are these tags and how do they relate to privileges? Does this require some kind of user-level privileges? What objects do you want to tag - just columns or something else? > Tag based access control > > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "tags" that prevent users/roles from not accessing or seeing the data. For > users/roles that have that tag, they should be able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2136) Bump metrics dependency to new namespace and version used by the rest of Hadoop
[ https://issues.apache.org/jira/browse/SENTRY-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liam Sargent reassigned SENTRY-2136: Assignee: Liam Sargent > Bump metrics dependency to new namespace and version used by the rest of > Hadoop > --- > > Key: SENTRY-2136 > URL: https://issues.apache.org/jira/browse/SENTRY-2136 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 > Environment: Sentry 2.1.0-SNAPSHOT > building on Mac OS X 10.13.1, Java 1.8u161 >Reporter: Liam Sargent >Assignee: Liam Sargent >Priority: Minor > Fix For: 2.1.0 > > Attachments: SENTRY-2136.001.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Sentry currently specifies the _com.codahale.metrics_ package, version 3.0.2, > in the top level pom.xml. When building Sentry, the Hadoop dependencies > specifiy version 3.2.2, which changed the maven repo namespace to > _io.dropwizard.metrics._ This conflict commonly causes test failures and > classpath issues. > After reaching out on the mailing list, there appears to be no legitimate > reason to be using the outdated (3.0.2) version of metrics, and we should be > able to safely bump to _io.dropwizard.metrics_ 3.2.2 to resolve the > conflicting dependencies with the greater Hadoop ecosystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2137) Improve and rework the CLI
[ https://issues.apache.org/jira/browse/SENTRY-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350879#comment-16350879 ] Liam Sargent commented on SENTRY-2137: -- [~akolb] Where do these two CLIs live? I am unfamiliar with their current implementation. > Improve and rework the CLI > -- > > Key: SENTRY-2137 > URL: https://issues.apache.org/jira/browse/SENTRY-2137 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Steve Moist >Priority: Minor > > Sentry can be improved by moving all of the privilige actions for hive (such > as grant/revoke) from beeline and into a centralized CLI. With this we can > do operations such as show all privileges for a role across HDFS, Hive, > Impala, etc in a single location and administer this in a single location. > In a cluster, it would be good to have the sentry cli as a standalone > executable, so building in a REST API for Sentry use would be needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2138) Sentry REST API
[ https://issues.apache.org/jira/browse/SENTRY-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350878#comment-16350878 ] Alexander Kolbasov commented on SENTRY-2138: CLI and restful API are a bit orthogonal (well, you need *some* API to build CLI). But having restful API isn't an obvious thing - what would it do better then existing API? Do we have a real need to improve/change API? If you think that we do, we should start discussing this. > Sentry REST API > --- > > Key: SENTRY-2138 > URL: https://issues.apache.org/jira/browse/SENTRY-2138 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Steve Moist >Priority: Major > > In an effort to decentralize the Sentry CLI, we need to build out a restful > API to administer Sentry privileges for all the components. > This would also have the benefit of allowing customers to manage and > integrate Sentry permissions in their own applications. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2137) Improve and rework the CLI
[ https://issues.apache.org/jira/browse/SENTRY-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350869#comment-16350869 ] Alexander Kolbasov commented on SENTRY-2137: I agree that having a nice usable CLI is nice (we have two now to play with) but it doesn't mean that we should move things *out* of beeline. Having REST API is quite possible as well but this is a different ask from CLI, so let's not mix it in one bucket. > Improve and rework the CLI > -- > > Key: SENTRY-2137 > URL: https://issues.apache.org/jira/browse/SENTRY-2137 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Steve Moist >Priority: Minor > > Sentry can be improved by moving all of the privilige actions for hive (such > as grant/revoke) from beeline and into a centralized CLI. With this we can > do operations such as show all privileges for a role across HDFS, Hive, > Impala, etc in a single location and administer this in a single location. > In a cluster, it would be good to have the sentry cli as a standalone > executable, so building in a REST API for Sentry use would be needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[ https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350862#comment-16350862 ] kalyan kumar kalvagadda commented on SENTRY-2109: - [~akolb] This is what the patch does functionally. # Make sure that additional snapshots are not taken because of GAPS in the event-id's # Make sure that notification are not missed even when there are GAPS and out-of-sequence notifications. we know that issue of gaps and out-of-sequence in the events will be solved with Hive fix done for HIVE-18526 . Only justification to have this patch was to safe guard sentry and be able to handle cases where events are not received in proper order. It's reasonable not assuming that other components would behave correctly and have some functionality in place to handle the error scenarios. I will revert the patch if you say so. > Fix the logic of identifying HMS out of Sync and handle gaps and > out-of-sequence notifications. > --- > > Key: SENTRY-2109 > URL: https://issues.apache.org/jira/browse/SENTRY-2109 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2109.001.patch, SENTRY-2109.002.patch, > SENTRY-2109.003.patch, SENTRY-2109.004.patch, SENTRY-2109.005.patch, > SENTRY-2109.006.patch, SENTRY-2109.007.patch, SENTRY-2109.008.patch, > SENTRY-2109.009.patch, SENTRY-2109.010.patch, SENTRY-2109.010.patch, > SENTRY-2109.011.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch, > SENTRY-2109.012.patch, Screenshot_HMS_NOTIFICATION_LOG.png > > > Currently HMSFollower proactively checks if sentry is out of sync with HMS > and initiates full snapshot, if needed. > There will be false positives with the current logic if there are gaps in the > event-id in the notification log sequence. > This jira is aimed at making that logic robust. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2140) Tag based access control
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Moist updated SENTRY-2140: Summary: Tag based access control (was: Tag based attribute control) > Tag based access control > > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "tags" that prevent users/roles from not accessing or seeing the data. For > users/roles that have that tag, they should be able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2140) Tag based attribute control
Steve Moist created SENTRY-2140: --- Summary: Tag based attribute control Key: SENTRY-2140 URL: https://issues.apache.org/jira/browse/SENTRY-2140 Project: Sentry Issue Type: New Feature Components: Core Reporter: Steve Moist As a user, I want to have finer grain control over which users/roles can view data in Hive. Some information such as Social Security Number is considered very confidential information. I want to be able to tag columns in Hive with "tags" that prevent users/roles from not accessing or seeing the data. For users/roles that have that tag, they should be able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2139) Improve documentation to include how to work with other services
Steve Moist created SENTRY-2139: --- Summary: Improve documentation to include how to work with other services Key: SENTRY-2139 URL: https://issues.apache.org/jira/browse/SENTRY-2139 Project: Sentry Issue Type: Task Components: Docs Affects Versions: 2.0.0 Reporter: Steve Moist The docs could use more information on how to administer, setup, control privileges and list out all the available commands to Sentry for each service. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2138) Sentry REST API
Steve Moist created SENTRY-2138: --- Summary: Sentry REST API Key: SENTRY-2138 URL: https://issues.apache.org/jira/browse/SENTRY-2138 Project: Sentry Issue Type: New Feature Affects Versions: 2.0.0 Reporter: Steve Moist In an effort to decentralize the Sentry CLI, we need to build out a restful API to administer Sentry privileges for all the components. This would also have the benefit of allowing customers to manage and integrate Sentry permissions in their own applications. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2137) Improve and rework the CLI
Steve Moist created SENTRY-2137: --- Summary: Improve and rework the CLI Key: SENTRY-2137 URL: https://issues.apache.org/jira/browse/SENTRY-2137 Project: Sentry Issue Type: New Feature Affects Versions: 2.0.0 Reporter: Steve Moist Sentry can be improved by moving all of the privilige actions for hive (such as grant/revoke) from beeline and into a centralized CLI. With this we can do operations such as show all privileges for a role across HDFS, Hive, Impala, etc in a single location and administer this in a single location. In a cluster, it would be good to have the sentry cli as a standalone executable, so building in a REST API for Sentry use would be needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2136) Bump metrics dependency to new namespace and version used by the rest of Hadoop
[ https://issues.apache.org/jira/browse/SENTRY-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liam Sargent updated SENTRY-2136: - Attachment: SENTRY-2136.001.patch > Bump metrics dependency to new namespace and version used by the rest of > Hadoop > --- > > Key: SENTRY-2136 > URL: https://issues.apache.org/jira/browse/SENTRY-2136 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 > Environment: Sentry 2.1.0-SNAPSHOT > building on Mac OS X 10.13.1, Java 1.8u161 >Reporter: Liam Sargent >Priority: Minor > Fix For: 2.1.0 > > Attachments: SENTRY-2136.001.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Sentry currently specifies the _com.codahale.metrics_ package, version 3.0.2, > in the top level pom.xml. When building Sentry, the Hadoop dependencies > specifiy version 3.2.2, which changed the maven repo namespace to > _io.dropwizard.metrics._ This conflict commonly causes test failures and > classpath issues. > After reaching out on the mailing list, there appears to be no legitimate > reason to be using the outdated (3.0.2) version of metrics, and we should be > able to safely bump to _io.dropwizard.metrics_ 3.2.2 to resolve the > conflicting dependencies with the greater Hadoop ecosystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2136) Bump metrics dependency to new namespace and version used by the rest of Hadoop
[ https://issues.apache.org/jira/browse/SENTRY-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liam Sargent updated SENTRY-2136: - Status: Patch Available (was: Open) > Bump metrics dependency to new namespace and version used by the rest of > Hadoop > --- > > Key: SENTRY-2136 > URL: https://issues.apache.org/jira/browse/SENTRY-2136 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 > Environment: Sentry 2.1.0-SNAPSHOT > building on Mac OS X 10.13.1, Java 1.8u161 >Reporter: Liam Sargent >Priority: Minor > Fix For: 2.1.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Sentry currently specifies the _com.codahale.metrics_ package, version 3.0.2, > in the top level pom.xml. When building Sentry, the Hadoop dependencies > specifiy version 3.2.2, which changed the maven repo namespace to > _io.dropwizard.metrics._ This conflict commonly causes test failures and > classpath issues. > After reaching out on the mailing list, there appears to be no legitimate > reason to be using the outdated (3.0.2) version of metrics, and we should be > able to safely bump to _io.dropwizard.metrics_ 3.2.2 to resolve the > conflicting dependencies with the greater Hadoop ecosystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2136) Bump metrics dependency to new namespace and version used by the rest of Hadoop
Liam Sargent created SENTRY-2136: Summary: Bump metrics dependency to new namespace and version used by the rest of Hadoop Key: SENTRY-2136 URL: https://issues.apache.org/jira/browse/SENTRY-2136 Project: Sentry Issue Type: Bug Components: Sentry Affects Versions: 2.0.0 Environment: Sentry 2.1.0-SNAPSHOT building on Mac OS X 10.13.1, Java 1.8u161 Reporter: Liam Sargent Fix For: 2.1.0 Sentry currently specifies the _com.codahale.metrics_ package, version 3.0.2, in the top level pom.xml. When building Sentry, the Hadoop dependencies specifiy version 3.2.2, which changed the maven repo namespace to _io.dropwizard.metrics._ This conflict commonly causes test failures and classpath issues. After reaching out on the mailing list, there appears to be no legitimate reason to be using the outdated (3.0.2) version of metrics, and we should be able to safely bump to _io.dropwizard.metrics_ 3.2.2 to resolve the conflicting dependencies with the greater Hadoop ecosystem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2109) Fix the logic of identifying HMS out of Sync and handle gaps and out-of-sequence notifications.
[ https://issues.apache.org/jira/browse/SENTRY-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350490#comment-16350490 ] Sergio Peña commented on SENTRY-2109: - [~vspec...@gmail.com] Thanks for the confirmation about keeping it on hold, but the patch was committed yesterday and there were still some concerns about it. Should we revert this patch to finish the tests and reviews with the HMS fix and see if that fixes the current problems? <- [~kkalyan] > Fix the logic of identifying HMS out of Sync and handle gaps and > out-of-sequence notifications. > --- > > Key: SENTRY-2109 > URL: https://issues.apache.org/jira/browse/SENTRY-2109 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.1.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2109.001.patch, SENTRY-2109.002.patch, > SENTRY-2109.003.patch, SENTRY-2109.004.patch, SENTRY-2109.005.patch, > SENTRY-2109.006.patch, SENTRY-2109.007.patch, SENTRY-2109.008.patch, > SENTRY-2109.009.patch, SENTRY-2109.010.patch, SENTRY-2109.010.patch, > SENTRY-2109.011.patch, SENTRY-2109.012.patch, SENTRY-2109.012.patch, > SENTRY-2109.012.patch, Screenshot_HMS_NOTIFICATION_LOG.png > > > Currently HMSFollower proactively checks if sentry is out of sync with HMS > and initiates full snapshot, if needed. > There will be false positives with the current logic if there are gaps in the > event-id in the notification log sequence. > This jira is aimed at making that logic robust. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
