[jira] [Commented] (SENTRY-2134) Apply Hive URI grants recursively to subdirectories
[ https://issues.apache.org/jira/browse/SENTRY-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363442#comment-16363442 ] BELUGA BEHR commented on SENTRY-2134: - Perhaps, for simplicity sake, Sentry Sync can be keyed off URI alone and no longer on database/table location. Not only is it simple, but it will give flexibility because two tables, within the same database, Sentry can control Hive/Impala access to both but allow only one to be open on HDFS. Finally, there is no need to worry about a database/table location and a URI colliding. > Apply Hive URI grants recursively to subdirectories > --- > > Key: SENTRY-2134 > URL: https://issues.apache.org/jira/browse/SENTRY-2134 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Affects Versions: 1.8.0, 2.0.0, 1.7.1 >Reporter: Ruslan Dautkhanov >Priority: Major > Labels: hive, uri > > Currently we need to add direct grants for all Hive tables' LOCATIONs. > Like, 'hdfs_staging/table1', 'hdfs_staging/table2', etc.. > It's not manageable this way. - we can't add grants for each and every table. > It would be great if we could just do one grant - > 'hdfs_staging/' so it would automatically be applied to > 'hdfs_staging/table1', 'hdfs_staging/table2', and other subdirectories. > There is probably a reason this wasn't implemented earlier? Thanks for > considering this improvement. > Also found another user's request on this - > https://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-Sentry-GRANT-ALL-ON-URI-not-cascaded-down-through/td-p/39928 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2134) Apply Hive URI grants recursively to subdirectories
[ https://issues.apache.org/jira/browse/SENTRY-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363434#comment-16363434 ] Alexander Kolbasov commented on SENTRY-2134: I think that supporting URI grants through Sentry may be a reasonable thing to do as long as it is clear hw they interplay with regular grants - what happens if there is table grant and URI grant? Or there is URI grant on a directory and column-level privilege? Would someone care to draft a proposal of the suggested behavior of URI grants? > Apply Hive URI grants recursively to subdirectories > --- > > Key: SENTRY-2134 > URL: https://issues.apache.org/jira/browse/SENTRY-2134 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Affects Versions: 1.8.0, 2.0.0, 1.7.1 >Reporter: Ruslan Dautkhanov >Priority: Major > Labels: hive, uri > > Currently we need to add direct grants for all Hive tables' LOCATIONs. > Like, 'hdfs_staging/table1', 'hdfs_staging/table2', etc.. > It's not manageable this way. - we can't add grants for each and every table. > It would be great if we could just do one grant - > 'hdfs_staging/' so it would automatically be applied to > 'hdfs_staging/table1', 'hdfs_staging/table2', and other subdirectories. > There is probably a reason this wasn't implemented earlier? Thanks for > considering this improvement. > Also found another user's request on this - > https://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-Sentry-GRANT-ALL-ON-URI-not-cascaded-down-through/td-p/39928 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2137) Improve and rework the CLI
[ https://issues.apache.org/jira/browse/SENTRY-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363431#comment-16363431 ] Alexander Kolbasov commented on SENTRY-2137: Note that there is also an interactive session-based Sentry CLI. > Improve and rework the CLI > -- > > Key: SENTRY-2137 > URL: https://issues.apache.org/jira/browse/SENTRY-2137 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Steve Moist >Priority: Minor > > Sentry can be improved by moving all of the privilige actions for hive (such > as grant/revoke) from beeline and into a centralized CLI. With this we can > do operations such as show all privileges for a role across HDFS, Hive, > Impala, etc in a single location and administer this in a single location. > In a cluster, it would be good to have the sentry cli as a standalone > executable, so building in a REST API for Sentry use would be needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2140) Tag based access control
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363430#comment-16363430 ] Alexander Kolbasov commented on SENTRY-2140: It would be nice to see a more detailed description of the proposal once you have it worked out. > Tag based access control > > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "tags" that prevent users/roles from not accessing or seeing the data. For > users/roles that have that tag, they should be able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2141) Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo correctly
[
https://issues.apache.org/jira/browse/SENTRY-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363408#comment-16363408
]
Hadoop QA commented on SENTRY-2141:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12910480/SENTRY-2141.001.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3662/console
This message is automatically generated.
> Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo
> correctly
> ---
>
> Key: SENTRY-2141
> URL: https://issues.apache.org/jira/browse/SENTRY-2141
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Na Li
>Assignee: Na Li
>Priority: Major
> Attachments: SENTRY-2141.001.patch
>
>
> sentry CreateTime is the difference, measured in milliseconds, between the
> current time and midnight, January 1, 1970 UTC. hive granttime is in seconds.
> So need to convert the time.
> The original code just cost the timestamp from long to int without converting
> milliseconds to seconds. Therefore, the timestamp value is wrong when
> retrieving the privilege from hive.
> The solution is to convert the time correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2115) Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
[
https://issues.apache.org/jira/browse/SENTRY-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363335#comment-16363335
]
Hadoop QA commented on SENTRY-2115:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12910457/SENTRY-2115.002.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3661/console
This message is automatically generated.
> Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
> -
>
> Key: SENTRY-2115
> URL: https://issues.apache.org/jira/browse/SENTRY-2115
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Critical
> Attachments: SENTRY-2115.001.patch, SENTRY-2115.002.patch
>
>
> *Current Behavior,*
> *Scenario-1:* When HDFS sync is disabled, and sentry is started for the first
> time.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-2:* When HDFS sync is disabled, and current event-id from HMS is
> less than last event-d processed by sentry
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-3:* When HDFS sync is disabled, and first event-id in the
> subsequent pull is not greater than the last event-id processed by sentry by
> 1.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into
> SENTRY_HMS_NOTIFICATION_ID.{color:#FF} This is wrong{color}
> *Scenario-4:* Initially HDFS sync was enabled and later disabled for while
> and then HDFS sync is enabled and sentry service is restarted to get it to
> effect.
> * On disabling HDFS sync, HMSFollower would update the
> SENTRY_HMS_NOTIFICATION_ID table but not the MSentryPathChange table.
> When HDFS sync is enabled again, HMSFollower would continue fetching new
> notifications and process them and update the MSentryPathChange and
> MAuthzPathsMapping info. This is not correct as sentry would not take a
> snapshot and will not have any path-mapping information about HMS objects. As
> a result, HDFS will ACL will not be added for the existing HMS
> objects.{color:#FF} This is wrong.{color}
> *Correct Behavior:*
> * Full snapshots need not be taken in all Scenario-1, Scenario-2 and
> Scenario-3.
> * When Sentry detects out-of-sync situations, it should reset
> SENTRY_HMS_NOTIFICATION_ID table and start processing the event in
> HMS_NOTIFICATION_LOG from beginning.
> * To handle scenario explained in *Scenario-4,* sentry should reset the
> mapping information when ever HDFS sync is disabled. That way it can learn
> from scratch when the feature is enabled back. There is no value is holding
> stale data even when we know it will have issues when the feature is enabled
> back.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2141) Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo correctly
[ https://issues.apache.org/jira/browse/SENTRY-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2141: -- Status: Patch Available (was: Open) > Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo > correctly > --- > > Key: SENTRY-2141 > URL: https://issues.apache.org/jira/browse/SENTRY-2141 > Project: Sentry > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2141.001.patch > > > sentry CreateTime is the difference, measured in milliseconds, between the > current time and midnight, January 1, 1970 UTC. hive granttime is in seconds. > So need to convert the time. > The original code just cost the timestamp from long to int without converting > milliseconds to seconds. Therefore, the timestamp value is wrong when > retrieving the privilege from hive. > The solution is to convert the time correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2141) Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo correctly
[ https://issues.apache.org/jira/browse/SENTRY-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-2141: -- Attachment: SENTRY-2141.001.patch > Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo > correctly > --- > > Key: SENTRY-2141 > URL: https://issues.apache.org/jira/browse/SENTRY-2141 > Project: Sentry > Issue Type: Bug >Affects Versions: 2.0.0, 2.1.0 >Reporter: Na Li >Assignee: Na Li >Priority: Major > Attachments: SENTRY-2141.001.patch > > > sentry CreateTime is the difference, measured in milliseconds, between the > current time and midnight, January 1, 1970 UTC. hive granttime is in seconds. > So need to convert the time. > The original code just cost the timestamp from long to int without converting > milliseconds to seconds. Therefore, the timestamp value is wrong when > retrieving the privilege from hive. > The solution is to convert the time correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2141) Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo correctly
[
https://issues.apache.org/jira/browse/SENTRY-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363290#comment-16363290
]
Na Li commented on SENTRY-2141:
---
Sentry is using milliseconds in
*{color:#FF}SentryStore.setCreateTime(){color}*, and the type is long.
{code:java}
private MSentryPrivilege convertToMSentryPrivilege(TSentryPrivilege privilege)
throws SentryInvalidInputException {
MSentryPrivilege mSentryPrivilege = new MSentryPrivilege();
mSentryPrivilege.setServerName(toNULLCol(safeTrimLower(privilege.getServerName(;
mSentryPrivilege.setDbName(toNULLCol(safeTrimLower(privilege.getDbName(;
mSentryPrivilege.setTableName(toNULLCol(safeTrimLower(privilege.getTableName(;
mSentryPrivilege.setColumnName(toNULLCol(safeTrimLower(privilege.getColumnName(;
mSentryPrivilege.setPrivilegeScope(safeTrim(privilege.getPrivilegeScope()));
mSentryPrivilege.setAction(toNULLCol(safeTrimLower(privilege.getAction(;
mSentryPrivilege.setCreateTime(System.currentTimeMillis());
mSentryPrivilege.setURI(toNULLCol(safeTrim(privilege.getURI(;
if ( !privilege.getGrantOption().equals(TSentryGrantOption.UNSET) ) {
mSentryPrivilege.setGrantOption(Boolean.valueOf(privilege.getGrantOption().toString()));
} else {
mSentryPrivilege.setGrantOption(null);
}
return mSentryPrivilege;
}
{code}
Hive is using seconds and type is int.
Then hive converts the time in seconds to milliseconds again at
*{color:#FF}(long)privilege.getGrantTime() * 1000L{color}*
{code:java}
DDLTask.writeGrantInfo
static String writeGrantInfo(List privileges, boolean
testMode) {
if (privileges != null && !privileges.isEmpty()) {
StringBuilder builder = new StringBuilder();
Collections.sort(privileges, new Comparator() {
public int compare(HivePrivilegeInfo o1, HivePrivilegeInfo o2) {
int compare = o1.getObject().compareTo(o2.getObject());
if (compare == 0) {
compare = o1.getPrincipal().compareTo(o2.getPrincipal());
}
if (compare == 0) {
compare = o1.getPrivilege().compareTo(o2.getPrivilege());
}
return compare;
}
});
Iterator var3 = privileges.iterator();
while(var3.hasNext()) {
HivePrivilegeInfo privilege = (HivePrivilegeInfo)var3.next();
HivePrincipal principal = privilege.getPrincipal();
HivePrivilegeObject resource = privilege.getObject();
HivePrincipal grantor = privilege.getGrantorPrincipal();
appendNonNull(builder, resource.getDbname(), true);
appendNonNull(builder, resource.getObjectName());
appendNonNull(builder, resource.getPartKeys());
appendNonNull(builder, resource.getColumns());
appendNonNull(builder, principal.getName());
appendNonNull(builder, principal.getType());
appendNonNull(builder, privilege.getPrivilege().getName());
appendNonNull(builder, privilege.isGrantOption());
appendNonNull(builder, testMode ? -1L : (long)privilege.getGrantTime()
* 1000L);
appendNonNull(builder, grantor.getName());
}
return builder.toString();
} else {
return "";
}
}
{code}
Therefore, sentry should convert the time accordingly
{code}
Old code uses "(int) tPrivilege.getCreateTime()"
public static HivePrivilegeInfo convert2HivePrivilegeInfo(TSentryPrivilege
tPrivilege,
HivePrincipal principal) {
HivePrivilege hivePrivilege = convert2HivePrivilege(tPrivilege.getAction());
HivePrivilegeObject hivePrivilegeObject =
convert2HivePrivilegeObject(tPrivilege);
// now sentry don't show grantor of a privilege
HivePrincipal grantor = new HivePrincipal(UNKONWN_GRANTOR,
HivePrincipalType.ROLE);
boolean grantOption =
tPrivilege.getGrantOption().equals(TSentryGrantOption.TRUE) ? true :
false;
return new HivePrivilegeInfo(principal, hivePrivilege, hivePrivilegeObject,
grantor,
grantOption, (int) tPrivilege.getCreateTime());
}
{code}
{code}
New code uses "(int)(tPrivilege.getCreateTime() / 1000)"
public static HivePrivilegeInfo convert2HivePrivilegeInfo(TSentryPrivilege
tPrivilege,
HivePrincipal principal) {
HivePrivilege hivePrivilege = convert2HivePrivilege(tPrivilege.getAction());
HivePrivilegeObject hivePrivilegeObject =
convert2HivePrivilegeObject(tPrivilege);
// now sentry don't show grantor of a privilege
HivePrincipal grantor = new HivePrincipal(UNKONWN_GRANTOR,
HivePrincipalType.ROLE);
// sentry CreateTime is the difference, measured in milliseconds,
// between the current time and midnight, January 1, 1970 UTC.
// hive granttime is in seconds. So need to convert the time.
int hiveCreateTime = (int)(tPrivilege.getCreateTime() / 1000);
boolean grantOption =
tPrivilege.getGrantOption().equals(TSentryGrantOption.TRUE) ? true :
false;
return new HivePrivilegeInfo(principal,
[jira] [Commented] (SENTRY-2134) Apply Hive URI grants recursively to subdirectories
[ https://issues.apache.org/jira/browse/SENTRY-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363281#comment-16363281 ] Ruslan Dautkhanov commented on SENTRY-2134: --- I think if you end up supporting URI grants too, it would have to act slightly differently: * ACLs for URI grant locations should be appended (merged with ) to ACLs that're already in hdfs * unlike ACLs for databases that overwrite hdfs ACLs This is because URI grants are normally given to tables in some locations that are accessed not by Hive for example, but there could be an external process that feeds data to that location and it's setup is done through HDFS acls. Just my two cents. But I agree it would be awesome to support URI grants through Sentry too - so we don't have to maintain this in Sentry and manually at HDFS level. Thank you. > Apply Hive URI grants recursively to subdirectories > --- > > Key: SENTRY-2134 > URL: https://issues.apache.org/jira/browse/SENTRY-2134 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Affects Versions: 1.8.0, 2.0.0, 1.7.1 >Reporter: Ruslan Dautkhanov >Priority: Major > Labels: hive, uri > > Currently we need to add direct grants for all Hive tables' LOCATIONs. > Like, 'hdfs_staging/table1', 'hdfs_staging/table2', etc.. > It's not manageable this way. - we can't add grants for each and every table. > It would be great if we could just do one grant - > 'hdfs_staging/' so it would automatically be applied to > 'hdfs_staging/table1', 'hdfs_staging/table2', and other subdirectories. > There is probably a reason this wasn't implemented earlier? Thanks for > considering this improvement. > Also found another user's request on this - > https://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-Sentry-GRANT-ALL-ON-URI-not-cascaded-down-through/td-p/39928 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2141) Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo correctly
Na Li created SENTRY-2141: - Summary: Sentry Privilege TimeStamp is not converted to grantTime in HivePrivilegeInfo correctly Key: SENTRY-2141 URL: https://issues.apache.org/jira/browse/SENTRY-2141 Project: Sentry Issue Type: Bug Affects Versions: 2.0.0, 2.1.0 Reporter: Na Li Assignee: Na Li sentry CreateTime is the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC. hive granttime is in seconds. So need to convert the time. The original code just cost the timestamp from long to int without converting milliseconds to seconds. Therefore, the timestamp value is wrong when retrieving the privilege from hive. The solution is to convert the time correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-853) Handle show grant on failure correctly
[
https://issues.apache.org/jira/browse/SENTRY-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363230#comment-16363230
]
Hadoop QA commented on SENTRY-853:
--
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12910459/SENTRY-853-001.patch
against master.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.thrift.TestSentryWebServerWithSSL
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3660/console
This message is automatically generated.
> Handle show grant on failure correctly
> -
>
> Key: SENTRY-853
> URL: https://issues.apache.org/jira/browse/SENTRY-853
> Project: Sentry
> Issue Type: Improvement
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-853-001.patch
>
>
> {noformat}
> 0: jdbc:hive2://a2110.halxg.cloudera.com:1000> show grant on table pageviews;
> Error: Error while compiling statement: FAILED: SemanticException Sentry does
> not allow privileges to be granted/revoked to/from: USER
> (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2134) Apply Hive URI grants recursively to subdirectories
[ https://issues.apache.org/jira/browse/SENTRY-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363215#comment-16363215 ] Sergio Peña commented on SENTRY-2134: - Agree. HDFS sync only syncs privileges granted to databases and tables, not URIs. If you grant a privilege at the database level, then this will apply the same privilege recursively on all the table locations. [~akolb] Do we have a plan to support URI on HDFS sync? > Apply Hive URI grants recursively to subdirectories > --- > > Key: SENTRY-2134 > URL: https://issues.apache.org/jira/browse/SENTRY-2134 > Project: Sentry > Issue Type: Improvement > Components: Hive Binding >Affects Versions: 1.8.0, 2.0.0, 1.7.1 >Reporter: Ruslan Dautkhanov >Priority: Major > Labels: hive, uri > > Currently we need to add direct grants for all Hive tables' LOCATIONs. > Like, 'hdfs_staging/table1', 'hdfs_staging/table2', etc.. > It's not manageable this way. - we can't add grants for each and every table. > It would be great if we could just do one grant - > 'hdfs_staging/' so it would automatically be applied to > 'hdfs_staging/table1', 'hdfs_staging/table2', and other subdirectories. > There is probably a reason this wasn't implemented earlier? Thanks for > considering this improvement. > Also found another user's request on this - > https://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-Sentry-GRANT-ALL-ON-URI-not-cascaded-down-through/td-p/39928 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2137) Improve and rework the CLI
[ https://issues.apache.org/jira/browse/SENTRY-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363202#comment-16363202 ] Sergio Peña commented on SENTRY-2137: - [~liamsargent] [~moist] There exists a command named 'sentryShell' which allows users to grant/revoke privileges through the command line: [https://cwiki.apache.org/confluence/display/SENTRY/Sentry+Simple+Shell] Is this Jira going to improve something? > Improve and rework the CLI > -- > > Key: SENTRY-2137 > URL: https://issues.apache.org/jira/browse/SENTRY-2137 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Steve Moist >Priority: Minor > > Sentry can be improved by moving all of the privilige actions for hive (such > as grant/revoke) from beeline and into a centralized CLI. With this we can > do operations such as show all privileges for a role across HDFS, Hive, > Impala, etc in a single location and administer this in a single location. > In a cluster, it would be good to have the sentry cli as a standalone > executable, so building in a REST API for Sentry use would be needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2140) Tag based access control
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363197#comment-16363197 ] Sergio Peña commented on SENTRY-2140: - Are these tags linked to a specific privilege? are roles tight to tags or tags are just a different identity linked to privileges > Tag based access control > > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "tags" that prevent users/roles from not accessing or seeing the data. For > users/roles that have that tag, they should be able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2139) Improve documentation to include how to work with other services
[ https://issues.apache.org/jira/browse/SENTRY-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363192#comment-16363192 ] Sergio Peña commented on SENTRY-2139: - This task sounds too general. Wouldn't be better to split this task in more specific documentation tasks? > Improve documentation to include how to work with other services > > > Key: SENTRY-2139 > URL: https://issues.apache.org/jira/browse/SENTRY-2139 > Project: Sentry > Issue Type: Task > Components: Docs >Affects Versions: 2.0.0 >Reporter: Steve Moist >Priority: Minor > > The docs could use more information on how to administer, setup, control > privileges and list out all the available commands to Sentry for each service. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object
[
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363162#comment-16363162
]
Hadoop QA commented on SENTRY-1242:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12910431/SENTRY-1242-001.patch
against master.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3659/console
This message is automatically generated.
> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch
>
>
> Enable show grant on table/db . This syntax is already supported by
> hive.
> This would be really useful for the admin to find out all policies on a hive
> object.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-853) Handle show grant on failure correctly
[
https://issues.apache.org/jira/browse/SENTRY-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363154#comment-16363154
]
Steve Moist commented on SENTRY-853:
I've changed the error message to show that the user can't perform a show
command rather than the existing one.
> Handle show grant on failure correctly
> -
>
> Key: SENTRY-853
> URL: https://issues.apache.org/jira/browse/SENTRY-853
> Project: Sentry
> Issue Type: Improvement
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-853-001.patch
>
>
> {noformat}
> 0: jdbc:hive2://a2110.halxg.cloudera.com:1000> show grant on table pageviews;
> Error: Error while compiling statement: FAILED: SemanticException Sentry does
> not allow privileges to be granted/revoked to/from: USER
> (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-853) Handle show grant on failure correctly
[
https://issues.apache.org/jira/browse/SENTRY-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated SENTRY-853:
---
Attachment: SENTRY-853-001.patch
> Handle show grant on failure correctly
> -
>
> Key: SENTRY-853
> URL: https://issues.apache.org/jira/browse/SENTRY-853
> Project: Sentry
> Issue Type: Improvement
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-853-001.patch
>
>
> {noformat}
> 0: jdbc:hive2://a2110.halxg.cloudera.com:1000> show grant on table pageviews;
> Error: Error while compiling statement: FAILED: SemanticException Sentry does
> not allow privileges to be granted/revoked to/from: USER
> (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-853) Handle show grant on failure correctly
[
https://issues.apache.org/jira/browse/SENTRY-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated SENTRY-853:
---
Fix Version/s: 2.1.0
Status: Patch Available (was: In Progress)
> Handle show grant on failure correctly
> -
>
> Key: SENTRY-853
> URL: https://issues.apache.org/jira/browse/SENTRY-853
> Project: Sentry
> Issue Type: Improvement
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-853-001.patch
>
>
> {noformat}
> 0: jdbc:hive2://a2110.halxg.cloudera.com:1000> show grant on table pageviews;
> Error: Error while compiling statement: FAILED: SemanticException Sentry does
> not allow privileges to be granted/revoked to/from: USER
> (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2115) Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
[
https://issues.apache.org/jira/browse/SENTRY-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kalyan kumar kalvagadda updated SENTRY-2115:
Attachment: SENTRY-2115.002.patch
> Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
> -
>
> Key: SENTRY-2115
> URL: https://issues.apache.org/jira/browse/SENTRY-2115
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Critical
> Attachments: SENTRY-2115.001.patch, SENTRY-2115.002.patch
>
>
> *Current Behavior,*
> *Scenario-1:* When HDFS sync is disabled, and sentry is started for the first
> time.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-2:* When HDFS sync is disabled, and current event-id from HMS is
> less than last event-d processed by sentry
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-3:* When HDFS sync is disabled, and first event-id in the
> subsequent pull is not greater than the last event-id processed by sentry by
> 1.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into
> SENTRY_HMS_NOTIFICATION_ID.{color:#FF} This is wrong{color}
> *Scenario-4:* Initially HDFS sync was enabled and later disabled for while
> and then HDFS sync is enabled and sentry service is restarted to get it to
> effect.
> * On disabling HDFS sync, HMSFollower would update the
> SENTRY_HMS_NOTIFICATION_ID table but not the MSentryPathChange table.
> When HDFS sync is enabled again, HMSFollower would continue fetching new
> notifications and process them and update the MSentryPathChange and
> MAuthzPathsMapping info. This is not correct as sentry would not take a
> snapshot and will not have any path-mapping information about HMS objects. As
> a result, HDFS will ACL will not be added for the existing HMS
> objects.{color:#FF} This is wrong.{color}
> *Correct Behavior:*
> * Full snapshots need not be taken in all Scenario-1, Scenario-2 and
> Scenario-3.
> * When Sentry detects out-of-sync situations, it should reset
> SENTRY_HMS_NOTIFICATION_ID table and start processing the event in
> HMS_NOTIFICATION_LOG from beginning.
> * To handle scenario explained in *Scenario-4,* sentry should reset the
> mapping information when ever HDFS sync is disabled. That way it can learn
> from scratch when the feature is enabled back. There is no value is holding
> stale data even when we know it will have issues when the feature is enabled
> back.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2115) Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
[
https://issues.apache.org/jira/browse/SENTRY-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kalyan kumar kalvagadda updated SENTRY-2115:
Attachment: (was: SENTRY-2115.002.patch)
> Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
> -
>
> Key: SENTRY-2115
> URL: https://issues.apache.org/jira/browse/SENTRY-2115
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Critical
> Attachments: SENTRY-2115.001.patch, SENTRY-2115.002.patch
>
>
> *Current Behavior,*
> *Scenario-1:* When HDFS sync is disabled, and sentry is started for the first
> time.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-2:* When HDFS sync is disabled, and current event-id from HMS is
> less than last event-d processed by sentry
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-3:* When HDFS sync is disabled, and first event-id in the
> subsequent pull is not greater than the last event-id processed by sentry by
> 1.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into
> SENTRY_HMS_NOTIFICATION_ID.{color:#FF} This is wrong{color}
> *Scenario-4:* Initially HDFS sync was enabled and later disabled for while
> and then HDFS sync is enabled and sentry service is restarted to get it to
> effect.
> * On disabling HDFS sync, HMSFollower would update the
> SENTRY_HMS_NOTIFICATION_ID table but not the MSentryPathChange table.
> When HDFS sync is enabled again, HMSFollower would continue fetching new
> notifications and process them and update the MSentryPathChange and
> MAuthzPathsMapping info. This is not correct as sentry would not take a
> snapshot and will not have any path-mapping information about HMS objects. As
> a result, HDFS will ACL will not be added for the existing HMS
> objects.{color:#FF} This is wrong.{color}
> *Correct Behavior:*
> * Full snapshots need not be taken in all Scenario-1, Scenario-2 and
> Scenario-3.
> * When Sentry detects out-of-sync situations, it should reset
> SENTRY_HMS_NOTIFICATION_ID table and start processing the event in
> HMS_NOTIFICATION_LOG from beginning.
> * To handle scenario explained in *Scenario-4,* sentry should reset the
> mapping information when ever HDFS sync is disabled. That way it can learn
> from scratch when the feature is enabled back. There is no value is holding
> stale data even when we know it will have issues when the feature is enabled
> back.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-853) Handle show grant on failure correctly
[
https://issues.apache.org/jira/browse/SENTRY-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363063#comment-16363063
]
Steve Moist commented on SENTRY-853:
I encountered this issue during work on SENTRY-1242, the error message is
unhelpful and incorrect.
> Handle show grant on failure correctly
> -
>
> Key: SENTRY-853
> URL: https://issues.apache.org/jira/browse/SENTRY-853
> Project: Sentry
> Issue Type: Improvement
>Reporter: Sravya Tirukkovalur
>Priority: Major
>
> {noformat}
> 0: jdbc:hive2://a2110.halxg.cloudera.com:1000> show grant on table pageviews;
> Error: Error while compiling statement: FAILED: SemanticException Sentry does
> not allow privileges to be granted/revoked to/from: USER
> (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (SENTRY-853) Handle show grant on failure correctly
[
https://issues.apache.org/jira/browse/SENTRY-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist reassigned SENTRY-853:
--
Assignee: Steve Moist
> Handle show grant on failure correctly
> -
>
> Key: SENTRY-853
> URL: https://issues.apache.org/jira/browse/SENTRY-853
> Project: Sentry
> Issue Type: Improvement
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
>
> {noformat}
> 0: jdbc:hive2://a2110.halxg.cloudera.com:1000> show grant on table pageviews;
> Error: Error while compiling statement: FAILED: SemanticException Sentry does
> not allow privileges to be granted/revoked to/from: USER
> (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-2115) Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
[
https://issues.apache.org/jira/browse/SENTRY-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363023#comment-16363023
]
Hadoop QA commented on SENTRY-2115:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12910428/SENTRY-2115.002.patch
against master.
{color:red}Overall:{color} -1 due to 4 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationWithHA
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationWithHA
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationWithHA
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/3658/console
This message is automatically generated.
> Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
> -
>
> Key: SENTRY-2115
> URL: https://issues.apache.org/jira/browse/SENTRY-2115
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Critical
> Attachments: SENTRY-2115.001.patch, SENTRY-2115.002.patch
>
>
> *Current Behavior,*
> *Scenario-1:* When HDFS sync is disabled, and sentry is started for the first
> time.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-2:* When HDFS sync is disabled, and current event-id from HMS is
> less than last event-d processed by sentry
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-3:* When HDFS sync is disabled, and first event-id in the
> subsequent pull is not greater than the last event-id processed by sentry by
> 1.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into
> SENTRY_HMS_NOTIFICATION_ID.{color:#FF} This is wrong{color}
> *Scenario-4:* Initially HDFS sync was enabled and later disabled for while
> and then HDFS sync is enabled and sentry service is restarted to get it to
> effect.
> * On disabling HDFS sync, HMSFollower would update the
> SENTRY_HMS_NOTIFICATION_ID table but not the MSentryPathChange table.
> When HDFS sync is enabled again, HMSFollower would continue fetching new
> notifications and process them and update the MSentryPathChange and
> MAuthzPathsMapping info. This is not correct as sentry would not take a
> snapshot and will not have any path-mapping information about HMS objects. As
> a result, HDFS will ACL will not be added for the existing HMS
> objects.{color:#FF} This is wrong.{color}
> *Correct Behavior:*
> * Full snapshots need not be taken in all Scenario-1, Scenario-2 and
> Scenario-3.
> * When Sentry detects out-of-sync situations, it should reset
> SENTRY_HMS_NOTIFICATION_ID table and start processing the event in
> HMS_NOTIFICATION_LOG from beginning.
> * To handle scenario explained in *Scenario-4,* sentry should reset the
> mapping information when ever HDFS sync is disabled. That way it can learn
> from scratch when the feature is enabled back. There is no value is holding
> stale data even when we know it will have issues when the feature is enabled
> back.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object
[ https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362995#comment-16362995 ] Steve Moist commented on SENTRY-1242: - https://reviews.apache.org/r/65639/ > Enable getting all privileges on a hive object > -- > > Key: SENTRY-1242 > URL: https://issues.apache.org/jira/browse/SENTRY-1242 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Sravya Tirukkovalur >Assignee: Steve Moist >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-1242-001.patch > > > Enable show grant on table/db . This syntax is already supported by > hive. > This would be really useful for the admin to find out all policies on a hive > object. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object
[ https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362988#comment-16362988 ] Steve Moist commented on SENTRY-1242: - Added the ability to do "show grant on table " and "show grant on database ". This only works for the current user to see their grants. > Enable getting all privileges on a hive object > -- > > Key: SENTRY-1242 > URL: https://issues.apache.org/jira/browse/SENTRY-1242 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Sravya Tirukkovalur >Assignee: Steve Moist >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-1242-001.patch > > > Enable show grant on table/db . This syntax is already supported by > hive. > This would be really useful for the admin to find out all policies on a hive > object. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-1242) Enable getting all privileges on a hive object
[ https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Moist updated SENTRY-1242: Fix Version/s: 2.1.0 Affects Version/s: 2.0.0 Status: Patch Available (was: In Progress) > Enable getting all privileges on a hive object > -- > > Key: SENTRY-1242 > URL: https://issues.apache.org/jira/browse/SENTRY-1242 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Sravya Tirukkovalur >Assignee: Steve Moist >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-1242-001.patch > > > Enable show grant on table/db . This syntax is already supported by > hive. > This would be really useful for the admin to find out all policies on a hive > object. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-1242) Enable getting all privileges on a hive object
[ https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Moist updated SENTRY-1242: Attachment: SENTRY-1242-001.patch > Enable getting all privileges on a hive object > -- > > Key: SENTRY-1242 > URL: https://issues.apache.org/jira/browse/SENTRY-1242 > Project: Sentry > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Sravya Tirukkovalur >Assignee: Steve Moist >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-1242-001.patch > > > Enable show grant on table/db . This syntax is already supported by > hive. > This would be really useful for the admin to find out all policies on a hive > object. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2115) Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
[
https://issues.apache.org/jira/browse/SENTRY-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kalyan kumar kalvagadda updated SENTRY-2115:
Attachment: SENTRY-2115.002.patch
> Incorrect behavior of HMsFollower when HDFSSync feature is disabled.
> -
>
> Key: SENTRY-2115
> URL: https://issues.apache.org/jira/browse/SENTRY-2115
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
>Priority: Critical
> Attachments: SENTRY-2115.001.patch, SENTRY-2115.002.patch
>
>
> *Current Behavior,*
> *Scenario-1:* When HDFS sync is disabled, and sentry is started for the first
> time.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-2:* When HDFS sync is disabled, and current event-id from HMS is
> less than last event-d processed by sentry
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into SENTRY_HMS_NOTIFICATION_ID.
> {color:#FF}This is wrong{color}
> *Scenario-3:* When HDFS sync is disabled, and first event-id in the
> subsequent pull is not greater than the last event-id processed by sentry by
> 1.
> * Sentry would take a full snapshot of HMS and just persists the event-id of
> the current notification-id of HMS into
> SENTRY_HMS_NOTIFICATION_ID.{color:#FF} This is wrong{color}
> *Scenario-4:* Initially HDFS sync was enabled and later disabled for while
> and then HDFS sync is enabled and sentry service is restarted to get it to
> effect.
> * On disabling HDFS sync, HMSFollower would update the
> SENTRY_HMS_NOTIFICATION_ID table but not the MSentryPathChange table.
> When HDFS sync is enabled again, HMSFollower would continue fetching new
> notifications and process them and update the MSentryPathChange and
> MAuthzPathsMapping info. This is not correct as sentry would not take a
> snapshot and will not have any path-mapping information about HMS objects. As
> a result, HDFS will ACL will not be added for the existing HMS
> objects.{color:#FF} This is wrong.{color}
> *Correct Behavior:*
> * Full snapshots need not be taken in all Scenario-1, Scenario-2 and
> Scenario-3.
> * When Sentry detects out-of-sync situations, it should reset
> SENTRY_HMS_NOTIFICATION_ID table and start processing the event in
> HMS_NOTIFICATION_LOG from beginning.
> * To handle scenario explained in *Scenario-4,* sentry should reset the
> mapping information when ever HDFS sync is disabled. That way it can learn
> from scratch when the feature is enabled back. There is no value is holding
> stale data even when we know it will have issues when the feature is enabled
> back.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
