[jira] [Assigned] (RANGER-3995) Policy update request fails if isDenyAllElse flag is set true in request json when using "/policy/apply" API
[
https://issues.apache.org/jira/browse/RANGER-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni reassigned RANGER-3995:
--
Assignee: Abhay Kulkarni
> Policy update request fails if isDenyAllElse flag is set true in request json
> when using "/policy/apply" API
>
>
> Key: RANGER-3995
> URL: https://issues.apache.org/jira/browse/RANGER-3995
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pradeep Agrawal
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> An issue was found with respect to "/policy/apply" API when the policy json
> contains denyAllElse flag set true.
> When a request is made to "/service/public/v2/api/policy/apply" using a
> policy json where denyAllElse flag is set to true,
> the policy is created successfully the first time.
> But if a request is made again using the same policy json, it gives the
> following error.
> {code:java}
> {
> "statusCode": 1,
> "msgDesc": "(0) Validation failure: error code[3049], reason[Deny or
> deny-exceptions are not supported if policy has isDenyAllElse flag set to
> true], field[policy items], subfield[null], type[] "
> } {code}
> Steps to reproduce :-
> 1. Make a POST request to the below mentioned API endpoint, using a policy
> json where isDenyAllElse flag is set true
> /service/public/v2/api/policy/apply
> 2. Fetch the policy using the newly created policy id, and try to make a POST
> request to "/policy/apply" using the policy json obtained from the GET
> request. The request results in an error
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (RANGER-3999) Implement more efficient way to handle _any access authorization
[ https://issues.apache.org/jira/browse/RANGER-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reassigned RANGER-3999: -- Assignee: Abhay Kulkarni > Implement more efficient way to handle _any access authorization > > > Key: RANGER-3999 > URL: https://issues.apache.org/jira/browse/RANGER-3999 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay >Assignee: Abhay Kulkarni >Priority: Major > > If a user-initiated operation requires checking if more than one permission > is granted, then currently, each permission requires a call to internal > Policy Engine API for the same accessed resource. This leads to many > repetitive computations which may be avoided if the policy engine API > supports multiple permissions. In that case, optimization may be achieved by > pushing authorization for multiple permissions down to the lowest possible > level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4000) unit test failure in JDK17
[
https://issues.apache.org/jira/browse/RANGER-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644051#comment-17644051
]
Abhay Kulkarni commented on RANGER-4000:
Ship it!
> unit test failure in JDK17
> --
>
> Key: RANGER-4000
> URL: https://issues.apache.org/jira/browse/RANGER-4000
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: RANGER-4000.patch
>
>
> Following expression in test_policyengine_geo.json fails with JDK17 (script
> engine from org.graalvm.js):
> {code:java}
> (!!country_code_format_long && !!country_code_format_dot &&
> (country_code_format_long == country_code_format_dot)){code}
>
> This expression should be updated with:
> {noformat}
> (country_code_format_long != null && country_code_format_dot != null &&
> (country_code_format_long == country_code_format_dot)) {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4007) HDFS Authorizer changes to take advantage of support for multiple access-types in the Ranger Access Request
Abhay Kulkarni created RANGER-4007: -- Summary: HDFS Authorizer changes to take advantage of support for multiple access-types in the Ranger Access Request Key: RANGER-4007 URL: https://issues.apache.org/jira/browse/RANGER-4007 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger authorizer plugin for HDFS, when asked to authorize certain accesses, requires to check if the user is authorized to do more than one permission for the requested resource. Currently, this is achieved by the plugin making separate call into Ranger policy engine to check authorization for each such permission. This works, but is inefficient. A recent improvement in Ranger policy engine (RANGER-3999) adds support for specifying multiple permissions that need to be authorized for a resource in one Policy engine call. This Jira updates the Ranger plugin for HDFS to use this feature, thereby improving performance of access authorization for certain access requests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-4009) Open access to some Policy Engine objects and metrics
[ https://issues.apache.org/jira/browse/RANGER-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reassigned RANGER-4009: -- Assignee: Abhay Kulkarni > Open access to some Policy Engine objects and metrics > - > > Key: RANGER-4009 > URL: https://issues.apache.org/jira/browse/RANGER-4009 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > It is useful to make some internal policy engine APIs and metrics accessible > to public to help make more efficient use of policy engine resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4009) Open access to some Policy Engine objects and metrics
Abhay Kulkarni created RANGER-4009: -- Summary: Open access to some Policy Engine objects and metrics Key: RANGER-4009 URL: https://issues.apache.org/jira/browse/RANGER-4009 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni It is useful to make some internal policy engine APIs and metrics accessible to public to help make more efficient use of policy engine resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4009) Open read access to some Policy Engine objects and metrics
[ https://issues.apache.org/jira/browse/RANGER-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4009: --- Summary: Open read access to some Policy Engine objects and metrics (was: Open access to some Policy Engine objects and metrics) > Open read access to some Policy Engine objects and metrics > -- > > Key: RANGER-4009 > URL: https://issues.apache.org/jira/browse/RANGER-4009 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > It is useful to make some internal policy engine APIs and metrics accessible > to public to help make more efficient use of policy engine resources. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3409) Update Jackson and remove Codehaus version
[ https://issues.apache.org/jira/browse/RANGER-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651797#comment-17651797 ] Abhay Kulkarni commented on RANGER-3409: [~bpatel] There is some work being done to formalize the serialization/deserialization of Java objects to JSON used throughout the Ranger code-base, mainly, to ensure only one library is used. This Jira may be used to track the progress. > Update Jackson and remove Codehaus version > -- > > Key: RANGER-3409 > URL: https://issues.apache.org/jira/browse/RANGER-3409 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Andrew Charneski >Priority: Blocker > > An old version of Jackson (Codehaus Jackson 1.9.13) is still being used. > Jackson has since moved namespaces with a reorganized library structure. > Update all references to the older version to use the newer version (which is > currently used in some modules). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4069) Add performance tracing instrumentation to Tag Enricher
Abhay Kulkarni created RANGER-4069: -- Summary: Add performance tracing instrumentation to Tag Enricher Key: RANGER-4069 URL: https://issues.apache.org/jira/browse/RANGER-4069 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger enriches the access request before it is evaluated for access. One of important steps during enrichment is computing applicable tags and add them to the request's context. Tag enricher is used to figure out the set of applicable tags. This Jira adds performance tracing instrumentation to measure time taken by Tag enricher code to find matching tags. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4070) Provide mechanism to manage potentially multiple enrichment of an access request
Abhay Kulkarni created RANGER-4070: -- Summary: Provide mechanism to manage potentially multiple enrichment of an access request Key: RANGER-4070 URL: https://issues.apache.org/jira/browse/RANGER-4070 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger enriches the access request before it is evaluated for access. The request enrichment is done when Ranger APIs provided in for access evaluation are invoked. There may be some control flows where there is a potential of same access request getting enriched multiple times. Request enrichment may be an expensive operation, hence a mechanism to control access enrichment is such scenarios is desirable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-3999) Implement more efficient way to handle _any access authorization
[ https://issues.apache.org/jira/browse/RANGER-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644065#comment-17644065 ] Abhay Kulkarni edited comment on RANGER-3999 at 2/13/23 10:37 PM: -- Commit details: master: [https://github.com/apache/ranger/commit/56d5bf9173dc2c6d04692a07e67eace5e5d98ed4] Additional review: [https://reviews.apache.org/r/74304/] and commit detail: [https://github.com/apache/ranger/commit/7a7215f67e7db807ee0401f2b41d7bb871a248f5] was (Author: abhayk): Commit details: master: https://github.com/apache/ranger/commit/56d5bf9173dc2c6d04692a07e67eace5e5d98ed4 > Implement more efficient way to handle _any access authorization > > > Key: RANGER-3999 > URL: https://issues.apache.org/jira/browse/RANGER-3999 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay >Assignee: Abhay Kulkarni >Priority: Major > > If a user-initiated operation requires checking if more than one permission > is granted, then currently, each permission requires a call to internal > Policy Engine API for the same accessed resource. This leads to many > repetitive computations which may be avoided if the policy engine API > supports multiple permissions. In that case, optimization may be achieved by > pushing authorization for multiple permissions down to the lowest possible > level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-3999) Implement more efficient way to handle _any access authorization
[ https://issues.apache.org/jira/browse/RANGER-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644065#comment-17644065 ] Abhay Kulkarni edited comment on RANGER-3999 at 2/16/23 6:41 PM: - Commit details: master: [https://github.com/apache/ranger/commit/56d5bf9173dc2c6d04692a07e67eace5e5d98ed4] Additional review: [https://reviews.apache.org/r/74304/] and commit detail: [https://github.com/apache/ranger/commit/7a7215f67e7db807ee0401f2b41d7bb871a248f5] Additional review: [https://reviews.apache.org/r/74308/] and commit details: https://github.com/apache/ranger/commit/4b941b2f0d7a8390155c61fa0960c42aa8a37b69 was (Author: abhayk): Commit details: master: [https://github.com/apache/ranger/commit/56d5bf9173dc2c6d04692a07e67eace5e5d98ed4] Additional review: [https://reviews.apache.org/r/74304/] and commit detail: [https://github.com/apache/ranger/commit/7a7215f67e7db807ee0401f2b41d7bb871a248f5] > Implement more efficient way to handle _any access authorization > > > Key: RANGER-3999 > URL: https://issues.apache.org/jira/browse/RANGER-3999 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay >Assignee: Abhay Kulkarni >Priority: Major > > If a user-initiated operation requires checking if more than one permission > is granted, then currently, each permission requires a call to internal > Policy Engine API for the same accessed resource. This leads to many > repetitive computations which may be avoided if the policy engine API > supports multiple permissions. In that case, optimization may be achieved by > pushing authorization for multiple permissions down to the lowest possible > level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4100) Compute the smallest set of evaluators returned by search of multiple Trie trees
Abhay Kulkarni created RANGER-4100: -- Summary: Compute the smallest set of evaluators returned by search of multiple Trie trees Key: RANGER-4100 URL: https://issues.apache.org/jira/browse/RANGER-4100 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni The algorithm that computes the intersection of evaluators returned when multiple trie structures are searched depends on the order in which the trie structures are searched. This is sub-optimal, when the evaluators returned by trie structure that is searched earlier happens to be a bigger set then the evaluators by the search of later trie structures. This Jira updates the intersection computation to be more optimal and not dependent for performance on the order of search of the trie structures. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4100) Efficient computation of the smallest set of evaluators returned by search of multiple Trie trees
[ https://issues.apache.org/jira/browse/RANGER-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4100: --- Summary: Efficient computation of the smallest set of evaluators returned by search of multiple Trie trees (was: Compute the smallest set of evaluators returned by search of multiple Trie trees) > Efficient computation of the smallest set of evaluators returned by search of > multiple Trie trees > - > > Key: RANGER-4100 > URL: https://issues.apache.org/jira/browse/RANGER-4100 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > The algorithm that computes the intersection of evaluators returned when > multiple trie structures are searched depends on the order in which the trie > structures are searched. This is sub-optimal, when the evaluators returned by > trie structure that is searched earlier happens to be a bigger set then the > evaluators by the search of later trie structures. > This Jira updates the intersection computation to be more optimal and not > dependent for performance on the order of search of the trie structures. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4108) Speed up loading of the service-definitions during Ranger startup
Abhay Kulkarni created RANGER-4108: -- Summary: Speed up loading of the service-definitions during Ranger startup Key: RANGER-4108 URL: https://issues.apache.org/jira/browse/RANGER-4108 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Currently, Service Definitions are persisted in several database tables in a normalized form. However, as operations on a Service Definition object is not granular, but act on the object as a whole, there is not much advantage in normalizing it for storage, and a distinct disadvantage of having to recreate it by accessing multiple database tables during the initialization of Ranger admin process. This Jira denormalizes service-definition object's persistent state and also removes several tables used only for normalizing service-definition object. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4115) Tags containing attributes not processed correctly by tagsync
Abhay Kulkarni created RANGER-4115: -- Summary: Tags containing attributes not processed correctly by tagsync Key: RANGER-4115 URL: https://issues.apache.org/jira/browse/RANGER-4115 Project: Ranger Issue Type: Bug Components: tagsync Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni During construction of a JSON payload to uploade to Ranger admin when processing CLASSIFICATION_ADD event, tagsync throws NullPointerException. This is due to recent change to RangerTagDef class by RANGER-3951 (optimize memory used by tags in plugins and server). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3987) Potential risk of OOM
[
https://issues.apache.org/jira/browse/RANGER-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697180#comment-17697180
]
Abhay Kulkarni commented on RANGER-3987:
A potential solution could be to execute doCreateOrUpdateXXPluginInfo() in its
own transaction (instead of queuing it up to be executed after the original
transaction is completed) if the following condition is true:
httpCode == HttpServletResponse.SC_NOT_MODIFIED.
> Potential risk of OOM
> -
>
> Key: RANGER-3987
> URL: https://issues.apache.org/jira/browse/RANGER-3987
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.2.0
>Reporter: KyrieG
>Assignee: KyrieG
>Priority: Critical
>
> During every policy-loading process of other components, the attribute
> "LastActivationTimeInMillis" is always set to System.currentTimeMillis(). See
> loadPolicy():
> {code:java}
> // from PolicyRefresher.java loadPolicy()
> //load policy from PolicyAdmin
> ServicePolicies svcPolicies = loadPolicyfromPolicyAdmin();
> if (svcPolicies == null) {
>//if Policy fetch from Policy Admin Fails, load from cache
>if (!policiesSetInPlugin) {
> svcPolicies = loadFromCache();
>}
> }
> if (PERF_POLICYENGINE_INIT_LOG.isDebugEnabled()) {
>long freeMemory = Runtime.getRuntime().freeMemory();
>long totalMemory = Runtime.getRuntime().totalMemory();
>PERF_POLICYENGINE_INIT_LOG.debug("In-Use memory: " + (totalMemory -
> freeMemory) + ", Free memory:" + freeMemory);
> }
> if (svcPolicies != null) {
>plugIn.setPolicies(svcPolicies);
>policiesSetInPlugin = true;
>serviceDefSetInPlugin = false;
>setLastActivationTimeInMillis(System.currentTimeMillis()); // always
> updated during each policy loading
>lastKnownVersion = svcPolicies.getPolicyVersion() != null ?
> svcPolicies.getPolicyVersion() : -1L;
> } else {
>if (!policiesSetInPlugin && !serviceDefSetInPlugin) {
> plugIn.setPolicies(null);
> serviceDefSetInPlugin = true;
>}
> } {code}
> In this case, the column "info" from table "x_plugin_info" would always need
> to be updated since it is a json string containing activationTime. See
> doCreateOrUpdateXXPluginInfo():
> {code:java}
> // from AssetMgr, doCreateOrUpdateXXPluginInfo().
> if (lastPolicyActivationTime != null && lastPolicyActivationTime > 0 &&
> (dbObj.getPolicyActivationTime() == null ||
> !dbObj.getPolicyActivationTime().equals(lastPolicyActivationTime))) {
>dbObj.setPolicyActivationTime(lastPolicyActivationTime);
>needsUpdating = true;
> } {code}
> Since doCreateOrUpdateXXPluginInfo() is a Runnble committed to
> RangerTransactionService. (RangerTransactionSynchronizationAdapter in Ranger
> 2.3.0 though, the risk might still be there). Also see
> doCreateOrUpdateXXPluginInfo():
> {code:java}
> // code placeholder
> commitWork = new Runnable() {
>@Override
>public void run() {
> doCreateOrUpdateXXPluginInfo(pluginInfo, entityType,
> isTagVersionResetNeeded, clusterName);
>}
> };
> ...
> activityLogger.commitAfterTransactionComplete(commitWork);{code}
> RangerTransactionService use a thread pool with unlimited work queue,
> ScheduledExecutorService, to store extra Runnables.
> In our cases, there are 1000+ hive and hbase instances, the ranger admin
> seems to be under tremendous pressure becuase every instance would
> periodically request policy-downloading API and trigger an update of the
> table "x_plugin_info". Since the core thread pool seems to be poor and DB is
> also likely under pressure, the work queue is stacking, leaking out JVM Heap
> and causing OOM finally.
> I think adding more core threads would help, but when the system grow, this
> part of code would bring a lot overhead, is there any solution?
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4129) ArrayIndexOutOfBounds exception may be thrown while processing events
Abhay Kulkarni created RANGER-4129: -- Summary: ArrayIndexOutOfBounds exception may be thrown while processing events Key: RANGER-4129 URL: https://issues.apache.org/jira/browse/RANGER-4129 Project: Ranger Issue Type: Bug Components: tagsync Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni If AtlasTagSource.buildAndUploadServiceTags() is called with empty AtlasTagSource.atlasEntityWithTags list, then an ArrayIndexOutOfBounds exception is thrown when AtlasTagSource.messages list is read. This may happen when the first notification processed by tagsync process is of type ENTITY_DELETE. This Jira also fixes a potential issue if the Kafka queue contains only a large number of notifications not expected to be handled by tagsync. In such cases, tagsync will not acknowledge receipt and it may cause back-pressure on Kafka queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4129) ArrayIndexOutOfBounds exception may be thrown while processing events
[ https://issues.apache.org/jira/browse/RANGER-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4129: --- Description: If AtlasTagSource.buildAndUploadServiceTags() is called with empty AtlasTagSource.atlasEntityWithTags list, then an ArrayIndexOutOfBounds exception is thrown when AtlasTagSource.messages list is read. This may happen when the first notification processed by tagsync process is of type ENTITY_DELETE. (was: If AtlasTagSource.buildAndUploadServiceTags() is called with empty AtlasTagSource.atlasEntityWithTags list, then an ArrayIndexOutOfBounds exception is thrown when AtlasTagSource.messages list is read. This may happen when the first notification processed by tagsync process is of type ENTITY_DELETE. This Jira also fixes a potential issue if the Kafka queue contains only a large number of notifications not expected to be handled by tagsync. In such cases, tagsync will not acknowledge receipt and it may cause back-pressure on Kafka queue.) > ArrayIndexOutOfBounds exception may be thrown while processing events > - > > Key: RANGER-4129 > URL: https://issues.apache.org/jira/browse/RANGER-4129 > Project: Ranger > Issue Type: Bug > Components: tagsync >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > If AtlasTagSource.buildAndUploadServiceTags() is called with empty > AtlasTagSource.atlasEntityWithTags list, then an ArrayIndexOutOfBounds > exception is thrown when AtlasTagSource.messages list is read. This may > happen when the first notification processed by tagsync process is of type > ENTITY_DELETE. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4130) Improve performance of event processing in agsync by optimizing number of commits to Kafka broker
Abhay Kulkarni created RANGER-4130: -- Summary: Improve performance of event processing in agsync by optimizing number of commits to Kafka broker Key: RANGER-4130 URL: https://issues.apache.org/jira/browse/RANGER-4130 Project: Ranger Issue Type: Improvement Components: tagsync Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni tagsync, currently, commits to Kafka every handled event after successfully uploading to Ranger admin. The commits are synchronous and may impose performance penalty. This Jira is to update event handling in tagsync by optmizing on the commits to Kafka while reducing the chances of Kafka delivering same message multiple times to tagsync. This Jira also fixes a potential issue if the Kafka queue contains only a large number of notifications not handled by tagsync. In such cases, tagsync does not do any commit to kafka, and that may cause back-pressure on Kafka queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4136) Incorrect processing of tag-deltas by RangerTagEnricher
Abhay Kulkarni created RANGER-4136: -- Summary: Incorrect processing of tag-deltas by RangerTagEnricher Key: RANGER-4136 URL: https://issues.apache.org/jira/browse/RANGER-4136 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger supports incremental tag downloads when configured. When a mapping for a service-resource is changed, the tag download request provides only the updated service-resource. RangerTagEnricher is expected to remove the existing service-resource and the associated tags from its cache and add it back with updated associated tags. Current code does not remove the existing service-resource and its associated tags correctly leading to incorrect tags to be retrieved for the service-resource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (RANGER-4136) Incorrect processing of tag-deltas by RangerTagEnricher
[ https://issues.apache.org/jira/browse/RANGER-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reopened RANGER-4136: > Incorrect processing of tag-deltas by RangerTagEnricher > --- > > Key: RANGER-4136 > URL: https://issues.apache.org/jira/browse/RANGER-4136 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Ranger supports incremental tag downloads when configured. When a mapping for > a service-resource is changed, the tag download request provides only the > updated service-resource. RangerTagEnricher is expected to remove the > existing service-resource and the associated tags from its cache and add it > back with updated associated tags. > Current code does not remove the existing service-resource and its associated > tags correctly leading to incorrect tags to be retrieved for the > service-resource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4136) Incorrect processing of tag-deltas by RangerTagEnricher
[ https://issues.apache.org/jira/browse/RANGER-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702023#comment-17702023 ] Abhay Kulkarni edited comment on RANGER-4136 at 3/28/23 3:35 AM: - Patch is at the review board: [https://reviews.apache.org/r/74350/] Additional patch is posted at the review board: https://reviews.apache.org/r/74366/ was (Author: abhayk): Patch is at the review board: https://reviews.apache.org/r/74350/ > Incorrect processing of tag-deltas by RangerTagEnricher > --- > > Key: RANGER-4136 > URL: https://issues.apache.org/jira/browse/RANGER-4136 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Ranger supports incremental tag downloads when configured. When a mapping for > a service-resource is changed, the tag download request provides only the > updated service-resource. RangerTagEnricher is expected to remove the > existing service-resource and the associated tags from its cache and add it > back with updated associated tags. > Current code does not remove the existing service-resource and its associated > tags correctly leading to incorrect tags to be retrieved for the > service-resource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4185) Improve debugging messages when policy-deltas are enabled
Abhay Kulkarni created RANGER-4185: -- Summary: Improve debugging messages when policy-deltas are enabled Key: RANGER-4185 URL: https://issues.apache.org/jira/browse/RANGER-4185 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Update debug logging to add more pertinent messages -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-4192) A higher priority Data-masking policy is not considered when computing Datamask type
[ https://issues.apache.org/jira/browse/RANGER-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reassigned RANGER-4192: -- Assignee: Abhay Kulkarni > A higher priority Data-masking policy is not considered when computing > Datamask type > > > Key: RANGER-4192 > URL: https://issues.apache.org/jira/browse/RANGER-4192 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Consider the following scenario: > # A tag-based masking policy masks certain column with a certain mask-type. > # A higher priority resource-based masking policy masks the same column with > a different mask-type. > In such scenario, the policy-engine reports the mask-type as one specified by > tag-based masking policy, but it should report mask-type as one specified by > a resource-based masking policy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4192) A higher priority Data-masking policy is not considered when computing Datamask type
Abhay Kulkarni created RANGER-4192: -- Summary: A higher priority Data-masking policy is not considered when computing Datamask type Key: RANGER-4192 URL: https://issues.apache.org/jira/browse/RANGER-4192 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Consider the following scenario: # A tag-based masking policy masks certain column with a certain mask-type. # A higher priority resource-based masking policy masks the same column with a different mask-type. In such scenario, the policy-engine reports the mask-type as one specified by tag-based masking policy, but it should report mask-type as one specified by a resource-based masking policy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4136) Incorrect processing of tag-deltas by RangerTagEnricher
[ https://issues.apache.org/jira/browse/RANGER-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703766#comment-17703766 ] Abhay Kulkarni edited comment on RANGER-4136 at 4/17/23 6:04 PM: - Commit details: master: [https://github.com/apache/ranger/commit/2e224cf9d4d28f3e23b5f8462a92024993a104bc] https://github.com/apache/ranger/commit/fb63f21cf6f5007f178eef8f11f68cf2c9a57279 was (Author: abhayk): Commit details: master: https://github.com/apache/ranger/commit/2e224cf9d4d28f3e23b5f8462a92024993a104bc > Incorrect processing of tag-deltas by RangerTagEnricher > --- > > Key: RANGER-4136 > URL: https://issues.apache.org/jira/browse/RANGER-4136 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Ranger supports incremental tag downloads when configured. When a mapping for > a service-resource is changed, the tag download request provides only the > updated service-resource. RangerTagEnricher is expected to remove the > existing service-resource and the associated tags from its cache and add it > back with updated associated tags. > Current code does not remove the existing service-resource and its associated > tags correctly leading to incorrect tags to be retrieved for the > service-resource. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4193) ServiceTagsProcessor fails to handle update of an existing Service-Resource
Abhay Kulkarni created RANGER-4193: -- Summary: ServiceTagsProcessor fails to handle update of an existing Service-Resource Key: RANGER-4193 URL: https://issues.apache.org/jira/browse/RANGER-4193 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni When an ENTITY_UPDATE notification is processed by tagsync process, it invokes Ranger admin's TagREST.importServiceTags() REST endpoint to add or update corresponding service-resource. However, if the incoming service-resource contains a GUID value, and Ranger admin's database does not have a service-resource that matches the GUID value, then instead of searching the database for the resource-signature, the implementation tries to create a service-resource. Such creation may fail with violation of uniqueness constraint for resource-signature value, resulting in the notification not correctly handled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4216) Reduce the interaction with HiveMetaStore when figuring out the owner of Hive object - Update patch for RANGER-3593
Abhay Kulkarni created RANGER-4216: -- Summary: Reduce the interaction with HiveMetaStore when figuring out the owner of Hive object - Update patch for RANGER-3593 Key: RANGER-4216 URL: https://issues.apache.org/jira/browse/RANGER-4216 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Patch for RANGER-3593 calls HiveMetaStore API to get owner of a Hive object to be checked for access. This is likely to cause performance and other issues during Hive operations. This patch needs to be modified appropriately to call HiveMetaStore APIs only if the Hive object does not owner information. The Hive Jira (https://issues.apache.org/jira/browse/HIVE-21833) needs to be merged into the upstream and any other Hive Jiras (such as https://issues.apache.org/jira/browse/HIVE-27285) that ensure that owner information is populated in the Hive objects before the access evaluation is called, also need to be merged in upstream Apache code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4219) Grant permission in Impala engine not working with {user} in ranger policy
Abhay Kulkarni created RANGER-4219:
--
Summary: Grant permission in Impala engine not working with {user}
in ranger policy
Key: RANGER-4219
URL: https://issues.apache.org/jira/browse/RANGER-4219
Project: Ranger
Issue Type: Bug
Components: Ranger
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni
1. users "test" and user has its own database "utest"
2. Ranger policy to grant permissions:
—
Database: u
{USER}
table: *
columns: *
Allow Conditions: user: \{USER}
permission: ALL +Delegate Admin
—
The grant does not work for Impala GRANTs. The following error is seen.
> GRANT SELECT ON TABLE utest.testtable TO ROLE rolename;
The response is:
InternalException: Error granting a privilege in Ranger. Ranger error message:
HTTP 403 Error: User doesn't have necessary permission to grant access
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4284) Additional logging messages to help with debugging when policy deltas are enabled
Abhay Kulkarni created RANGER-4284: -- Summary: Additional logging messages to help with debugging when policy deltas are enabled Key: RANGER-4284 URL: https://issues.apache.org/jira/browse/RANGER-4284 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni When policy deltas are enabled, it will help to debug any issues regarding computation and application of policy deltas. The Jira addresses this by printing the contents of Ranger policies in Ranger admin server and the policy-deltas downloaded to ranger plugin as well as contents of Ranger policies in the plugin generated by applying downloaded deltas to existing policies. This behavior is controlled by the following configuration parameters. Ranger-Admin: ranger.admin.policy.save.to.disk (default:false) : If true, prints policy-cache in a file with name suffixed by the policy-version in the directory specified by ranger.admin.policy.cache.dir configuration parameter. Plugin: ranger.plugin..preserve.deltas (default:false): If true, prints the downloaded policy-deltas in a file with name suffixed by the policy-version in a sub-directory named 'deltas' under the directory specified by 'ranger.plugin..policy.cache.dir' configuration parameter. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4291) If a ROW_FILTER type policy resources match, then an audit log record with Result=Denied i created
Abhay Kulkarni created RANGER-4291: -- Summary: If a ROW_FILTER type policy resources match, then an audit log record with Result=Denied i created Key: RANGER-4291 URL: https://issues.apache.org/jira/browse/RANGER-4291 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni ROW_FILTER (and DATA_MASKING) policies are supported for Hive service type. For a Hive resource being authorized, if a ROW_FILTER (or DATA_MASKING) policy's resource-specification are matched but none of the policy-items match, then effectively the policy did not match. However, if the policy has audit enabled, then an audit log record is created with Access Type=ROW_FILTER and Result=Denied. This is a spurious and misleading audit record, and it should not be generated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4291) If a ROW_FILTER type policy resources match, then an audit log record with Result=Denied is created
[ https://issues.apache.org/jira/browse/RANGER-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4291: --- Summary: If a ROW_FILTER type policy resources match, then an audit log record with Result=Denied is created (was: If a ROW_FILTER type policy resources match, then an audit log record with Result=Denied i created) > If a ROW_FILTER type policy resources match, then an audit log record with > Result=Denied is created > --- > > Key: RANGER-4291 > URL: https://issues.apache.org/jira/browse/RANGER-4291 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > ROW_FILTER (and DATA_MASKING) policies are supported for Hive service type. > For a Hive resource being authorized, if a ROW_FILTER (or DATA_MASKING) > policy's resource-specification are matched but none of the policy-items > match, then effectively the policy did not match. However, if the policy has > audit enabled, then an audit log record is created with Access > Type=ROW_FILTER and Result=Denied. > This is a spurious and misleading audit record, and it should not be > generated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4291) If a ROW_FILTER type policy resources match, then an audit log record with Result=Denied is created
[ https://issues.apache.org/jira/browse/RANGER-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4291: --- Description: ROW_FILTER policies are supported for Hive service type. For a Hive resource being authorized, if a ROW_FILTER policy's resource-specification are matched but none of the policy-items match, then effectively the policy did not match. However, if the policy has audit enabled, then an audit log record is created with Access Type=ROW_FILTER and Result=Denied. This is a spurious and misleading audit record, and it should not be generated. was: ROW_FILTER (and DATA_MASKING) policies are supported for Hive service type. For a Hive resource being authorized, if a ROW_FILTER (or DATA_MASKING) policy's resource-specification are matched but none of the policy-items match, then effectively the policy did not match. However, if the policy has audit enabled, then an audit log record is created with Access Type=ROW_FILTER and Result=Denied. This is a spurious and misleading audit record, and it should not be generated. > If a ROW_FILTER type policy resources match, then an audit log record with > Result=Denied is created > --- > > Key: RANGER-4291 > URL: https://issues.apache.org/jira/browse/RANGER-4291 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > ROW_FILTER policies are supported for Hive service type. For a Hive resource > being authorized, if a ROW_FILTER policy's resource-specification are > matched but none of the policy-items match, then effectively the policy did > not match. However, if the policy has audit enabled, then an audit log record > is created with Access Type=ROW_FILTER and Result=Denied. > This is a spurious and misleading audit record, and it should not be > generated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4378) Expand implied grants in the policy-items for being able to compare policy-cache dumps from server and client
Abhay Kulkarni created RANGER-4378: -- Summary: Expand implied grants in the policy-items for being able to compare policy-cache dumps from server and client Key: RANGER-4378 URL: https://issues.apache.org/jira/browse/RANGER-4378 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Expand implied grants in the policy-items for being able to compare policy-cache dumps from server and client. It is difficult to compare these dumps because implied-grants are processed are processed differently. Also, currently, the expansion of implied grants is done at the initialization of the policy-engine for each policy-item in every policy. This is expensive, as the computation of implied-grants depends on the service-definition and may be done only once during construction of a policy-engine. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4379) Assorted debugging help : save policy-cache at Ranger-admin and policy-cache as well as downloaded policy-deltas on plugin side
Abhay Kulkarni created RANGER-4379: -- Summary: Assorted debugging help : save policy-cache at Ranger-admin and policy-cache as well as downloaded policy-deltas on plugin side Key: RANGER-4379 URL: https://issues.apache.org/jira/browse/RANGER-4379 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Assorted debugging help : 1. Save policy-cache at Ranger-admin and policy-cache as well as downloaded policy-deltas on plugin side. Relevant configuration variables: Plugin: ranger.plugin..preserve.deltas [default:false] ranger.plugin..max.versions.to.preserve [default: 50] Ranger Admin: ranger.admin.policy.save.to.disk [default:false] ranger.admin.policy.max.versions.to.save.to.disk [default:1] 2. Better formatting of Trie dump -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4478) Incorrect trie updates when processing deltas
Abhay Kulkarni created RANGER-4478: -- Summary: Incorrect trie updates when processing deltas Key: RANGER-4478 URL: https://issues.apache.org/jira/browse/RANGER-4478 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni When policy-delta feature is enabled, updates to Ranger policies result in only the updates (deltas) being delivered to the plugin embedded in the service. In a long-running service and in the face of frequent ranger updates, sometimes, the internal structures of the policy-engine are not correctly updated. This may result in evaluation of incomplete set of policies when computing result of authorization request. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4478) Incorrect trie updates when processing deltas
[ https://issues.apache.org/jira/browse/RANGER-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4478: --- Description: When a plugin is configured to receive policy-deltas, policy engine processing of the deltas to update its resource tries can result some policies to be not included in the trie. (was: When policy-delta feature is enabled, updates to Ranger policies result in only the updates (deltas) being delivered to the plugin embedded in the service. In a long-running service and in the face of frequent ranger updates, sometimes, the internal structures of the policy-engine are not correctly updated. This may result in evaluation of incomplete set of policies when computing result of authorization request.) > Incorrect trie updates when processing deltas > - > > Key: RANGER-4478 > URL: https://issues.apache.org/jira/browse/RANGER-4478 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > When a plugin is configured to receive policy-deltas, policy engine > processing of the deltas to update its resource tries can result some > policies to be not included in the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4478) Incorrect trie updates when processing deltas
[ https://issues.apache.org/jira/browse/RANGER-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776393#comment-17776393 ] Abhay Kulkarni edited comment on RANGER-4478 at 10/23/23 8:24 PM: -- Patch is available here - [https://reviews.apache.org/r/74680/] Follow-up patch is available here -https://reviews.apache.org/r/74694/ was (Author: abhayk): Patch is available here - https://reviews.apache.org/r/74680/ > Incorrect trie updates when processing deltas > - > > Key: RANGER-4478 > URL: https://issues.apache.org/jira/browse/RANGER-4478 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > When a plugin is configured to receive policy-deltas, policy engine > processing of the deltas to update its resource tries can result some > policies to be not included in the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4478) Incorrect trie updates when processing deltas
[ https://issues.apache.org/jira/browse/RANGER-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776394#comment-17776394 ] Abhay Kulkarni edited comment on RANGER-4478 at 10/23/23 10:50 PM: --- Commit details: master: [https://github.com/apache/ranger/commit/0e7d63022252dc3c74478aa32bffac3ea755fee9] [https://github.com/apache/ranger/commit/36ce62eabbcc38112b15e376411fb053ef8d2ed9] was (Author: abhayk): Commit details: master: https://github.com/apache/ranger/commit/0e7d63022252dc3c74478aa32bffac3ea755fee9 > Incorrect trie updates when processing deltas > - > > Key: RANGER-4478 > URL: https://issues.apache.org/jira/browse/RANGER-4478 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > When a plugin is configured to receive policy-deltas, policy engine > processing of the deltas to update its resource tries can result some > policies to be not included in the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4478) Incorrect trie updates when processing deltas
[ https://issues.apache.org/jira/browse/RANGER-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776393#comment-17776393 ] Abhay Kulkarni edited comment on RANGER-4478 at 10/27/23 11:33 PM: --- Patch is available here - [https://reviews.apache.org/r/74680/] Follow-up patch is available here -[https://reviews.apache.org/r/74694/] Follow-up patch is available here - https://reviews.apache.org/r/74702/ was (Author: abhayk): Patch is available here - [https://reviews.apache.org/r/74680/] Follow-up patch is available here -https://reviews.apache.org/r/74694/ > Incorrect trie updates when processing deltas > - > > Key: RANGER-4478 > URL: https://issues.apache.org/jira/browse/RANGER-4478 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > When a plugin is configured to receive policy-deltas, policy engine > processing of the deltas to update its resource tries can result some > policies to be not included in the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-4478) Incorrect trie updates when processing deltas
[ https://issues.apache.org/jira/browse/RANGER-4478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776394#comment-17776394 ] Abhay Kulkarni edited comment on RANGER-4478 at 11/4/23 9:00 PM: - Commit details: master: [https://github.com/apache/ranger/commit/0e7d63022252dc3c74478aa32bffac3ea755fee9] [https://github.com/apache/ranger/commit/36ce62eabbcc38112b15e376411fb053ef8d2ed9] [https://github.com/apache/ranger/commit/3f78f9fae635b4dc1febdd1aad99e485cde412d6] was (Author: abhayk): Commit details: master: [https://github.com/apache/ranger/commit/0e7d63022252dc3c74478aa32bffac3ea755fee9] [https://github.com/apache/ranger/commit/36ce62eabbcc38112b15e376411fb053ef8d2ed9] > Incorrect trie updates when processing deltas > - > > Key: RANGER-4478 > URL: https://issues.apache.org/jira/browse/RANGER-4478 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > When a plugin is configured to receive policy-deltas, policy engine > processing of the deltas to update its resource tries can result some > policies to be not included in the trie. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4515) Enhance perf-tracer to get CPU time when possible
Abhay Kulkarni created RANGER-4515: -- Summary: Enhance perf-tracer to get CPU time when possible Key: RANGER-4515 URL: https://issues.apache.org/jira/browse/RANGER-4515 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Some JVM versions provide the precision of nanoseconds for CPU time as well as user time. Use nanosecond precision whenever available. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4565) Enhance Ranger's performance tracing module to optionally collect statistical information
Abhay Kulkarni created RANGER-4565: -- Summary: Enhance Ranger's performance tracing module to optionally collect statistical information Key: RANGER-4565 URL: https://issues.apache.org/jira/browse/RANGER-4565 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger's performance tracing module supports tracking the time spent in specified API(s), as specified in the log4j configuration file. Sometimes it is useful to collect the performance data when an API is called frequently, and instead of tracking performance of a single invocation of the API it is more interesting to see how API performed over a large number of invocations. This Jira addresses this requirement. The following configuration parameters are introduced to control the behavior of Ranger's performance tracking to enable collecting statistical data. ranger.perf.aggregate.data - If set to true, the performance data is recorded for specified APIs in aggregate, else data for each API call is printed (default:false) ranger.perf.aggregate.data.dump.interval - If set to value more than 0, aggregate performance data is printed with specified periodicity in seconds (default:0) ranger.perf.aggregate.data.lock.enabled - If set to true, aggregate performance data is updated using R-W lock, otherwise locking is not used (default:false). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4609) Support in File-based Tag Retriever to provide tag-deltas
Abhay Kulkarni created RANGER-4609:
--
Summary: Support in File-based Tag Retriever to provide tag-deltas
Key: RANGER-4609
URL: https://issues.apache.org/jira/browse/RANGER-4609
Project: Ranger
Issue Type: Improvement
Components: Ranger
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni
As a part of test infrastructure, a file-based tag retriever class is created.
When the TagEnricher is configured with this class, a plugin gets the tag from
the configured file.
Please refer to the Service-definition for tag for tagRetrieverClassName in
enricherOptions.
If the value of this option is set to
org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever
then, the tags will be fetched for a file named by the value of the enricher
option
"serviceTagsFileName".
{code:java}
"contextEnrichers":[
{
"itemId": 1,
"name" : "TagEnricher",
"enricher" : "org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
"enricherOptions" : {
"tagRetrieverClassName":
"org.apache.ranger.plugin.contextenricher.RangerAdminTagRetriever",
"tagRefresherPollingInterval": 6
}
}]{code}
This enhancement supports specifying another enricher option "tagFileCount".
When this is set to a value greater than 0, each successive call made to fetch
tags will provide tags from the file name obtained by adding a suffix [0..
tagFileCount-1].json to the file named by the option "serviceTagsFileName".
When tag-delta files are set up with appropriate suffixes, this simulates the
fetching of successive tag-deltas for processing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4609) Support in File-based Tag Retriever to provide tag-deltas
[
https://issues.apache.org/jira/browse/RANGER-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni updated RANGER-4609:
---
Description:
As a part of test infrastructure, a file-based tag retriever class is created.
When the TagEnricher is configured with this class, a plugin gets the tag from
the configured file.
Please refer to the Service-definition for tag for tagRetrieverClassName in
enricherOptions.
If the value of this option is set to
org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever
then, the tags will be fetched for a file named by the value of the enricher
option
"serviceTagsFileName".
{code:java}
"contextEnrichers":[
{
"itemId": 1,
"name" : "TagEnricher",
"enricher" : "org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
"enricherOptions" : {
"tagRetrieverClassName":
"org.apache.ranger.plugin.contextenricher.RangerAdminTagRetriever",
"tagRefresherPollingInterval": 6
}
}]{code}
This enhancement supports specifying another enricher option "tagFileCount".
When this is set to a value greater than 0, each successive call made to fetch
tags will provide tags from the file name obtained by adding a suffix [0..
tagFileCount-1].json to the file named by the option "serviceTagsFileName".
When tag-delta files are set up with appropriate suffixes, this simulates the
fetching of successive tag-deltas for processing.
An example may as follows:
{code:java}
"contextEnrichers": [
{
"itemId": 1,
"name": "TagEnricher",
"enricher":
"org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
"enricherOptions" :
{"tagRetrieverClassName":"org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever",
"tagRefresherPollingInterval":1000,
"serviceTagsFileName":"testdata/test_hive_tag.json", "deDupTags": "false",
"tagFileCount": "2"}
}
] {code}
where testdata directory contains the following files:
test_hive_tag.json :
{code:java}
{
"op": "add_or_update",
"serviceName": "test_hive",
"tagVersion": 5,
"tagDefinitions": {
"1": {
"name": "PII",
"source": "Atlas",
"attributeDefs": [],
"id": 1,
"isEnabled": true,
"version": 1
}
},
"tags": {
"1": {
"type": "PII",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": [],
"id": 1,
"isEnabled": true
}
},
"serviceResources": [
{
"serviceName": "test_hive",
"resourceElements": {
"database": {
"values": [
"default"
],
"isExcludes": false,
"isRecursive": false
},
"table": {
"values": [
"sample_07"
],
"isExcludes": false,
"isRecursive": false
}
},
"id": 1,
"isEnabled": true,
"version": 2
}
],
"resourceToTagIds": {
"1": [
1
]
},
"isDelta": false,
"tagsChangeExtent": "ALL",
"cachedTags": {}
} {code}
file test_hive_tag.json_0.json
{code:java}
{
"op": "add_or_update",
"serviceName": "test_hive",
"tagVersion": 4,
"tagDefinitions": {
"2": {
"id": 2,
"isEnabled": true,
"version": 1,
"name": "TEST_TAG",
"source": "Atlas",
"attributeDefs": []
}
},
"tags": {
"1": {
"id": 1,
"isEnabled": true,
"type": "PII",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": []
},
"2": {
"id": 2,
"isEnabled": true,
"type": "TEST_TAG",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": []
}
},
"serviceResources": [
{
"id": 1,
"isEnabled": true,
"version": 4,
"serviceName": "test_hive",
"resourceElements": {
"database": {
"values": [
"default"
],
"isExcludes": false,
"isRecursive": false
},
"table": {
"values": [
"sample_07"
],
"isExcludes": false,
"isRecursive": false
}
}
}
],
"resourceToTagIds": {
"1": [
1,
2
]
},
"isDelta": true,
"tagsCh
[jira] [Created] (RANGER-4639) Provide an option to bypass evaluation of chained plugin if the parent plugin has applicable policy
Abhay Kulkarni created RANGER-4639: -- Summary: Provide an option to bypass evaluation of chained plugin if the parent plugin has applicable policy Key: RANGER-4639 URL: https://issues.apache.org/jira/browse/RANGER-4639 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni When a chained plugin is configured, for every access request processed by the parent plugin is also processed by chained plugin. It may be appropriate, in some cases, under a configuration option, to bypass the evaluation of chained plugin when an applicable policy is found by the parent plugin. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4639) Provide an option to bypass evaluation of chained plugin if the parent plugin has applicable policy
[ https://issues.apache.org/jira/browse/RANGER-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4639: --- Description: When a chained plugin is configured, for every access request processed by the parent plugin is also processed by chained plugin. It may be appropriate, in some cases, under a configuration option, to bypass the evaluation of chained plugin when an applicable policy is found by the parent plugin. This is controlled by a configuration parameter: ranger.plugin..bypass.chained.plugin.evaluation.if.access.is.determined with a default value of false. If it is set to true, then the chanined plugin evaluation will be bypassed - if the base-plugin found applicable policy. was:When a chained plugin is configured, for every access request processed by the parent plugin is also processed by chained plugin. It may be appropriate, in some cases, under a configuration option, to bypass the evaluation of chained plugin when an applicable policy is found by the parent plugin. > Provide an option to bypass evaluation of chained plugin if the parent plugin > has applicable policy > --- > > Key: RANGER-4639 > URL: https://issues.apache.org/jira/browse/RANGER-4639 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > When a chained plugin is configured, for every access request processed by > the parent plugin is also processed by chained plugin. It may be appropriate, > in some cases, under a configuration option, to bypass the evaluation of > chained plugin when an applicable policy is found by the parent plugin. > > This is controlled by a configuration parameter: > ranger.plugin..bypass.chained.plugin.evaluation.if.access.is.determined > with a default value of false. If it is set to true, then the chanined plugin > evaluation will be bypassed - if the base-plugin found applicable policy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4655) Execute and read permissions granted to a user in different HDFS policies does not take effect.
Abhay Kulkarni created RANGER-4655: -- Summary: Execute and read permissions granted to a user in different HDFS policies does not take effect. Key: RANGER-4655 URL: https://issues.apache.org/jira/browse/RANGER-4655 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni This bug is shown up when multiple accesses are requested using one access-request and all of the requested accesses need to be granted in order the access-request to be allowed. Test Case: Policy 1: Granted the "public" group "execute" permission to "/" HDFS policy recursively. Policy 2: Granted only the "read" permission to user for "/hdp" Doing a list on "/hdp" fails with permission denied for access READ_EXECUTE. However, the same works when "execute" permission is granted in Policy 2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-4655) Execute and read permissions granted to a user in different HDFS policies does not take effect.
[ https://issues.apache.org/jira/browse/RANGER-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-4655: --- Description: This bug is shown up when multiple accesses are requested using one access-request and all of the requested accesses need to be granted in order the access-request to be allowed. This appears to be regression introduced by RANGER-3999 Test Case: Policy 1: Granted the "public" group "execute" permission to "/" HDFS policy recursively. Policy 2: Granted only the "read" permission to user for "/hdp" Doing a list on "/hdp" fails with permission denied for access READ_EXECUTE. However, the same works when "execute" permission is granted in Policy 2. was: This bug is shown up when multiple accesses are requested using one access-request and all of the requested accesses need to be granted in order the access-request to be allowed. Test Case: Policy 1: Granted the "public" group "execute" permission to "/" HDFS policy recursively. Policy 2: Granted only the "read" permission to user for "/hdp" Doing a list on "/hdp" fails with permission denied for access READ_EXECUTE. However, the same works when "execute" permission is granted in Policy 2. > Execute and read permissions granted to a user in different HDFS policies > does not take effect. > > > Key: RANGER-4655 > URL: https://issues.apache.org/jira/browse/RANGER-4655 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > This bug is shown up when multiple accesses are requested using one > access-request and all of the requested accesses need to be granted in order > the access-request to be allowed. > > This appears to be regression introduced by RANGER-3999 > Test Case: > Policy 1: Granted the "public" group "execute" permission to "/" HDFS policy > recursively. > Policy 2: Granted only the "read" permission to user for "/hdp" > Doing a list on "/hdp" fails with permission denied for access READ_EXECUTE. > However, the same works when "execute" permission is granted in Policy 2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4722) HDFS authorization logic for directory hierarchy rooted at "/" is incorrect
Abhay Kulkarni created RANGER-4722: -- Summary: HDFS authorization logic for directory hierarchy rooted at "/" is incorrect Key: RANGER-4722 URL: https://issues.apache.org/jira/browse/RANGER-4722 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger authorization logic for the HDFS commands that require authorization of the entire directory hierarchy rooted at the specified directory argument is incorrect as it does not correctly compute the sub-directory paths. The paths of sub-directories that need to be authorized incorrectly contain an extra '/' character, which leads to incorrect authorization result. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-4745) Enhance handling of subAccess authorization in Ranger HDFS plugin
[
https://issues.apache.org/jira/browse/RANGER-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni reassigned RANGER-4745:
--
Assignee: Abhay Kulkarni
> Enhance handling of subAccess authorization in Ranger HDFS plugin
> -
>
> Key: RANGER-4745
> URL: https://issues.apache.org/jira/browse/RANGER-4745
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
>
> Currently Ranger performs authorization of the HDFS commands which require
> access to the hierarchy of files/directory rooted at the argument passed to
> the HDFS command as described below. Some examples of such commands are :
>
>
> {quote}hdfs dfs -count -q -h -v ; hdfs dfs -R
> {quote}
> HDFS Authorization Interface
> When these commands are invoked, HDFS Namenode builds a tree of i-nodes
> corresponding to , and passes it to the authorizer with a flag
> indicating that subAccess (access to the directory hierarchy rooted at
> ) is to be checked.
> Ranger implementation
> For each directory in the hierarchy rooted at , Ranger code checks
> if the requested permissions (typically read and execute) are allowed using
> only Ranger policies. If any directory in the top-down path starting from
> does not allow access, then the authorization steps done until
> then are discarded, and the HDFS default authorizer is called upon to check
> the access with the same arguments. The default authorizer only checks the
> HDFS ACLs (and not any Ranger policies) on each directory in the hierarchy to
> determine the access.
> Design of new Ranger implementation
> For each directory in the hierarchy rooted at , new Ranger design
> 1. Checks if the requested permissions are allowed using only Ranger policies
> 2. If the access is denied, the authorization steps done until this point are
> discarded, and the HDFS default authorizer is called upon to check the access
> with the original set of argument, and the result of default authorizer is
> returned to Namenode.
> 3. Otherwise, if the access is not determined, a new set of arguments are
> constructed for the directory being processed and HDFS default authorizer is
> called to check the access with the modified set of arguments.
> 4. If the default authorizer does not allow the access, then the result is
> returned to Namenode.
> 5. Otherwise, the processing continues with the next directory.
> Performance considerations
> The new implementation may have some impact on the performance. A few cases
> are as follows.
> 1. There is a Ranger policy that allows requested permissions recursively to
> some directory in the hierarchy. Depending on how deep this directory is in
> the hierarchy, the number of directories for which the access evaluation is
> requested will change. Higher this directory in the hierarchy, lesser the
> number of evaluations. In the existing implementation, a short-circuiting of
> calls for evaluating Ranger policies will, in general, happen earlier, and
> the default authorizer will be called upon the handle the authorization.
> 2. In the worst case, if there is no Ranger policy for any directory in the
> hierarchy, then each directory in the hierachy there will be a target of
> access evaluation by Ranger and by the default authorizer (if the HDFS ACLs
> for each directory allow requested accesses).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4745) Enhance handling of subAccess authorization in Ranger HDFS plugin
Abhay Kulkarni created RANGER-4745:
--
Summary: Enhance handling of subAccess authorization in Ranger
HDFS plugin
Key: RANGER-4745
URL: https://issues.apache.org/jira/browse/RANGER-4745
Project: Ranger
Issue Type: Improvement
Components: Ranger
Reporter: Abhay Kulkarni
Currently Ranger performs authorization of the HDFS commands which require
access to the hierarchy of files/directory rooted at the argument passed to the
HDFS command as described below. Some examples of such commands are :
{quote}hdfs dfs -count -q -h -v ; hdfs dfs -R
{quote}
HDFS Authorization Interface
When these commands are invoked, HDFS Namenode builds a tree of i-nodes
corresponding to , and passes it to the authorizer with a flag
indicating that subAccess (access to the directory hierarchy rooted at
) is to be checked.
Ranger implementation
For each directory in the hierarchy rooted at , Ranger code checks
if the requested permissions (typically read and execute) are allowed using
only Ranger policies. If any directory in the top-down path starting from
does not allow access, then the authorization steps done until then
are discarded, and the HDFS default authorizer is called upon to check the
access with the same arguments. The default authorizer only checks the HDFS
ACLs (and not any Ranger policies) on each directory in the hierarchy to
determine the access.
Design of new Ranger implementation
For each directory in the hierarchy rooted at , new Ranger design
1. Checks if the requested permissions are allowed using only Ranger policies
2. If the access is denied, the authorization steps done until this point are
discarded, and the HDFS default authorizer is called upon to check the access
with the original set of argument, and the result of default authorizer is
returned to Namenode.
3. Otherwise, if the access is not determined, a new set of arguments are
constructed for the directory being processed and HDFS default authorizer is
called to check the access with the modified set of arguments.
4. If the default authorizer does not allow the access, then the result is
returned to Namenode.
5. Otherwise, the processing continues with the next directory.
Performance considerations
The new implementation may have some impact on the performance. A few cases are
as follows.
1. There is a Ranger policy that allows requested permissions recursively to
some directory in the hierarchy. Depending on how deep this directory is in the
hierarchy, the number of directories for which the access evaluation is
requested will change. Higher this directory in the hierarchy, lesser the
number of evaluations. In the existing implementation, a short-circuiting of
calls for evaluating Ranger policies will, in general, happen earlier, and the
default authorizer will be called upon the handle the authorization.
2. In the worst case, if there is no Ranger policy for any directory in the
hierarchy, then each directory in the hierachy there will be a target of access
evaluation by Ranger and by the default authorizer (if the HDFS ACLs for each
directory allow requested accesses).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4767) Deleted policies are still taking effect if all policies in a repo for a security zone are deleted
[
https://issues.apache.org/jira/browse/RANGER-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni updated RANGER-4767:
---
Summary: Deleted policies are still taking effect if all policies in a repo
for a security zone are deleted (was: Deleted policies are still taking effect
if all policies in a repo are deleted)
> Deleted policies are still taking effect if all policies in a repo for a
> security zone are deleted
> --
>
> Key: RANGER-4767
> URL: https://issues.apache.org/jira/browse/RANGER-4767
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Reporter: Abhishek
>Assignee: Abhay Kulkarni
>Priority: Major
>
> If all the policies in a repo are deleted, then still the previous policies
> are taking effect.
> If there are no policies in the repo, then the following error is seen in the
> logs
> while syncing the policies, and the previously existing policies are still
> taking effect and operations are allowed through those policies
> {code:java|bgColor=#f4f5f7}
> 2024-04-02T16:09:42.913Z ERROR
> PolicyRefresher(serviceName=cm_trino)-233
> org.apache.ranger.plugin.service.RangerBasePlugin setPolicies: policy
> engine initialization failed! Leaving current policy engine as-is. Exception
> :
> java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()"
> because "this.policies" is null
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:887)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.(RangerPolicyRepository.java:229)
> at
> org.apache.ranger.plugin.policyengine.PolicyEngine.(PolicyEngine.java:264)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.(RangerPolicyEngineImpl.java:104)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:363)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:264)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:210)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4767) Deleted policies are still taking effect if all policies for a security zone are deleted
[
https://issues.apache.org/jira/browse/RANGER-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni updated RANGER-4767:
---
Summary: Deleted policies are still taking effect if all policies for a
security zone are deleted (was: Deleted policies are still taking effect if
all policies in a repo for a security zone are deleted)
> Deleted policies are still taking effect if all policies for a security zone
> are deleted
>
>
> Key: RANGER-4767
> URL: https://issues.apache.org/jira/browse/RANGER-4767
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Reporter: Abhishek
>Assignee: Abhay Kulkarni
>Priority: Major
>
> If all the policies in a repo are deleted, then still the previous policies
> are taking effect.
> If there are no policies in the repo, then the following error is seen in the
> logs
> while syncing the policies, and the previously existing policies are still
> taking effect and operations are allowed through those policies
> {code:java|bgColor=#f4f5f7}
> 2024-04-02T16:09:42.913Z ERROR
> PolicyRefresher(serviceName=cm_trino)-233
> org.apache.ranger.plugin.service.RangerBasePlugin setPolicies: policy
> engine initialization failed! Leaving current policy engine as-is. Exception
> :
> java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()"
> because "this.policies" is null
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:887)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.(RangerPolicyRepository.java:229)
> at
> org.apache.ranger.plugin.policyengine.PolicyEngine.(PolicyEngine.java:264)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.(RangerPolicyEngineImpl.java:104)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:363)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:264)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:210)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (RANGER-4767) Deleted policies are still taking effect if all policies for a security zone are deleted
[
https://issues.apache.org/jira/browse/RANGER-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni updated RANGER-4767:
---
Description:
If all the policies for a security zone are deleted, then still the previous
policies are taking effect.
If there are no policies in the repo, then the following error is seen in the
logs
while syncing the policies, and the previously existing policies are still
taking effect and operations are allowed through those policies
{code:java|bgColor=#f4f5f7}
2024-04-02T16:09:42.913ZERROR
PolicyRefresher(serviceName=cm_trino)-233
org.apache.ranger.plugin.service.RangerBasePlugin setPolicies: policy
engine initialization failed! Leaving current policy engine as-is. Exception :
java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()"
because "this.policies" is null
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:887)
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.(RangerPolicyRepository.java:229)
at
org.apache.ranger.plugin.policyengine.PolicyEngine.(PolicyEngine.java:264)
at
org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.(RangerPolicyEngineImpl.java:104)
at
org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:363)
at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:264)
at
org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:210)
{code}
was:
If all the policies in a repo are deleted, then still the previous policies are
taking effect.
If there are no policies in the repo, then the following error is seen in the
logs
while syncing the policies, and the previously existing policies are still
taking effect and operations are allowed through those policies
{code:java|bgColor=#f4f5f7}
2024-04-02T16:09:42.913ZERROR
PolicyRefresher(serviceName=cm_trino)-233
org.apache.ranger.plugin.service.RangerBasePlugin setPolicies: policy
engine initialization failed! Leaving current policy engine as-is. Exception :
java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()"
because "this.policies" is null
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:887)
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.(RangerPolicyRepository.java:229)
at
org.apache.ranger.plugin.policyengine.PolicyEngine.(PolicyEngine.java:264)
at
org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.(RangerPolicyEngineImpl.java:104)
at
org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:363)
at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:264)
at
org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:210)
{code}
> Deleted policies are still taking effect if all policies for a security zone
> are deleted
>
>
> Key: RANGER-4767
> URL: https://issues.apache.org/jira/browse/RANGER-4767
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Reporter: Abhishek
>Assignee: Abhay Kulkarni
>Priority: Major
>
> If all the policies for a security zone are deleted, then still the previous
> policies are taking effect.
> If there are no policies in the repo, then the following error is seen in the
> logs
> while syncing the policies, and the previously existing policies are still
> taking effect and operations are allowed through those policies
> {code:java|bgColor=#f4f5f7}
> 2024-04-02T16:09:42.913Z ERROR
> PolicyRefresher(serviceName=cm_trino)-233
> org.apache.ranger.plugin.service.RangerBasePlugin setPolicies: policy
> engine initialization failed! Leaving current policy engine as-is. Exception
> :
> java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()"
> because "this.policies" is null
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:887)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.(RangerPolicyRepository.java:229)
> at
> org.apache.ranger.plugin.policyengine.PolicyEngine.(PolicyEngine.java:264)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.(RangerPolicyEngineImpl.java:104)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:363)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:264)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:210)
> {code}
[jira] [Assigned] (RANGER-4786) Ranger override policy is not working
[
https://issues.apache.org/jira/browse/RANGER-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni reassigned RANGER-4786:
--
Assignee: Abhay Kulkarni
> Ranger override policy is not working
> -
>
> Key: RANGER-4786
> URL: https://issues.apache.org/jira/browse/RANGER-4786
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Reporter: Abhay
>Assignee: Abhay Kulkarni
>Priority: Major
>
> STEPS TO REPRODUCE:
> Update default “all - path” policy in cm_hdfs service. Add a deny condition
> for \{USER} to read,write,execute permissions. With this policy all users
> will be denied to access hdfs locations.
> Now, add a policy for user "u1" to get "RWX" permissions on path "/testpath"
> recursively. Enable *ovverride* for this policy.
> Perform kinit by user u1 and dp hdfs operations.
> {noformat}
> hdfs dfs -ls /testpath
> {noformat}
> The access is denied by default "all - path" policy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4817) Optimize Ranger HDFS Authorization by combining multiple authorization calls
Abhay Kulkarni created RANGER-4817:
--
Summary: Optimize Ranger HDFS Authorization by combining multiple
authorization calls
Key: RANGER-4817
URL: https://issues.apache.org/jira/browse/RANGER-4817
Project: Ranger
Issue Type: Improvement
Components: Ranger
Reporter: Abhay Kulkarni
The focus of optimizations described below is to minimize the number of times
the Ranger policy-engine is called to authorize a NameNode RPC without
modifying the Namenode authorization interface or authorization call sequence.
This optimization is possible as the Namenode calls the authorizer more than
once to authorize some RPCs, as observed during the testing. To ensure that the
authorizer is provided a consistent context to represent a RPC, some
improvements are needed in the Namenode. Related Namenode JIRAs are
{*}HDFS-17478{*}: Avoid creation of AccessControlEnforcer object for every call
to the authorizer, and
{*}HDFS-17500{*}: Provide operation name consistently in the caller-context
provided to checkPermissionWithContext() API.
Ranger authorizer is updated to leverage this context to optimize authorization
calls for the RPC. In particular, the following RPC operations' authorization
logic is updated.
List of operations with optimized authorization checks.
# Create file: operation name “create”
# Rename file: operation name “rename”
# Delete file: operation name “delete”
# Create directory: operation name “mkdirs”
# List directory contents: operation name “listStatus”
# Rename directory: operation name “rename”
# Delete directory: operation name “delete”
# Get Encryption Zone for a directory: operation name “getEZForPath”
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (RANGER-4817) Optimize Ranger HDFS Authorization by combining multiple authorization calls
[
https://issues.apache.org/jira/browse/RANGER-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni reassigned RANGER-4817:
--
Assignee: Abhay Kulkarni
> Optimize Ranger HDFS Authorization by combining multiple authorization calls
>
>
> Key: RANGER-4817
> URL: https://issues.apache.org/jira/browse/RANGER-4817
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
>
> The focus of optimizations described below is to minimize the number of times
> the Ranger policy-engine is called to authorize a NameNode RPC without
> modifying the Namenode authorization interface or authorization call sequence.
> This optimization is possible as the Namenode calls the authorizer more than
> once to authorize some RPCs, as observed during the testing. To ensure that
> the authorizer is provided a consistent context to represent a RPC, some
> improvements are needed in the Namenode. Related Namenode JIRAs are
> {*}HDFS-17478{*}: Avoid creation of AccessControlEnforcer object for every
> call to the authorizer, and
> {*}HDFS-17500{*}: Provide operation name consistently in the caller-context
> provided to checkPermissionWithContext() API.
> Ranger authorizer is updated to leverage this context to optimize
> authorization calls for the RPC. In particular, the following RPC operations'
> authorization logic is updated.
>
> List of operations with optimized authorization checks.
> # Create file: operation name “create”
> # Rename file: operation name “rename”
> # Delete file: operation name “delete”
> # Create directory: operation name “mkdirs”
> # List directory contents: operation name “listStatus”
> # Rename directory: operation name “rename”
> # Delete directory: operation name “delete”
> # Get Encryption Zone for a directory: operation name “getEZForPath”
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (RANGER-4820) Support authorization of multiple accesses grouped by access groups in one policy engine call
Abhay Kulkarni created RANGER-4820: -- Summary: Support authorization of multiple accesses grouped by access groups in one policy engine call Key: RANGER-4820 URL: https://issues.apache.org/jira/browse/RANGER-4820 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Currently, Ranger policy engine supports authorization of multiple accesses for a given resource in a single call to the Ranger plugin's isAccessAllowed() API. However, it has some limitations which are addressed by this JIRA. Limitation: If multiple accesses are to be authorized, then the current authorization logic in Ranger policy engine is designed to allow the request to succeed (that is, grant access) only if all requested accesses are granted. This Jira supports organizing accesses in groups where each group is granted access if any access in the group is allowed, and the request is successful (that is, user is allowed access) only if all groups are granted access. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-4820) Support authorization of multiple accesses grouped by access groups in one policy engine call
[ https://issues.apache.org/jira/browse/RANGER-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reassigned RANGER-4820: -- Assignee: Abhay Kulkarni > Support authorization of multiple accesses grouped by access groups in one > policy engine call > - > > Key: RANGER-4820 > URL: https://issues.apache.org/jira/browse/RANGER-4820 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Currently, Ranger policy engine supports authorization of multiple accesses > for a given resource in a single call to the Ranger plugin's > isAccessAllowed() API. However, it has some limitations which are addressed > by this JIRA. > Limitation: If multiple accesses are to be authorized, then the current > authorization logic in Ranger policy engine is designed to allow the request > to succeed (that is, grant access) only if all requested accesses are granted. > This Jira supports organizing accesses in groups where each group is granted > access if any access in the group is allowed, and the request is successful > (that is, user is allowed access) only if all groups are granted access. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4823) Incorrect processing of downloaded policies in plugin when policy-deltas are enabled
Abhay Kulkarni created RANGER-4823: -- Summary: Incorrect processing of downloaded policies in plugin when policy-deltas are enabled Key: RANGER-4823 URL: https://issues.apache.org/jira/browse/RANGER-4823 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni When policy deltas are enabled, and there is no material change in policy-set after previous policy download processed by Ranger admin, the ServicePolicies object downloaded contains null value instead of an empty list for policyDeltas attribute because of change made to the annotations by RANGER-3948. As the plugin considers empty-list value differently than null value, the policy-engine built by the plugin does not correctly reflect the existing policy-set, and leads to incorrect authorization result. A material change to policy-set indicates that there is at least one policy added/deleted/updated to previous policy-set. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4824) Remove ACL-based policy engine unit test code
Abhay Kulkarni created RANGER-4824: -- Summary: Remove ACL-based policy engine unit test code Key: RANGER-4824 URL: https://issues.apache.org/jira/browse/RANGER-4824 Project: Ranger Issue Type: Task Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger unit test framework for policy engine functional tests uses two different types of policy-engines for executing unit tests; one is base on interpreted model that is used in production and another is based on the ACLs computed from the set of Ranger policies. As the latter is not used anywhere other than during unit tests, for the brevity of the code-base, it need not be used for unit testing and needs be removed from the code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-4852) De-duplicated tags do not work correctly when tag-deltas are enabled.
Abhay Kulkarni created RANGER-4852: -- Summary: De-duplicated tags do not work correctly when tag-deltas are enabled. Key: RANGER-4852 URL: https://issues.apache.org/jira/browse/RANGER-4852 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni When tags are de-duplicated and tag deltas are enabled, multiple tagged entities point to the same base tag object. Removing tag association from any of the tagged entities may cause all entities to lose their associations with the base tag. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-3093) Add required libraries to ranger-tools to ensure that perftester scripts run correctly
Abhay Kulkarni created RANGER-3093: -- Summary: Add required libraries to ranger-tools to ensure that perftester scripts run correctly Key: RANGER-3093 URL: https://issues.apache.org/jira/browse/RANGER-3093 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni ranger-tools project does not include all needed libraries required for running performance testing tool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3098) Updates to validity period of tag are not reflected in Ranger database
Abhay Kulkarni created RANGER-3098: -- Summary: Updates to validity period of tag are not reflected in Ranger database Key: RANGER-3098 URL: https://issues.apache.org/jira/browse/RANGER-3098 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni If validity period of a tag is updated upstream, then the Ranger database is not getting updated accordingly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3106) Add unit test cases for TrieNode.undoSetup()
Abhay Kulkarni created RANGER-3106: -- Summary: Add unit test cases for TrieNode.undoSetup() Key: RANGER-3106 URL: https://issues.apache.org/jira/browse/RANGER-3106 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni TrieNode.undoSetup() is internally called to clean up a node in the Trie structure. This API is invoked only when incremental updates are applied to an existing Trie structure. This Jira addresses a small improvement to implementation of this API and corresponding unit test case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3107) Optimize memory requirements for storing Tag/Policy/Zone trie - Trade-off processing time for memory
Abhay Kulkarni created RANGER-3107: -- Summary: Optimize memory requirements for storing Tag/Policy/Zone trie - Trade-off processing time for memory Key: RANGER-3107 URL: https://issues.apache.org/jira/browse/RANGER-3107 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Trie structure is devised to index tagged entities and policies for a fast look-up. However, it may be take a lot of memory. This is a result of a design decision to optimize processing (mainly searching) time at the cost of extra memory. It is desirable to find a more optimal trade-off between memory needs and processing requirements, especially for supporting a large number or policies and tagged entities. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3122) Support delegate-admin for specific permissions
Abhay Kulkarni created RANGER-3122: -- Summary: Support delegate-admin for specific permissions Key: RANGER-3122 URL: https://issues.apache.org/jira/browse/RANGER-3122 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Currently delegate-admin cannot be marked for specific permissions. It is all-or-nothing for the permissions defined in resource policy. Ranger should have ability for granting delegate-admin for specific permissions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3131) Remove some warnings in Maven build output
Abhay Kulkarni created RANGER-3131: -- Summary: Remove some warnings in Maven build output Key: RANGER-3131 URL: https://issues.apache.org/jira/browse/RANGER-3131 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Maven warnings: [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.ranger:ranger-intg:jar:3.0.0-SNAPSHOT [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.codehaus.mojo:exec-maven-plugin @ line 94, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3122) Support delegate-admin for specific permissions
[ https://issues.apache.org/jira/browse/RANGER-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250073#comment-17250073 ] Abhay Kulkarni edited comment on RANGER-3122 at 1/6/21, 10:44 PM: -- Patch is available at the review board: [https://reviews.apache.org/r/73094/] Additional patch to ensure that datamask and rowfilter policies are correctly filtered for delegated-admin feature. https://reviews.apache.org/r/73120/ was (Author: abhayk): Patch is available at the review board: https://reviews.apache.org/r/73094/ > Support delegate-admin for specific permissions > --- > > Key: RANGER-3122 > URL: https://issues.apache.org/jira/browse/RANGER-3122 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Currently delegate-admin cannot be marked for specific permissions. It is > all-or-nothing for the permissions defined in resource policy. Ranger should > have ability for granting delegate-admin for specific permissions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3122) Support delegate-admin for specific permissions
[ https://issues.apache.org/jira/browse/RANGER-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254241#comment-17254241 ] Abhay Kulkarni edited comment on RANGER-3122 at 1/7/21, 12:01 AM: -- Commit details: master: [https://github.com/apache/ranger/commit/9d459e0255484277d802b5d8e9fcdd936f3aed04] ranger-2.2: [https://github.com/apache/ranger/commit/f277f4f1dcc91bdd0487fdce046bb5d08fee34b5] Part 2: master: [https://github.com/apache/ranger/commit/4deb34cb8e189eaa466eec49066338339ed2f88c] ranger-2.2: [https://github.com/apache/ranger/commit/1a0fb59ce12064b60d98e234103aa8308639995f] was (Author: abhayk): Commit details: master: [https://github.com/apache/ranger/commit/9d459e0255484277d802b5d8e9fcdd936f3aed04] ranger-2.2: [https://github.com/apache/ranger/commit/f277f4f1dcc91bdd0487fdce046bb5d08fee34b5] > Support delegate-admin for specific permissions > --- > > Key: RANGER-3122 > URL: https://issues.apache.org/jira/browse/RANGER-3122 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Currently delegate-admin cannot be marked for specific permissions. It is > all-or-nothing for the permissions defined in resource policy. Ranger should > have ability for granting delegate-admin for specific permissions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3122) Support delegate-admin for specific permissions
[ https://issues.apache.org/jira/browse/RANGER-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250073#comment-17250073 ] Abhay Kulkarni edited comment on RANGER-3122 at 1/8/21, 8:46 PM: - Patch is available at the review board: [https://reviews.apache.org/r/73094/] Additional patch to ensure that datamask and rowfilter policies are correctly filtered for delegated-admin feature. [https://reviews.apache.org/r/73120/] Additional patch to ensure that if policies are requested by service-admin user, then the returned list contains unique set of policies. https://reviews.apache.org/r/73125/ was (Author: abhayk): Patch is available at the review board: [https://reviews.apache.org/r/73094/] Additional patch to ensure that datamask and rowfilter policies are correctly filtered for delegated-admin feature. https://reviews.apache.org/r/73120/ > Support delegate-admin for specific permissions > --- > > Key: RANGER-3122 > URL: https://issues.apache.org/jira/browse/RANGER-3122 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Currently delegate-admin cannot be marked for specific permissions. It is > all-or-nothing for the permissions defined in resource policy. Ranger should > have ability for granting delegate-admin for specific permissions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3122) Support delegate-admin for specific permissions
[ https://issues.apache.org/jira/browse/RANGER-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254241#comment-17254241 ] Abhay Kulkarni edited comment on RANGER-3122 at 1/8/21, 9:13 PM: - Commit details: master: [https://github.com/apache/ranger/commit/9d459e0255484277d802b5d8e9fcdd936f3aed04] ranger-2.2: [https://github.com/apache/ranger/commit/f277f4f1dcc91bdd0487fdce046bb5d08fee34b5] Part 2: master: [https://github.com/apache/ranger/commit/4deb34cb8e189eaa466eec49066338339ed2f88c] ranger-2.2: [https://github.com/apache/ranger/commit/1a0fb59ce12064b60d98e234103aa8308639995f] Part 3: master: [https://github.com/apache/ranger/commit/d11233ad41311d6fec9e66c4ee91984d7cc216a6] ranger-2.2: [https://github.com/apache/ranger/commit/7e987c2865c501f77bb349c3eb8d5dc2b37a057a] was (Author: abhayk): Commit details: master: [https://github.com/apache/ranger/commit/9d459e0255484277d802b5d8e9fcdd936f3aed04] ranger-2.2: [https://github.com/apache/ranger/commit/f277f4f1dcc91bdd0487fdce046bb5d08fee34b5] Part 2: master: [https://github.com/apache/ranger/commit/4deb34cb8e189eaa466eec49066338339ed2f88c] ranger-2.2: [https://github.com/apache/ranger/commit/1a0fb59ce12064b60d98e234103aa8308639995f] > Support delegate-admin for specific permissions > --- > > Key: RANGER-3122 > URL: https://issues.apache.org/jira/browse/RANGER-3122 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Currently delegate-admin cannot be marked for specific permissions. It is > all-or-nothing for the permissions defined in resource policy. Ranger should > have ability for granting delegate-admin for specific permissions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3147) enhance resource-trie to enable finding evaluators for a given resource and its children
Abhay Kulkarni created RANGER-3147: -- Summary: enhance resource-trie to enable finding evaluators for a given resource and its children Key: RANGER-3147 URL: https://issues.apache.org/jira/browse/RANGER-3147 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni For the resource modeled as a path (with a path separator configured in its definition) it may be desired to search its Trie data to find an exact match for the resource being searched and its children. Ranger access requests need to support an additional scope specification (such as SELF_OR_CHILD) to implement this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3156) RangerResouceTrie.add() and RangerResourceTrie.delete() do not work correctly for the resources containing wildcards
Abhay Kulkarni created RANGER-3156: -- Summary: RangerResouceTrie.add() and RangerResourceTrie.delete() do not work correctly for the resources containing wildcards Key: RANGER-3156 URL: https://issues.apache.org/jira/browse/RANGER-3156 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni RangerResourceTrie class supports add() and delete() APIs to incrementally update the Trie data structure with a single evaluator. When the evaluator's resource contains wildcards, both APIs leave the Trie tree in an inconsistent state. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2724) Support EXECUTE permission in HBase Authorisation
[
https://issues.apache.org/jira/browse/RANGER-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273870#comment-17273870
]
Abhay Kulkarni commented on RANGER-2724:
Additional review request:
[https://reviews.apache.org/r/73162/]
Additional commits:
master:
[https://github.com/apache/ranger/commit/47cfd468570600723543923336ab3690275ab1c3]
ranger-2.2:
[https://github.com/apache/ranger/commit/8a542c14a4991d06b61a514233726c81d62662b6]
> Support EXECUTE permission in HBase Authorisation
> -
>
> Key: RANGER-2724
> URL: https://issues.apache.org/jira/browse/RANGER-2724
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 2.1.0
>
> Attachments:
> 0001-RANGER-2724-Support-EXECUTE-permission-in-HBase-Auth.patch,
> 0001-RANGER-2724-Support-EXECUTE-permission-in-HBase-Auth.patch,
> RANGER-2724.patch, Screen Shot 2020-03-18 at 12.58.42 PM.png, Screen Shot
> 2020-03-19 at 4.00.22 PM.png
>
>
> HBase Authorisation supports execute permission along with read,write,create
> and admin.
> https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_sg_hbase_authorization.html#
> # Read (R) - can read data at the given scope
> # Write (W) - can write data at the given scope
> # Execute (X) - can execute coprocessor endpoints at the given scope
> # Create (C) - can create tables or drop tables (even those they did not
> create) at the given scope
> # Admin (A) - can perform cluster operations such as balancing the cluster or
> assigning regions at the given scope
> In the HBase we can define the endpoint implementations to extend the HBase
> functionality without touching the core. These endpoints can be called in
> table scope.
> Example: Endpoint section in
> https://blogs.apache.org/hbase/entry/coprocessor_introduction
> To run the endpoint implementations users can be authorised with execute
> permissions. There are hooks also coprocessor hooks also supported for this
> endpoint invocations.
> {noformat}
> public Message
> preEndpointInvocation(ObserverContext ctx,
> Service service, String methodName, Message request) throws IOException {
> if (this.shouldCheckExecPermission && !(service instanceof
> AccessControlService)) {
> this.requirePermission(ctx, "invoke(" +
> service.getDescriptorForType().getName() + "." + methodName + ")",
> this.getTableName((RegionCoprocessorEnvironment)ctx.getEnvironment()),
> (byte[])null, (byte[])null, Action.EXEC);
> }
> return request;
> }
> public void
> postEndpointInvocation(ObserverContext ctx,
> Service service, String methodName, Message request, Builder responseBuilder)
> throws IOException {
> }
> {noformat}
> Any way this execute permission is optional but better to have in the Ranger
> mainly in case of Phoenix we do use most of the coprocessor endpoints to
> tackle with meta data for tables/resources to be written to system tables. So
> It would be helpful to have it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (RANGER-3178) FIx spurious full policy/tag downloads when incremental policy/tag updates are enabled
Abhay Kulkarni created RANGER-3178: -- Summary: FIx spurious full policy/tag downloads when incremental policy/tag updates are enabled Key: RANGER-3178 URL: https://issues.apache.org/jira/browse/RANGER-3178 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni A few spurious full policy downloads were observed when # A policy change immediately following a service change. # A tag policy change In such cases, full policy downloads are not required and should be avoided. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (RANGER-3147) enhance resource-trie to enable finding evaluators for a given resource and its children
[ https://issues.apache.org/jira/browse/RANGER-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reopened RANGER-3147: Need to fix path matching algorithm for recursive, non-wildcard path policy when access request scope is SELF_OR_CHILD > enhance resource-trie to enable finding evaluators for a given resource and > its children > > > Key: RANGER-3147 > URL: https://issues.apache.org/jira/browse/RANGER-3147 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > For the resource modeled as a path (with a path separator configured in its > definition) it may be desired to search its Trie data to find an exact match > for the resource being searched and its children. Ranger access requests need > to support an additional scope specification (such as SELF_OR_CHILD) to > implement this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3192) Use concurrent read-write lock to ensure that access evaluation and policy/tag updates are mutually exclusive in multi-threaded environment.
Abhay Kulkarni created RANGER-3192: -- Summary: Use concurrent read-write lock to ensure that access evaluation and policy/tag updates are mutually exclusive in multi-threaded environment. Key: RANGER-3192 URL: https://issues.apache.org/jira/browse/RANGER-3192 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger uses copy and switch method to handle reads and writes to policy and tag repositories in a multi-threaded environment. Using read/write lock to handle concurrent accesses will save on copy which is more memory and CPU efficient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3192) Use read-write locks for managing access to policy-engine and tag-repository
[ https://issues.apache.org/jira/browse/RANGER-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-3192: --- Description: Use concurrent read-write lock to ensure that access evaluation and policy/tag updates are mutually exclusive in multi-threaded environment Ranger uses copy and switch method to handle reads and writes to policy and tag repositories in a multi-threaded environment. Using read/write lock to handle concurrent accesses will save on copy which is more memory and CPU efficient. was:Ranger uses copy and switch method to handle reads and writes to policy and tag repositories in a multi-threaded environment. Using read/write lock to handle concurrent accesses will save on copy which is more memory and CPU efficient. > Use read-write locks for managing access to policy-engine and tag-repository > > > Key: RANGER-3192 > URL: https://issues.apache.org/jira/browse/RANGER-3192 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Use concurrent read-write lock to ensure that access evaluation and > policy/tag updates are mutually exclusive in multi-threaded environment > Ranger uses copy and switch method to handle reads and writes to policy and > tag repositories in a multi-threaded environment. Using read/write lock to > handle concurrent accesses will save on copy which is more memory and CPU > efficient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3192) Use read-write locks for managing access to policy-engine and tag-repository
[ https://issues.apache.org/jira/browse/RANGER-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-3192: --- Summary: Use read-write locks for managing access to policy-engine and tag-repository (was: Use concurrent read-write lock to ensure that access evaluation and policy/tag updates are mutually exclusive in multi-threaded environment.) > Use read-write locks for managing access to policy-engine and tag-repository > > > Key: RANGER-3192 > URL: https://issues.apache.org/jira/browse/RANGER-3192 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Ranger uses copy and switch method to handle reads and writes to policy and > tag repositories in a multi-threaded environment. Using read/write lock to > handle concurrent accesses will save on copy which is more memory and CPU > efficient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3192) Use read-write locks for managing access to policy-engine and tag-repository
[ https://issues.apache.org/jira/browse/RANGER-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292015#comment-17292015 ] Abhay Kulkarni edited comment on RANGER-3192 at 3/13/21, 3:08 AM: -- Patch is available at the review board: [https://reviews.apache.org/r/73206/] Additional patch https://reviews.apache.org/r/73227/ was (Author: abhayk): Patch is available at the review board: https://reviews.apache.org/r/73206/ > Use read-write locks for managing access to policy-engine and tag-repository > > > Key: RANGER-3192 > URL: https://issues.apache.org/jira/browse/RANGER-3192 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Use concurrent read-write lock to ensure that access evaluation and > policy/tag updates are mutually exclusive in multi-threaded environment > Ranger uses copy and switch method to handle reads and writes to policy and > tag repositories in a multi-threaded environment. Using read/write lock to > handle concurrent accesses will save on copy which is more memory and CPU > efficient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3192) Use read-write locks for managing access to policy-engine and tag-repository
[ https://issues.apache.org/jira/browse/RANGER-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17292180#comment-17292180 ] Abhay Kulkarni edited comment on RANGER-3192 at 3/13/21, 3:09 AM: -- Commit details: master: [https://github.com/apache/ranger/commit/b69c2df7625696b3d38cf1cfbbcf1db4d634c7b0] ranger-2.2: [https://github.com/apache/ranger/commit/c20d0e0a65aed8f7fbd8020acfe1612c9b7887f8] Additional commit: master: [https://github.com/apache/ranger/commit/4a6b9d19870bd3a272fa86d005899530f7cdf739] ranger-2.2: [https://github.com/apache/ranger/commit/30836f8def9a7cf8eeac1e0cf13634edf3a243ea] was (Author: abhayk): Commit details: master: [https://github.com/apache/ranger/commit/b69c2df7625696b3d38cf1cfbbcf1db4d634c7b0] ranger-2.2: [https://github.com/apache/ranger/commit/c20d0e0a65aed8f7fbd8020acfe1612c9b7887f8] > Use read-write locks for managing access to policy-engine and tag-repository > > > Key: RANGER-3192 > URL: https://issues.apache.org/jira/browse/RANGER-3192 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Use concurrent read-write lock to ensure that access evaluation and > policy/tag updates are mutually exclusive in multi-threaded environment > Ranger uses copy and switch method to handle reads and writes to policy and > tag repositories in a multi-threaded environment. Using read/write lock to > handle concurrent accesses will save on copy which is more memory and CPU > efficient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3208) NPE in Ranger policy engine when processing SELF_OR_CHILD scoped search
Abhay Kulkarni created RANGER-3208: -- Summary: NPE in Ranger policy engine when processing SELF_OR_CHILD scoped search Key: RANGER-3208 URL: https://issues.apache.org/jira/browse/RANGER-3208 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni The following is the stack trace seen when Ranger policy engine searches Trie object with SELF_OR_CHILD scope. java.lang.NullPointerException at java.util.AbstractCollection.addAll(AbstractCollection.java:343) at org.apache.ranger.plugin.policyengine.RangerResourceTrie$TrieNode.collectChildEvaluators(RangerResourceTrie.java:1161) at org.apache.ranger.plugin.policyengine.RangerResourceTrie.lambda$getEvaluatorsForResource$0(RangerResourceTrie.java:604) at java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1628) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647) at org.apache.ranger.plugin.policyengine.RangerResourceTrie.getEvaluatorsForResource(RangerResourceTrie.java:604) at org.apache.ranger.plugin.policyengine.RangerResourceTrie.getEvaluatorsForResource(RangerResourceTrie.java:180) at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.getLikelyMatchPolicyEvaluators(RangerPolicyRepository.java:811) at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.getLikelyMatchAccessPolicyEvaluators(RangerPolicyRepository.java:764) at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.getLikelyMatchPolicyEvaluators(RangerPolicyRepository.java:741) at org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.evaluatePoliciesNoAudit(RangerPolicyEngineImpl.java:585) at org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.zoneAwareAccessEvaluationWithNoAudit(RangerPolicyEngineImpl.java:486) at org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.evaluatePolicies(RangerPolicyEngineImpl.java:110) at org.apache.ranger.plugin.service.RangerBasePlugin.isAccessAllowed(RangerBasePlugin.java:356) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3218) User getting denied even after having tag based policy
Abhay Kulkarni created RANGER-3218: -- Summary: User getting denied even after having tag based policy Key: RANGER-3218 URL: https://issues.apache.org/jira/browse/RANGER-3218 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Steps 1.Created a database "vehicle1" with table "cars" and inserted some data into table with hive user. 2.Tried to access "vehicle1" with user 'unixuser1' which will be denied since policy is not there. select * from vehicle1.cars; 3.Created a tag "tag1" in Atlas and assigned to database (vehicle1) 4.Created a unzone policy for "tag1" in cm_tag and gave permission to "unixuser1". 5.Again tried to access the data with user 'unixuser1' but still it is getting denied after having policy for the resource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3218) User getting denied even after having tag based policy
[ https://issues.apache.org/jira/browse/RANGER-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306310#comment-17306310 ] Abhay Kulkarni edited comment on RANGER-3218 at 3/23/21, 9:56 PM: -- Patch is available at the review board: [https://reviews.apache.org/r/73245/] Additional patch: https://reviews.apache.org/r/73250/ was (Author: abhayk): Patch is available at the review board: https://reviews.apache.org/r/73245/ > User getting denied even after having tag based policy > -- > > Key: RANGER-3218 > URL: https://issues.apache.org/jira/browse/RANGER-3218 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Steps > 1.Created a database "vehicle1" with table "cars" and inserted some data into > table with hive user. > 2.Tried to access "vehicle1" with user 'unixuser1' which will be denied since > policy is not there. > select * from vehicle1.cars; > 3.Created a tag "tag1" in Atlas and assigned to database (vehicle1) > 4.Created a unzone policy for "tag1" in cm_tag and gave permission to > "unixuser1". > 5.Again tried to access the data with user 'unixuser1' but still it is > getting denied after having policy for the resource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3218) User getting denied even after having tag based policy
[ https://issues.apache.org/jira/browse/RANGER-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306503#comment-17306503 ] Abhay Kulkarni edited comment on RANGER-3218 at 3/23/21, 10:42 PM: --- Commit details: master: [https://github.com/apache/ranger/commit/0737fe845c6406af8388d0ef57d0071a8e06dea1] ranger-2.2: [https://github.com/apache/ranger/commit/1ca3846f592013bab38f1fcbdab2edb6d8b5bf5c] Additional commit: master: [https://github.com/apache/ranger/commit/3a504bd5af6946f020fb7e9b2f10610bb2c2dafe] ranger-2.2: [https://github.com/apache/ranger/commit/c32688c2c89d5ea646b6e112932978d19edca8d7] was (Author: abhayk): Commit details: master: [https://github.com/apache/ranger/commit/0737fe845c6406af8388d0ef57d0071a8e06dea1] ranger-2.2: [https://github.com/apache/ranger/commit/1ca3846f592013bab38f1fcbdab2edb6d8b5bf5c] > User getting denied even after having tag based policy > -- > > Key: RANGER-3218 > URL: https://issues.apache.org/jira/browse/RANGER-3218 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Steps > 1.Created a database "vehicle1" with table "cars" and inserted some data into > table with hive user. > 2.Tried to access "vehicle1" with user 'unixuser1' which will be denied since > policy is not there. > select * from vehicle1.cars; > 3.Created a tag "tag1" in Atlas and assigned to database (vehicle1) > 4.Created a unzone policy for "tag1" in cm_tag and gave permission to > "unixuser1". > 5.Again tried to access the data with user 'unixuser1' but still it is > getting denied after having policy for the resource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3220) Zone name is not getting populated for tag based policy.
Abhay Kulkarni created RANGER-3220: -- Summary: Zone name is not getting populated for tag based policy. Key: RANGER-3220 URL: https://issues.apache.org/jira/browse/RANGER-3220 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Zone name is not getting populated in access audit record even when the audit record shows that a tag policy in a security zone made the access decision when Ranger plugins are chained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-3220) Zone name is not getting populated for tag based policy.
[ https://issues.apache.org/jira/browse/RANGER-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307487#comment-17307487 ] Abhay Kulkarni commented on RANGER-3220: Patch is available at https://reviews.apache.org/r/73251/ > Zone name is not getting populated for tag based policy. > > > Key: RANGER-3220 > URL: https://issues.apache.org/jira/browse/RANGER-3220 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Zone name is not getting populated in access audit record even when the audit > record shows that a tag policy in a security zone made the access decision > when Ranger plugins are chained. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3224) Not able to delete security-zone
Abhay Kulkarni created RANGER-3224: -- Summary: Not able to delete security-zone Key: RANGER-3224 URL: https://issues.apache.org/jira/browse/RANGER-3224 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Steps 1. Create a security zone and have a tag service associated with it. 2. Create a tag policy in the tag service for the security zone. 3. Edit security zone to remove tag service association. 4. Now, try to delete the security zone. Zone deletion fails. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (RANGER-3147) enhance resource-trie to enable finding evaluators for a given resource and its children
[
https://issues.apache.org/jira/browse/RANGER-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni reopened RANGER-3147:
Uncovered a bug in handling of policy with resource specification containing
\{USER} macro when the access-request is made with SELF_OR_CHILD scope.
> enhance resource-trie to enable finding evaluators for a given resource and
> its children
>
>
> Key: RANGER-3147
> URL: https://issues.apache.org/jira/browse/RANGER-3147
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
>
> For the resource modeled as a path (with a path separator configured in its
> definition) it may be desired to search its Trie data to find an exact match
> for the resource being searched and its children. Ranger access requests need
> to support an additional scope specification (such as SELF_OR_CHILD) to
> implement this feature.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-3147) enhance resource-trie to enable finding evaluators for a given resource and its children
[ https://issues.apache.org/jira/browse/RANGER-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313965#comment-17313965 ] Abhay Kulkarni commented on RANGER-3147: Additional patch is available at the review board: https://reviews.apache.org/r/73268/ > enhance resource-trie to enable finding evaluators for a given resource and > its children > > > Key: RANGER-3147 > URL: https://issues.apache.org/jira/browse/RANGER-3147 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > For the resource modeled as a path (with a path separator configured in its > definition) it may be desired to search its Trie data to find an exact match > for the resource being searched and its children. Ranger access requests need > to support an additional scope specification (such as SELF_OR_CHILD) to > implement this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3249) Enhance RangerScriptExecuteContext class to provide APIs for comprehensive tag information
Abhay Kulkarni created RANGER-3249: -- Summary: Enhance RangerScriptExecuteContext class to provide APIs for comprehensive tag information Key: RANGER-3249 URL: https://issues.apache.org/jira/browse/RANGER-3249 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Use case: When an accessed resource is associated with multiple tags of the same type but with different attribute values, current API to retrieve value of an attribute incorrectly returns a scalar attribute value of any of the associated tags. This may lead to incorrect access evaluation. The API needs to return an array of values to caller. The caller then may decide how to use the values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3250) Add relevant indexes to database table to speed up ingress processing of tagged entities
Abhay Kulkarni created RANGER-3250: -- Summary: Add relevant indexes to database table to speed up ingress processing of tagged entities Key: RANGER-3250 URL: https://issues.apache.org/jira/browse/RANGER-3250 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Tagged entities are persisted in a Relational database table. During ingress of tagged entities, Ranger admin needs to look up this table. Indexing the table on the columns that are used for look-up will speed up ingress rate. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-3249) Enhance RangerScriptExecutionContext class to provide APIs for comprehensive tag information
[ https://issues.apache.org/jira/browse/RANGER-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni updated RANGER-3249: --- Summary: Enhance RangerScriptExecutionContext class to provide APIs for comprehensive tag information (was: Enhance RangerScriptExecuteContext class to provide APIs for comprehensive tag information) > Enhance RangerScriptExecutionContext class to provide APIs for comprehensive > tag information > > > Key: RANGER-3249 > URL: https://issues.apache.org/jira/browse/RANGER-3249 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Use case: When an accessed resource is associated with multiple tags of the > same type but with different attribute values, current API to retrieve value > of an attribute incorrectly returns a scalar attribute value of any of the > associated tags. This may lead to incorrect access evaluation. The API needs > to return an array of values to caller. The caller then may decide how to use > the values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3253) Make incremental policy change computation more resilient
Abhay Kulkarni created RANGER-3253: -- Summary: Make incremental policy change computation more resilient Key: RANGER-3253 URL: https://issues.apache.org/jira/browse/RANGER-3253 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Ranger admin, when incremental policies are enabled, retrieves changes to policies from database since last provided policy-version and applies these changes on the cached policies to compute new set of policies. This computation needs to be more resilient - for example - if a change suggests that a policy is created, but it already exists in the policy-cache, then it should not be added again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (RANGER-3253) Make incremental policy change computation more resilient
[ https://issues.apache.org/jira/browse/RANGER-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhay Kulkarni reopened RANGER-3253: Messages at 'WARN' level are not necessarily describing unusual condition. > Make incremental policy change computation more resilient > - > > Key: RANGER-3253 > URL: https://issues.apache.org/jira/browse/RANGER-3253 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Abhay Kulkarni >Assignee: Abhay Kulkarni >Priority: Major > > Ranger admin, when incremental policies are enabled, retrieves changes to > policies from database since last provided policy-version and applies these > changes on the cached policies to compute new set of policies. This > computation needs to be more resilient - for example - if a change suggests > that a policy is created, but it already exists in the policy-cache, then it > should not be added again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3280) Ensure that the policy/tag versions gets correctly updated for every change to service/policy/tag
Abhay Kulkarni created RANGER-3280: -- Summary: Ensure that the policy/tag versions gets correctly updated for every change to service/policy/tag Key: RANGER-3280 URL: https://issues.apache.org/jira/browse/RANGER-3280 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Every change to service/policy/tag should be recorded by incrementing the policy-version (or tag-version) for the service. Policy-version(or tag-version) is updated in a separate transaction after the original change that caused the version change has been committed. This opens up a small window where the policy-version may be overwritten when multiple such transactions are concurrently active, leading to some 'lost' updates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3284) Simplify processing of tasks scheduled to execute after current transaction is completed
Abhay Kulkarni created RANGER-3284: -- Summary: Simplify processing of tasks scheduled to execute after current transaction is completed Key: RANGER-3284 URL: https://issues.apache.org/jira/browse/RANGER-3284 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Abhay Kulkarni Assignee: Abhay Kulkarni Some tasks, such as updating policy/tag versions of a service after a policy/tag is updated, are scheduled to run after the transaction changing policy/tag is completed. The scheduling and execution of such tasks is complicated and can be simplified. -- This message was sent by Atlassian Jira (v8.3.4#803005)
