[
https://issues.apache.org/jira/browse/RANGER-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781211#comment-16781211
]
Don Bosco Durai commented on RANGER-2341:
-----------------------------------------
[~abhayk] this would be a good feature. Few questions...
> Cache management in ranger-admin is enhanced to use this table to figure out
> changes using a previously known version number (provided by module
> requesting updated policies).
Seems more like more like redo logs in database, which I feel is a good
approach.
> Backward compatibility is maintained with older plugins by adding another
> parameter to REST API for downloading policies.
Should we do the other way? New plugins should pass the addition param, so that
older plugins will work without change?
> Policy deltas are disabled by default.
I feel, we should enable this by default. This is a good feature and let the
plugins decide whether to use or not.
> Policy delta table is cleared of records older than a week on restart of
> ranger-admin.
I not sure whether restart should be the trigger, but might be okay for now
till have an inbuilt scheduler. I assume, we will make the the retention period
configurable.
> Support for Incremental policy updates to improve performance of ranger-admin
> and plugins by optimal building of policy-engine
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: RANGER-2341
> URL: https://issues.apache.org/jira/browse/RANGER-2341
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
> Affects Versions: master
> Reporter: Abhay Kulkarni
> Assignee: Abhay Kulkarni
> Priority: Major
> Fix For: master
>
>
> Requirements:
> Currently, every change to any policy causes rebuilding of policy-engine from
> scratch. There are several disadvantages:
> 1. Compute time for rebuilding
> 2. Large traffic from ranger-admin to each of the plugins
> 3. Large demand on JVM memory system resulting in frequent garbage collection
> and pauses of JVM.
> It will be more optimal to communicate only the changes and apply them to
> existing policy-engine.
> Design notes:
> Policy changes are logged into a new database table.
> Cache management in ranger-admin is enhanced to use this table to figure out
> changes using a previously known version number (provided by module
> requesting updated policies).
> Policy engine supports update operation that accepts policy-deltas and
> returns a new policy engine with deltas applied.
> Resource Trie structures are copied from older policy-engine selectively, and
> not rebuilt from scratch.
> Backward compatibility is maintained with older plugins by adding another
> parameter to REST API for downloading policies.
> Ranger admin as well as component plugins may be configured to optionally use
> policy deltas for its internal policy-engines. Policy deltas are disabled by
> default. In ranger-admin, policy-deltas are enabled in the ranger-admin by
> setting configuration variable 'ranger.admin.supports.policy.deltas' to true.
> In individual plugins, policy-deltas are enabled by setting configuration
> variable 'ranger.plugin.<service-type>.policy.rest.supports.policy.deltas' to
> "true".
> Policy delta table is cleared of records older than a week on restart of
> ranger-admin.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
