Abhay Kulkarni created RANGER-2341:
--------------------------------------
Summary: Support for Incremental policy updates to improve
performance of ranger-admin and plugins by optimal building of policy-engine
Key: RANGER-2341
URL: https://issues.apache.org/jira/browse/RANGER-2341
Project: Ranger
Issue Type: Improvement
Components: Ranger
Affects Versions: master
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni
Fix For: master
Requirements:
Currently, every change to any policy causes rebuilding of policy-engine from
scratch. There are several disadvantages:
1. Compute time for rebuilding
2. Large traffic from ranger-admin to each of the plugins
3. Large demand on JVM memory system resulting in frequent garbage collection
and pauses of JVM.
It will be more optimal to communicate only the changes and apply them to
existing policy-engine.
Design notes:
Policy changes are logged into a new database table.
Cache management in ranger-admin is enhanced to use this table to figure out
changes using a previously known version number (provided by module requesting
updated policies).
Policy engine supports update operation that accepts policy-deltas and returns
a new policy engine with deltas applied.
Resource Trie structures are copied from older policy-engine selectively, and
not rebuilt from scratch.
Backward compatibility is maintained with older plugins by adding another
parameter to REST API for downloading policies.
Ranger admin as well as component plugins may be configured to optionally use
policy deltas for its internal policy-engines. Policy deltas are disabled by
default. In ranger-admin, policy-deltas are enabled in the ranger-admin by
setting configuration variable 'ranger.admin.supports.policy.deltas' to true.
In individual plugins, policy-deltas are enabled by setting configuration
variable 'ranger.plugin.<service-type>.policy.rest.supports.policy.deltas' to
"true".
Policy delta table is cleared of records older than a week on restart of
ranger-admin.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
