[
https://issues.apache.org/jira/browse/RANGER-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyom Mani Tiwari updated RANGER-5851:
-------------------------------------
Description:
RANGER-5838 added constraintDefs to RangerServiceDef. Ranger Admin drops them
when a service def is created or updated. This stores them with the service def
and validates them.
This does not depend on RANGER-5842 and does not change policy evaluation.
Constraint policies stay in x_policy.policy_text. They do not need a table. Do
not add constraintDefs to the Hive, HDFS, or Kafka service defs here. The
plugin-capability gate for adding them to an existing service def is a later
Jira.
*Scope*
* Store constraintDefs on service-def create and update, and return them on
get, through REST, ServiceDBStore, and RangerServiceDefServiceBase.
* Delete them when the service def is deleted. On update, remove rows that are
no longer in the service def.
* Include them in service-def export and import, and in the service def
downloaded by plugins.
* Validate them in RangerServiceDefValidator.
*Out of scope*
* Policy-engine evaluation.
* A check that rejects removing or renaming a constraintDef while policies
still reference it. That can scan policy_text in a follow-up.
* RangerPluginCapability.
* The Constraints tab and the Agent Studio service def.
*Storage*
Add x_constraint_def, following x_policy_condition_def.
* Standard columns: id, guid, create_time, update_time, added_by_id, upd_by_id.
* def_id, a foreign key to x_service_def, and item_id.
* name, label, description, rb_key_label, rb_key_description. Same string sizes
as x_policy_condition_def.
* resources, access_types, parameters, and options, stored as JSON text. Use
MEDIUMTEXT on MySQL, TEXT on Postgres and SQL Anywhere, CLOB on Oracle, and
nvarchar(max) on SQL Server. Do not use VARCHAR(1024). A parameter list will
not fit there.
* sort_order, set the same way as x_policy_condition_def.
RangerConstraintParamDef values are stored in the parameters JSON on the
constraint row. They are always read with that row, so there is no child table.
The JSON fields are itemId, name, label, description, type, subType, mandatory,
defaultValue, rbKeyLabel, and rbKeyDescription.
Schema changes:
* Add the table to the current schema for MySQL, Postgres, Oracle, SQL Server,
and SQL Anywhere.
* Add an upgrade patch for each flavor.
* Update the drop scripts.
*Code*
* XXConstraintDef and XXConstraintDefDao, registered in RangerDaoManager, with
find-by-def-id and find-by-def-id-and-name.
* RangerServiceDefServiceBase maps constraintDefs both ways, as it does for
policy conditions.
* ServiceDBStore creates, updates, and deletes the rows with the service def.
*Validation*
In RangerServiceDefValidator:
* Names and itemIds are unique and not empty.
* Each resource name and access type exists on the service def.
* For every named resource, each ancestor, found through the resource parent,
is also in that constraint's resources list. A resources list that names UDF
and not Database is invalid. An empty resources list is valid. The engine
treats an empty list as the access hierarchy.
* Do not call getResourceHierarchiesForConstraint. That method is part of
RANGER-5842 and is not on master. The parent-link check above is enough.
* Parameter names are unique and not empty.
* Parameter type is one of bool, enum, int, string, password, path, and user.
user is required for requireApproval. The existing config-type set does not
include it.
* When type is enum, subType is required and names a RangerEnumDef on the same
service def. For every other type, subType is empty.
* mandatory may be null. The model treats null as false. defaultValue is stored
as a string.
*Testing*
* Validator unit tests. Cover a chain of Database and UDF as valid, UDF without
Database as invalid, and an empty resources list as valid. Accept type user.
Reject an unknown type. Reject an enum whose subType does not match an enum def.
* ServiceDBStore and RangerServiceDefServiceBase round-trip: create, update by
adding, changing, and removing a constraintDef, get, and delete.
* Apply the upgrade patch on each database flavor.
was:
RANGER-5838 added {{constraintDefs}} to {{{}RangerServiceDef{}}}. Ranger Admin
doesn’t store them yet, so they’re dropped when a service def is created or
updated. This sub-task adds that storage.
_Scope_
* Store {{constraintDefs}} on service def create and update, and return them
on get, through REST and {{ServiceDBStore}} /
{{{}RangerServiceDefServiceBase{}}}.
* Validate constraintDefs in {{{}RangerServiceDefValidator{}}}:
** names and itemIds are unique and not empty;
** {{resources}} and {{accessTypes}} refer to resources and access types that
exist in the service def;
** each {{resources}} list forms a valid chain that includes its ancestors;
** parameter names are unique, and parameter {{type}} values are supported.
* Include constraintDefs in service-def export/import and in the service def
sent to plugins.
* Updates: decide what happens to constraint policies that reference a
constraintDef that is removed or renamed. Proposal: reject the update while
policies reference it, as is done for access types in use.
_Storage: new table {{x_constraint_def}}_
* Columns:
** standard: {{{}id{}}}, {{{}guid{}}}, {{{}create_time{}}},
{{{}update_time{}}}, {{{}added_by_id{}}}, {{{}upd_by_id{}}};
** {{def_id}} (FK to {{{}x_service_def{}}}) and {{{}item_id{}}};
** {{{}name{}}}, {{{}label{}}}, {{{}description{}}}, {{{}rb_key_label{}}},
{{{}rb_key_description{}}};
** {{{}resources{}}}, {{{}access_types{}}}, {{{}parameters{}}}, {{options}} as
JSON text;
** {{{}sort_order{}}}.
* Parameters ({{{}RangerConstraintParamDef{}}}) are stored as JSON on the
constraint row. They’re always read with their constraint, so there’s no
separate table.
* Schema changes:
** the table in the current schema for MySQL, Postgres, Oracle, SQL Server and
SQL Anywhere;
** an upgrade patch for each flavor;
** the drop scripts updated.
_Code_
* {{XXConstraintDef}} entity and {{{}XXConstraintDefDao{}}}, registered in
{{{}RangerDaoManager{}}}, with find-by-def-id and find-by-def-id-and-name
queries.
* {{{}RangerServiceDefServiceBase{}}}: map constraintDefs both ways, as for
{{{}policyConditions{}}}.
* {{{}ServiceDBStore{}}}:
** create, update and delete constraintDefs with the service def, removing
those dropped on update;
** delete them when the service def is deleted.
* Include constraintDefs in service-def export/import and in the service def
sent to plugins.
_Validation ({{{}RangerServiceDefValidator{}}})_
* Names and itemIds are unique and not empty.
* {{resources}} and {{accessTypes}} refer to resources and access types that
exist in the service def.
* {{resources}} covers whole hierarchy chains, including ancestors, so
{{getResourceHierarchiesForConstraint()}} returns at least one hierarchy.
* Parameter names are unique and parameter {{type}} values are supported.
* A constraintDef can’t be removed or renamed while policies reference it. The
check uses the policy-ref data added in B2; until B2 lands, it’s a TODO.
_Testing_
* Validator unit tests.
* {{ServiceDBStore}} / {{RangerServiceDefServiceBase}} round-trip tests:
create, update (add, change and remove a constraintDef), get and delete.
* The upgrade patch applied on each DB flavor.
> Admin: persist constraintDefs of service definitions
> ----------------------------------------------------
>
> Key: RANGER-5851
> URL: https://issues.apache.org/jira/browse/RANGER-5851
> Project: Ranger
> Issue Type: New Feature
> Components: admin
> Reporter: Vyom Mani Tiwari
> Assignee: Vyom Mani Tiwari
> Priority: Major
>
> RANGER-5838 added constraintDefs to RangerServiceDef. Ranger Admin drops them
> when a service def is created or updated. This stores them with the service
> def and validates them.
> This does not depend on RANGER-5842 and does not change policy evaluation.
> Constraint policies stay in x_policy.policy_text. They do not need a table.
> Do not add constraintDefs to the Hive, HDFS, or Kafka service defs here. The
> plugin-capability gate for adding them to an existing service def is a later
> Jira.
> *Scope*
> * Store constraintDefs on service-def create and update, and return them on
> get, through REST, ServiceDBStore, and RangerServiceDefServiceBase.
> * Delete them when the service def is deleted. On update, remove rows that
> are no longer in the service def.
> * Include them in service-def export and import, and in the service def
> downloaded by plugins.
> * Validate them in RangerServiceDefValidator.
> *Out of scope*
> * Policy-engine evaluation.
> * A check that rejects removing or renaming a constraintDef while policies
> still reference it. That can scan policy_text in a follow-up.
> * RangerPluginCapability.
> * The Constraints tab and the Agent Studio service def.
> *Storage*
> Add x_constraint_def, following x_policy_condition_def.
> * Standard columns: id, guid, create_time, update_time, added_by_id,
> upd_by_id.
> * def_id, a foreign key to x_service_def, and item_id.
> * name, label, description, rb_key_label, rb_key_description. Same string
> sizes as x_policy_condition_def.
> * resources, access_types, parameters, and options, stored as JSON text. Use
> MEDIUMTEXT on MySQL, TEXT on Postgres and SQL Anywhere, CLOB on Oracle, and
> nvarchar(max) on SQL Server. Do not use VARCHAR(1024). A parameter list will
> not fit there.
> * sort_order, set the same way as x_policy_condition_def.
> RangerConstraintParamDef values are stored in the parameters JSON on the
> constraint row. They are always read with that row, so there is no child
> table. The JSON fields are itemId, name, label, description, type, subType,
> mandatory, defaultValue, rbKeyLabel, and rbKeyDescription.
> Schema changes:
> * Add the table to the current schema for MySQL, Postgres, Oracle, SQL
> Server, and SQL Anywhere.
> * Add an upgrade patch for each flavor.
> * Update the drop scripts.
> *Code*
> * XXConstraintDef and XXConstraintDefDao, registered in RangerDaoManager,
> with find-by-def-id and find-by-def-id-and-name.
> * RangerServiceDefServiceBase maps constraintDefs both ways, as it does for
> policy conditions.
> * ServiceDBStore creates, updates, and deletes the rows with the service def.
> *Validation*
> In RangerServiceDefValidator:
> * Names and itemIds are unique and not empty.
> * Each resource name and access type exists on the service def.
> * For every named resource, each ancestor, found through the resource parent,
> is also in that constraint's resources list. A resources list that names UDF
> and not Database is invalid. An empty resources list is valid. The engine
> treats an empty list as the access hierarchy.
> * Do not call getResourceHierarchiesForConstraint. That method is part of
> RANGER-5842 and is not on master. The parent-link check above is enough.
> * Parameter names are unique and not empty.
> * Parameter type is one of bool, enum, int, string, password, path, and user.
> user is required for requireApproval. The existing config-type set does not
> include it.
> * When type is enum, subType is required and names a RangerEnumDef on the
> same service def. For every other type, subType is empty.
> * mandatory may be null. The model treats null as false. defaultValue is
> stored as a string.
> *Testing*
> * Validator unit tests. Cover a chain of Database and UDF as valid, UDF
> without Database as invalid, and an empty resources list as valid. Accept
> type user. Reject an unknown type. Reject an enum whose subType does not
> match an enum def.
> * ServiceDBStore and RangerServiceDefServiceBase round-trip: create, update
> by adding, changing, and removing a constraintDef, get, and delete.
> * Apply the upgrade patch on each database flavor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
