[jira] [Commented] (RANGER-3830) Estimated date for Apache Ranger 2.3.0 and 3.0.0 release
[ https://issues.apache.org/jira/browse/RANGER-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566671#comment-17566671 ] Ramesh Mani commented on RANGER-3830: - [~puram.vaibhav] Apache Ranger 2.3.0 is already released. Please review the Release Notes for features and improvements [https://ranger.apache.org/] > Estimated date for Apache Ranger 2.3.0 and 3.0.0 release > > > Key: RANGER-3830 > URL: https://issues.apache.org/jira/browse/RANGER-3830 > Project: Ranger > Issue Type: Wish > Components: documentation >Reporter: Phani Vaibhav Puram >Priority: Major > > Hi All, > > We have a project that uses apache Ranger 2.2.0 and we would now want to > upgrade it to the latest version available before we move to production. Is > there any estimated date for Apache Ranger 2.3.0 and 3.0.0 so that we can > plan our project accordingly? Also any feature list for the above releases > would be helpful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-3830) Estimated date for Apache Ranger 2.3.0 and 3.0.0 release
Phani Vaibhav Puram created RANGER-3830: --- Summary: Estimated date for Apache Ranger 2.3.0 and 3.0.0 release Key: RANGER-3830 URL: https://issues.apache.org/jira/browse/RANGER-3830 Project: Ranger Issue Type: Wish Components: documentation Reporter: Phani Vaibhav Puram Hi All, We have a project that uses apache Ranger 2.2.0 and we would now want to upgrade it to the latest version available before we move to production. Is there any estimated date for Apache Ranger 2.3.0 and 3.0.0 so that we can plan our project accordingly? Also any feature list for the above releases would be helpful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-3824) [Ranger] : /service/tags/resources error message is not proper for duplicate resource & not able to update resource resource
[ https://issues.apache.org/jira/browse/RANGER-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reassigned RANGER-3824: --- Assignee: Pradeep Agrawal > [Ranger] : /service/tags/resources error message is not proper for duplicate > resource & not able to update resource resource > > > Key: RANGER-3824 > URL: https://issues.apache.org/jira/browse/RANGER-3824 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Anupam Rai >Assignee: Pradeep Agrawal >Priority: Major > > Hi > Post request for /service/tags/resources error message is not proper for > duplicate resource. > Steps to reproduce : > 1. Make post request to /service/tags/resources API > eg : > {code:java} > curl --location --request POST 'https://host:6182/service/tags/resources' \ > --header 'Authorization: Basic auth==' \ > --header 'Content-Type: application/json' \ > --data-raw '{"serviceName": "cm_hdfs", "resourceElements": {"path": > {"values": ["/dummy/ddd"]}}} > ' {code} > Response : 400 Bad Request > {code:java} > Exception [EclipseLink-4002 > ] (Eclipse Persistence Services - 2.7.7.v20200504-69f2c2b80d): > org.eclipse.persistence.exceptions.DatabaseException > Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key > value violates unique constraint > "x_service_resource_idx_svc_id_resource_signature" > Detail: Key (service_id, resource_signature)=(1, > e7bc212b933a480d3dbb6ae3b6168f0482d5d7a1646693c8ba1801ed818ca837) already > exists. > Error Code: 0 > Call: INSERT INTO x_service_resource (id, ADDED_BY_ID, CREATE_TIME, guid, > is_enabled, resource_signature, service_id, service_resource_elements_text, > tags_text, UPDATE_TIME, UPD_BY_ID, version) VALUES (?, ?, ?, ?, ?, ?, ?, ?, > ?, ?, ?, ?) > bind => [ > 12 parameters bound > ] {code} > Expected : Error message to be proper > Condition 1 : updateIfexists = true , should update existing resource > Condition 2: updateIfexists = False , should give proper error message > Thanks -- This message was sent by Atlassian Jira (v8.20.10#820010)
Review Request 74058: RANGER-3829: Incremental Sync Value to be read from usersync config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74058/ --- Review request for ranger, Sailaja Polavarapu and Velmurugan Periasamy. Bugs: RANGER-3829 https://issues.apache.org/jira/browse/RANGER-3829 Repository: ranger Description --- Config parameter ranger.usersync.ldap.deltasync is hardcoded to true, the review allows the param to be read from config file. Diffs - security-admin/src/main/java/org/apache/ranger/view/VXLdapSyncSourceInfo.java d349a9e25 ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/LdapSyncSourceInfo.java 227c89792 ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java 550775f65 ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java 531e35a95 Diff: https://reviews.apache.org/r/74058/diff/1/ Testing --- Thanks, Abhishek Kumar
[jira] [Created] (RANGER-3829) Incremental Sync value is always true under Ranger Audit (Usersync)
Abhishek Kumar created RANGER-3829: -- Summary: Incremental Sync value is always true under Ranger Audit (Usersync) Key: RANGER-3829 URL: https://issues.apache.org/jira/browse/RANGER-3829 Project: Ranger Issue Type: Bug Components: usersync Reporter: Abhishek Kumar Assignee: Abhishek Kumar Disabled the Incremental Sync in the Usersync configs but the *_Ranger UI -> Audit -> Usersync -> Sync Details_* shows the Incremental Sync value as always true. I could see the configs as - {code:java} [root@c3245-node2 conf]# cat ranger-ugsync-site.xml | grep -a2 delta ranger.usersync.ldap.deltasync false {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74042: Fix ConcurrentModificationException in UnixUserGroupBuilder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74042/#review224561 --- Ship it! Ship It! - Sailaja Polavarapu On July 14, 2022, 12:17 a.m., Abhishek Kumar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74042/ > --- > > (Updated July 14, 2022, 12:17 a.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, and > Sailaja Polavarapu. > > > Bugs: RANGER-3813 > https://issues.apache.org/jira/browse/RANGER-3813 > > > Repository: ranger > > > Description > --- > > Line number 426 in > ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java > updates the map while iteration which raises the exception > ConcurrentModificationException. > > > Steps to reproduce the issue: > 1. Set nss and enumerateGroupMembers to true. > 2. Create a user with userid < minimumUserId. > 3. Add it to a group with groupId >= minimumGroupId. > 4. Ensure the user is part of multiple groups and any one group that the user > is part of does not show the user as its member on executing: getent group > > > Diffs > - > > > ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java > 7653dfdbe > > > Diff: https://reviews.apache.org/r/74042/diff/5/ > > > Testing > --- > > Tested the fix on a live cluster. > > > Thanks, > > Abhishek Kumar > >
[jira] [Comment Edited] (RANGER-3387) Ranger Admin Header Validation.
[ https://issues.apache.org/jira/browse/RANGER-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566563#comment-17566563 ] Sailaja Polavarapu edited comment on RANGER-3387 at 7/14/22 12:19 AM: -- Merged follow-up changes to master to handle requests from Knox gateway was (Author: spolavarapu): Merged changes to master. > Ranger Admin Header Validation. > --- > > Key: RANGER-3387 > URL: https://issues.apache.org/jira/browse/RANGER-3387 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.2.0 >Reporter: Mateen N Mansoori >Assignee: Sailaja Polavarapu >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3387-Added-extra-validation-for-handling-PUT-.patch, > 0001-RANGER-3387-Ranger-Admin-Header-Validation.patch > > > Ranger Admin Header Validation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74042: Fix ConcurrentModificationException in UnixUserGroupBuilder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74042/ --- (Updated July 13, 2022, 11:57 p.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, and Sailaja Polavarapu. Changes --- fixed string concat! Bugs: RANGER-3813 https://issues.apache.org/jira/browse/RANGER-3813 Repository: ranger Description --- Line number 426 in ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java updates the map while iteration which raises the exception ConcurrentModificationException. Steps to reproduce the issue: 1. Set nss and enumerateGroupMembers to true. 2. Create a user with userid < minimumUserId. 3. Add it to a group with groupId >= minimumGroupId. 4. Ensure the user is part of multiple groups and any one group that the user is part of does not show the user as its member on executing: getent group Diffs (updated) - ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java 7653dfdbe Diff: https://reviews.apache.org/r/74042/diff/4/ Changes: https://reviews.apache.org/r/74042/diff/3-4/ Testing --- Tested the fix on a live cluster. Thanks, Abhishek Kumar
Review Request 74057: Plugin for Fine-grained Access Control over nested structures
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74057/ --- Review request for ranger. Repository: ranger Description --- It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. Diffs - plugin-nestedstructure/CONTRIBUTING PRE-CREATION plugin-nestedstructure/LICENSE PRE-CREATION plugin-nestedstructure/NOTICE PRE-CREATION plugin-nestedstructure/README.md PRE-CREATION plugin-nestedstructure/conf/log4j.properties PRE-CREATION plugin-nestedstructure/conf/nestedstructure_servicedef.json PRE-CREATION plugin-nestedstructure/conf/ranger-nestedstructure-audit.xml PRE-CREATION plugin-nestedstructure/conf/ranger-nestedstructure-policymgr-ssl.xml PRE-CREATION plugin-nestedstructure/conf/ranger-nestedstructure-security.xml PRE-CREATION plugin-nestedstructure/pom.xml PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/AccessResult.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/DataMasker.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/ExampleClient.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/FieldLevelAccess.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/JsonManipulator.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskTypes.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/MaskingException.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAccessType.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructureAuthorizer.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Resource.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/NestedStructure_Service.java PRE-CREATION plugin-nestedstructure/src/main/java/org.apache.ranger/authorization.nestedstructure.authorizer/RecordFilterJavaScript.java PRE-CREATION plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestDataMasker.java PRE-CREATION plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestJsonManipulator.java PRE-CREATION plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java PRE-CREATION pom.xml 0945f4b1d tagsync/src/main/java/org/apache/ranger/tagsync/nestedstructureplugin/AtlasNestedStructureResourceMapper.java PRE-CREATION tagsync/src/test/java/org/apache/ranger/tagsync/nestedstructureplugin/ResourceTests.java PRE-CREATION Diff: https://reviews.apache.org/r/74057/diff/1/ Testing --- Thanks, Barbara Eckman
[jira] [Updated] (RANGER-3828) Fine-grained Access Control over nested structures
[ https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barbara Eckman updated RANGER-3828: --- Description: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. was: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. *patch is coming soon* > Fine-grained Access Control over nested structures > -- > > Key: RANGER-3828 > URL: https://issues.apache.org/jira/browse/RANGER-3828 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Reporter: Barbara Eckman >Priority: Major > > It would be nice to be able to do fine-grained access control (FGA) over > nested structures, e.g., the JSON responses of API calls. This requires the > individual attributes in a JSON object to be first-class metadata objects > which can be tagged and on which policies can be written. We have built a > plugin and the corresponding Apache Atlas metadata structures and > tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our > instigating use case was FGA over the JSON responses of API calls, but this > plugin has potential value anywhere FGA over the individual attributes of > nested structures is needed, eg JSON messages read from Kafka topics. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566428#comment-17566428 ] Andras Katona commented on RANGER-3809: --- Could we keep this ticket open, as for the proper authorizeByResourceType implementation too? We just made the plugin less ugly.. :) > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Mani resolved RANGER-3809. - Resolution: Fixed > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Mani updated RANGER-3809: Affects Version/s: 3.0.0 > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Mani updated RANGER-3809: Fix Version/s: 3.0.0 > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566421#comment-17566421 ] Ramesh Mani commented on RANGER-3809: - [~akatona] Thanks for the contribution. Patch has been committed to Apache master. Please close the the PR. [https://gitbox.apache.org/repos/asf?p=ranger.git;a=commit;h=0ec279474e0439f6c5e7d4497db191fb7cc99bc1] > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74056: RANGER-3809: Dummy impl for RangerKafkaAuthorizer#authorizeByResourceType
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74056/#review224560 --- Ship it! Ship It! - Ramesh Mani On July 13, 2022, 4:29 p.m., Andras Katona wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74056/ > --- > > (Updated July 13, 2022, 4:29 p.m.) > > > Review request for ranger, Ramesh Mani and Viktor Somogyi-Vass. > > > Bugs: RANGER-3809 > https://issues.apache.org/jira/browse/RANGER-3809 > > > Repository: ranger > > > Description > --- > > Since the current implementation of the acls() call throws > UnsupportedOperationException, it masks an authorization error if a > Kafka client tries to call the InitProducerId API and doesn't have > idempotent_write permission on the cluster nor it has a transactional.id > configured. > > Until a proper implementation of the acls() method is done by RANGER-3809 > we override authorizeByResourceType to get an access denied on the > client side instead of an exception. > > > Diffs > - > > > plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java > 64f622586 > > plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java > e82de1849 > > plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerSASLSSLTest.java > b25d1fdd1 > > plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java > d24ee1e57 > > plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerTopicCreationTest.java > 201064970 > > plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaTestUtils.java > dc8277051 > plugin-kafka/src/test/resources/kafka-policies.json 70c978cce > > > Diff: https://reviews.apache.org/r/74056/diff/1/ > > > Testing > --- > > unit tests, manually > > > Thanks, > > Andras Katona > >
[jira] [Updated] (RANGER-3828) Fine-grained Access Control over nested structures
[ https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barbara Eckman updated RANGER-3828: --- Description: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. *patch is coming soon* was: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. ** patch is coming soon ** > Fine-grained Access Control over nested structures > -- > > Key: RANGER-3828 > URL: https://issues.apache.org/jira/browse/RANGER-3828 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Reporter: Barbara Eckman >Priority: Major > > It would be nice to be able to do fine-grained access control (FGA) over > nested structures, e.g., the JSON responses of API calls. This requires the > individual attributes in a JSON object to be first-class metadata objects > which can be tagged and on which policies can be written. We have built a > plugin and the corresponding Apache Atlas metadata structures and > tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our > instigating use case was FGA over the JSON responses of API calls, but this > plugin has potential value anywhere FGA over the individual attributes of > nested structures is needed, eg JSON messages read from Kafka topics. > *patch is coming soon* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3828) Fine-grained Access Control over nested structures
[ https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barbara Eckman updated RANGER-3828: --- Description: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. ** patch is coming soon ** was:It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. > Fine-grained Access Control over nested structures > -- > > Key: RANGER-3828 > URL: https://issues.apache.org/jira/browse/RANGER-3828 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Reporter: Barbara Eckman >Priority: Major > > It would be nice to be able to do fine-grained access control (FGA) over > nested structures, e.g., the JSON responses of API calls. This requires the > individual attributes in a JSON object to be first-class metadata objects > which can be tagged and on which policies can be written. We have built a > plugin and the corresponding Apache Atlas metadata structures and > tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our > instigating use case was FGA over the JSON responses of API calls, but this > plugin has potential value anywhere FGA over the individual attributes of > nested structures is needed, eg JSON messages read from Kafka topics. > ** patch is coming soon ** -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3828) Fine-grained Access Control over nested structures
[ https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barbara Eckman updated RANGER-3828: --- Description: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures and tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. (was: It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics.) > Fine-grained Access Control over nested structures > -- > > Key: RANGER-3828 > URL: https://issues.apache.org/jira/browse/RANGER-3828 > Project: Ranger > Issue Type: New Feature > Components: plugins, Ranger >Reporter: Barbara Eckman >Priority: Major > > It would be nice to be able to do fine-grained access control (FGA) over > nested structures, e.g., the JSON responses of API calls. This requires the > individual attributes in a JSON object to be first-class metadata objects > which can be tagged and on which policies can be written. We have built a > plugin and the corresponding Apache Atlas metadata structures and > tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures. Our > instigating use case was FGA over the JSON responses of API calls, but this > plugin has potential value anywhere FGA over the individual attributes of > nested structures is needed, eg JSON messages read from Kafka topics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-3828) Fine-grained Access Control over nested structures
Barbara Eckman created RANGER-3828: -- Summary: Fine-grained Access Control over nested structures Key: RANGER-3828 URL: https://issues.apache.org/jira/browse/RANGER-3828 Project: Ranger Issue Type: New Feature Components: plugins, Ranger Reporter: Barbara Eckman It would be nice to be able to do fine-grained access control (FGA) over nested structures, e.g., the JSON responses of API calls. This requires the individual attributes in a JSON object to be first-class metadata objects which can be tagged and on which policies can be written. We have built a plugin and the corresponding Apache Atlas metadata structures to support TBAC/RBAC/ABAC FGA over JSON structures. Our instigating use case was FGA over the JSON responses of API calls, but this plugin has potential value anywhere FGA over the individual attributes of nested structures is needed, eg JSON messages read from Kafka topics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407 ] Andras Katona edited comment on RANGER-3809 at 7/13/22 4:34 PM: First we'll do a dummy impl for the {{authorizeByResourceType }} method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ was (Author: akatona): First we'll do a dummy impl for the method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407 ] Andras Katona edited comment on RANGER-3809 at 7/13/22 4:34 PM: First we'll do a dummy impl for the {{authorizeByResourceType}} method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ was (Author: akatona): First we'll do a dummy impl for the {{authorizeByResourceType }} method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407 ] Andras Katona commented on RANGER-3809: --- First we'll do a dummy impl for the method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
Review Request 74056: RANGER-3809: Dummy impl for RangerKafkaAuthorizer#authorizeByResourceType
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74056/ --- Review request for ranger, Ramesh Mani and Viktor Somogyi-Vass. Bugs: RANGER-3809 https://issues.apache.org/jira/browse/RANGER-3809 Repository: ranger Description --- Since the current implementation of the acls() call throws UnsupportedOperationException, it masks an authorization error if a Kafka client tries to call the InitProducerId API and doesn't have idempotent_write permission on the cluster nor it has a transactional.id configured. Until a proper implementation of the acls() method is done by RANGER-3809 we override authorizeByResourceType to get an access denied on the client side instead of an exception. Diffs - plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java 64f622586 plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java e82de1849 plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerSASLSSLTest.java b25d1fdd1 plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java d24ee1e57 plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerTopicCreationTest.java 201064970 plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaTestUtils.java dc8277051 plugin-kafka/src/test/resources/kafka-policies.json 70c978cce Diff: https://reviews.apache.org/r/74056/diff/1/ Testing --- unit tests, manually Thanks, Andras Katona
Re: Review Request 74035: RANGER-3790: Ranger tagsync module should not depend on kafka server
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74035/ --- (Updated July 13, 2022, 12:58 p.m.) Review request for ranger. Changes --- Further Testing done. Bugs: RANGER-3790 https://issues.apache.org/jira/browse/RANGER-3790 Repository: ranger Description --- This commit removes the unused kafka core dependency from the assembly xml of the tagsync module, so it will not be added to the distribution. Diffs - distro/src/main/assembly/tagsync.xml c467e5a97 Diff: https://reviews.apache.org/r/74035/diff/1/ Testing (updated) --- Tested manually by checking the dependencies for tagsync module in the distribution. Tested the TagSync module on a manually provisioned cluster. Thanks, Patrik Márton
[jira] [Updated] (RANGER-3827) Ranger should block policy creation with incorrect permission
[ https://issues.apache.org/jira/browse/RANGER-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dharshana M Krishnamoorthy updated RANGER-3827: --- Description: {code:java} entity_allow_get_on_hive_table_policy_payload {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': True, u'type': u'entity-update-classification'}, {u'isAllowed': True, u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}}, u'description': u'', u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': None, u'none': [], u'end-one-entity-classification': None, u'name': u'entity_allow_all_hive_table', u'denyExceptions': [], u'policyLabels': []} {code} For an entity type with 'None' , classification related policies can also be added when creating policy via api, which is ideally incorrect This should be blocked at policy creation level itself was: {code:java} entity_allow_get_on_hive_table_policy_payload {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': True, u'type': u'entity-update-classification'}, {u'isAllowed': True, u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}}, u'description': u'', u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': None, u'none': [], u'end-one-entity-classification': None, u'name': u'entity_allow_all_hive_table', u'denyExceptions': [], u'policyLabels': []} {code} !image-2022-07-13-17-18-15-207.png|width=1415,height=691! For an entity type with 'None' , classification related policies can also be added when creating policy via api, which is ideally incorrect This should be blocked at policy creation level itself > Ranger should block policy creation with incorrect permission > - > > Key: RANGER-3827 > URL: https://issues.apache.org/jira/browse/RANGER-3827 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Dharshana M Krishnamoorthy >Priority: Major > Attachments: Screenshot 2022-07-12 at 12.01.09 AM.png > > > {code:java} > entity_allow_get_on_hive_table_policy_payload > {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': > [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, > {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, > u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, > {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': > True, u'type': u'entity-update-classification'}, {u'isAllowed': True, > u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, > u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': > None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': > {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, > u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], > u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, > u'values': [u'*'], u'isRecursive': False}}, u'description': u'', > u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', > u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': > None, u
[jira] [Updated] (RANGER-3827) Ranger should block policy creation with incorrect permission
[ https://issues.apache.org/jira/browse/RANGER-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dharshana M Krishnamoorthy updated RANGER-3827: --- Description: {code:java} entity_allow_get_on_hive_table_policy_payload {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': True, u'type': u'entity-update-classification'}, {u'isAllowed': True, u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}}, u'description': u'', u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': None, u'none': [], u'end-one-entity-classification': None, u'name': u'entity_allow_all_hive_table', u'denyExceptions': [], u'policyLabels': []} {code} Details/screenshot of the incorrect data attached in the attachments For an entity type with 'None' , classification related policies can also be added when creating policy via api, which is ideally incorrect This should be blocked at policy creation level itself was: {code:java} entity_allow_get_on_hive_table_policy_payload {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': True, u'type': u'entity-update-classification'}, {u'isAllowed': True, u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}}, u'description': u'', u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': None, u'none': [], u'end-one-entity-classification': None, u'name': u'entity_allow_all_hive_table', u'denyExceptions': [], u'policyLabels': []} {code} For an entity type with 'None' , classification related policies can also be added when creating policy via api, which is ideally incorrect This should be blocked at policy creation level itself > Ranger should block policy creation with incorrect permission > - > > Key: RANGER-3827 > URL: https://issues.apache.org/jira/browse/RANGER-3827 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Dharshana M Krishnamoorthy >Priority: Major > Attachments: Screenshot 2022-07-12 at 12.01.09 AM.png > > > {code:java} > entity_allow_get_on_hive_table_policy_payload > {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': > [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, > {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, > u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, > {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': > True, u'type': u'entity-update-classification'}, {u'isAllowed': True, > u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, > u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': > None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': > {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, > u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], > u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, > u'values': [u'*'], u'isRecursive': False}}, u'description': u'', > u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', > u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type':
[jira] [Updated] (RANGER-3827) Ranger should block policy creation with incorrect permission
[ https://issues.apache.org/jira/browse/RANGER-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dharshana M Krishnamoorthy updated RANGER-3827: --- Attachment: Screenshot 2022-07-12 at 12.01.09 AM.png > Ranger should block policy creation with incorrect permission > - > > Key: RANGER-3827 > URL: https://issues.apache.org/jira/browse/RANGER-3827 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Dharshana M Krishnamoorthy >Priority: Major > Attachments: Screenshot 2022-07-12 at 12.01.09 AM.png > > > {code:java} > entity_allow_get_on_hive_table_policy_payload > {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': > [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, > {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, > u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, > {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': > True, u'type': u'entity-update-classification'}, {u'isAllowed': True, > u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, > u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': > None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': > {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, > u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], > u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, > u'values': [u'*'], u'isRecursive': False}}, u'description': u'', > u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', > u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': > None, u'none': [], u'end-one-entity-classification': None, u'name': > u'entity_allow_all_hive_table', u'denyExceptions': [], u'policyLabels': []} > {code} > > For an entity type with 'None' , classification related policies can also be > added when creating policy via api, which is ideally incorrect > This should be blocked at policy creation level itself -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (RANGER-3827) Ranger should block policy creation with incorrect permission
Dharshana M Krishnamoorthy created RANGER-3827: -- Summary: Ranger should block policy creation with incorrect permission Key: RANGER-3827 URL: https://issues.apache.org/jira/browse/RANGER-3827 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Dharshana M Krishnamoorthy {code:java} entity_allow_get_on_hive_table_policy_payload {u'allowExceptions': [], u'end-one-entity': None, u'policyItems': [{u'users': [u'hrt_16'], u'accesses': [{u'isAllowed': True, u'type': u'entity-read'}, {u'isAllowed': True, u'type': u'entity-create'}, {u'isAllowed': True, u'type': u'entity-update'}, {u'isAllowed': True, u'type': u'entity-delete'}, {u'isAllowed': True, u'type': u'entity-add-classification'}, {u'isAllowed': True, u'type': u'entity-update-classification'}, {u'isAllowed': True, u'type': u'entity-remove-classification'}]}], u'policyPriority': 0, u'service': 'cm_atlas', u'isEnabled': True, u'end-two-entity-classification': None, u'end-one-entity-type': None, u'type': None, u'resources': {u'entity': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}, u'entity-type': {u'isExcludes': False, u'values': [u'hive_table'], u'isRecursive': False}, u'entity-classification': {u'isExcludes': False, u'values': [u'*'], u'isRecursive': False}}, u'description': u'', u'isAuditEnabled': True, u'isDenyAllElse': False, u'policyType': u'0', u'denyPolicyItems': [], u'end-two-entity': None, u'end-two-entity-type': None, u'none': [], u'end-one-entity-classification': None, u'name': u'entity_allow_all_hive_table', u'denyExceptions': [], u'policyLabels': []} {code} !image-2022-07-13-17-18-15-207.png|width=1415,height=691! For an entity type with 'None' , classification related policies can also be added when creating policy via api, which is ideally incorrect This should be blocked at policy creation level itself -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74053: RANGER-3821 : Update commons-codec version to 1.15
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74053/#review224558 --- Ship it! Ship It! - Mateen Mansoori On July 11, 2022, 8:35 a.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74053/ > --- > > (Updated July 11, 2022, 8:35 a.m.) > > > Review request for ranger, Dhaval Shah, Kirby Zhou, Madhan Neethiraj, Mateen > Mansoori, Mehul Parikh, Pradeep Agrawal, and Ramesh Mani. > > > Bugs: RANGER-3821 > https://issues.apache.org/jira/browse/RANGER-3821 > > > Repository: ranger > > > Description > --- > > Update commons-codec version to 1.15 > > > Diffs > - > > pom.xml 24ae3d884 > > > Diff: https://reviews.apache.org/r/74053/diff/1/ > > > Testing > --- > > 1. Verfied Ranger Policy and User CRUDs operations > 2. Verfied Ranger KMS key operations > > > Thanks, > > bhavik patel > >