[jira] [Commented] (RANGER-3842) Ranger Kafka test connection timeout in Kerberos environment
[ https://issues.apache.org/jira/browse/RANGER-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571925#comment-17571925 ] Andras Katona commented on RANGER-3842: --- It seems to be a configuration issue (and not a bug), I don't know how ranger should be configured, yet just tried out this test connection function providing the params (sasl.mechanism, security.protocol, bootstrap.servers, kafka.keytab and kafka.principal) and worked for me. On the other hand, you have to configure kafka broker (the kafka plugin in it) to be able to connect to ranger in {{ranger-kafka-security.xml}}. > Ranger Kafka test connection timeout in Kerberos environment > > > Key: RANGER-3842 > URL: https://issues.apache.org/jira/browse/RANGER-3842 > Project: Ranger > Issue Type: Bug > Components: plugins, Ranger >Affects Versions: 2.3.0 > Environment: linux >Reporter: kejie >Priority: Major > Attachments: image-2022-07-26-11-38-37-426.png, > image-2022-07-26-11-38-55-377.png, image-2022-07-26-11-42-13-846.png > > > Excuse me, how can I configure my Kafka to work normally? > The following is my related configuration and log information! > > ranger-admin.log > > {code:java} > 2022-07-26 02:52:13,658 [http-nio-6080-exec-9] ERROR [ServiceMgr.java:198] > ==> ServiceMgr.validateConfig Error:java.util.concurrent.TimeoutException > ... > ... > ... > 2022-07-26 00:00:28,532 [http-nio-6080-exec-5] INFO [RESTErrorUtil.java:346] > Request failed. loginId=null, logMessage=Unauthenticated access not allowed > javax.ws.rs.WebApplicationException: null{code} > > > > > This is my Kafka server.log > > {code:java} > [2022-07-26 11:27:05,359] WARN Error getting Roles. secureMode=false, > user=root (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, > serviceName=kmr_kafka > (org.apache.ranger.admin.client.RangerAdminRESTClient){code} > > > > ranger-admin ui > !image-2022-07-26-11-38-37-426.png! > !image-2022-07-26-11-42-13-846.png! > !image-2022-07-26-11-38-55-377.png! > ranger-admin install.properties kerberos config > {code:java} > # Kerberos Config - > spnego_principal=HTTP/kmr-58106bd2-gn-127160a8-master-1-1.ksc@kingsoft.com > spnego_keytab=/etc/kmr/krb5/data/keytabs/kmr.keytab > token_valid=30 > cookie_domain= > cookie_path=/ > admin_principal=hadoop/kmr-58106bd2-gn-127160a8-master-1-1.ksc@kingsoft.com > admin_keytab=/etc/kmr/krb5/data/keytabs/kmr.keytab > lookup_principal=hadoop/kmr-58106bd2-gn-127160a8-master-1-1.ksc@kingsoft.com > lookup_keytab=/etc/kmr/krb5/data/keytabs/kmr.keytab > hadoop_conf=/mnt/kmr/hadoop/1/hadoop-3.1.1/etc/hadoop {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570278#comment-17570278 ] Andras Katona commented on RANGER-3809: --- Thank you! > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation of {{authorizeByResourceType}} uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin, but it is not really efficient and it is recommended > to implement/override it in custom authorizer implementation (meaning Ranger > in our case). > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} > With [this > commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1] > {{authorizeByResourceType}} got an implementation with a basic validation > check and a (constant) denial response. It's just making > UnsupportedOperationException disappear and having an expected response for > initProducer authorization. > Until the proper implementation is done, the idempotent_write access should > be granted for the producer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation of {{authorizeByResourceType}} uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin, but it is not really efficient and it is recommended to implement/override it in custom authorizer implementation (meaning Ranger in our case). {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *authorizer implementation. * 3. The default implementation does not support any custom authorizer configs or other access *rules apart from ACLs. * * @param requestContext Request context including request resourceType, security protocol and listener name * @param op The ACL operation to check * @param resourceType The resource type to check * @return Return {@link AuthorizationResult#ALLOWED} if the caller is authorized * to perform the given ACL operation on at least one resource of the * given type. Return {@link AuthorizationResult#DENIED} otherwise. */ default AuthorizationResult authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation op, ResourceType resourceType) { {code} In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default (KAFKA-13598) and the authorization of producer initialization fails, in case the user doesn't have the deprecated idempotent_write access, as it will call the {{authorizeByResourceType}} and that calls {{acls}}. {code} public Iterable acls(AclBindingFilter filter) { logger.error("(getting) acls is not supported by Ranger for Kafka"); throw new UnsupportedOperationException("(getting) acls is not supported by Ranger for Kafka"); } {code} With [this commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1] {{authorizeByResourceType}} got an implementation with a basic validation check and a (constant) denial response. It's just making UnsupportedOperationException disappear and having an expected response for initProducer authorization. Until the proper implementation is done, the idempotent_write access should be granted for the producer. was: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation of uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in c
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation of uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *authorizer implementation. * 3. The default implementation does not support any custom authorizer configs or other access *rules apart from ACLs. * * @param requestContext Request context including request resourceType, security protocol and listener name * @param op The ACL operation to check * @param resourceType The resource type to check * @return Return {@link AuthorizationResult#ALLOWED} if the caller is authorized * to perform the given ACL operation on at least one resource of the * given type. Return {@link AuthorizationResult#DENIED} otherwise. */ default AuthorizationResult authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation op, ResourceType resourceType) { {code} In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default (KAFKA-13598) and the authorization of producer initialization fails, in case the user doesn't have the deprecated idempotent_write access, as it will call the {{authorizeByResourceType}} and that calls {{acls}}. {code} public Iterable acls(AclBindingFilter filter) { logger.error("(getting) acls is not supported by Ranger for Kafka"); throw new UnsupportedOperationException("(getting) acls is not supported by Ranger for Kafka"); } {code} With [this commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1] {{authorizeByResourceType}} got an implementation with a basic validation check and a (constant) denial response. It's just making UnsupportedOperationException disappear and having an expected response for initProducer authorization. Until the proper implementation is done, the idempotent_write access should be granted for the producer. was: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementat
[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570039#comment-17570039 ] Andras Katona edited comment on RANGER-3809 at 7/22/22 1:57 PM: Last time we haven't changed the shim, so the dummy implementation is still not called by Kafka... I'm fixing it, could you pls take a look? https://reviews.apache.org/r/74066/ This shim modification will be good for the real implementation of authorizeByResourceType method as well. was (Author: akatona): Last time we haven't changed the shim, so the dummy implementation is still not called by Kafka... https://reviews.apache.org/r/74066/ This shim modification will be good for the real implementation of authorizeByResourceType method as well. > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} > With [this > commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1] > {{authorizeByResourceType}} got an implementation with a basic validation > check and a (constant) denial response. It's just making > UnsupportedOperationException disappear and having an expected response for > initProducer authorization. > Until the proper implementation is done, the id
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570039#comment-17570039 ] Andras Katona commented on RANGER-3809: --- Last time we haven't changed the shim, so the dummy implementation is still not called by Kafka... https://reviews.apache.org/r/74066/ This shim modification will be good for the real implementation of authorizeByResourceType method as well. > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} > With [this > commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1] > {{authorizeByResourceType}} got an implementation with a basic validation > check and a (constant) denial response. It's just making > UnsupportedOperationException disappear and having an expected response for > initProducer authorization. > Until the proper implementation is done, the idempotent_write access should > be granted for the producer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *authorizer implementation. * 3. The default implementation does not support any custom authorizer configs or other access *rules apart from ACLs. * * @param requestContext Request context including request resourceType, security protocol and listener name * @param op The ACL operation to check * @param resourceType The resource type to check * @return Return {@link AuthorizationResult#ALLOWED} if the caller is authorized * to perform the given ACL operation on at least one resource of the * given type. Return {@link AuthorizationResult#DENIED} otherwise. */ default AuthorizationResult authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation op, ResourceType resourceType) { {code} In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default (KAFKA-13598) and the authorization of producer initialization fails, in case the user doesn't have the deprecated idempotent_write access, as it will call the {{authorizeByResourceType}} and that calls {{acls}}. {code} public Iterable acls(AclBindingFilter filter) { logger.error("(getting) acls is not supported by Ranger for Kafka"); throw new UnsupportedOperationException("(getting) acls is not supported by Ranger for Kafka"); } {code} With [this commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1] {{authorizeByResourceType}} got an implementation with a basic validation check and a (constant) denial response. It's just making UnsupportedOperationException disappear and having an expected response for initProducer authorization. Until the proper implementation is done, the idempotent_write access should be granted for the producer. was: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566679#comment-17566679 ] Andras Katona commented on RANGER-3809: --- Sorry for reopening, but the implementation should be done under this ticket. Viktor put this ticket number into the code: https://github.com/apache/ranger/blob/3ce8112e723e65c93de092f6a14a418383abc8d0/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332 What changed is that the acls() operation implementation is no longer an option, adjusting the description. > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Fix Version/s: (was: 3.0.0) > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona reopened RANGER-3809: --- > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566428#comment-17566428 ] Andras Katona commented on RANGER-3809: --- Could we keep this ticket open, as for the proper authorizeByResourceType implementation too? We just made the plugin less ugly.. :) > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 3.0.0 >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407 ] Andras Katona edited comment on RANGER-3809 at 7/13/22 4:34 PM: First we'll do a dummy impl for the {{authorizeByResourceType }} method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ was (Author: akatona): First we'll do a dummy impl for the method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407 ] Andras Katona edited comment on RANGER-3809 at 7/13/22 4:34 PM: First we'll do a dummy impl for the {{authorizeByResourceType}} method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ was (Author: akatona): First we'll do a dummy impl for the {{authorizeByResourceType }} method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407 ] Andras Katona commented on RANGER-3809: --- First we'll do a dummy impl for the method and asking to still use deprecated idempotent_write cluster operation/right. https://reviews.apache.org/r/74056/ > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona reassigned RANGER-3809: - Assignee: Ramesh Mani (was: Andras Katona) > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Ramesh Mani >Priority: Major > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. > The default implementation uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one > * resource of the given type. > * > * Custom authorizer implementations should consider overriding this > default implementation because: > * 1. The default implementation iterates all AclBindings multiple times, > without any caching > *by principal, host, operation, permission types, and resource > types. More efficient > *implementations may be added in custom authorizers that directly > access cached entries. > * 2. The default implementation cannot integrate with any audit logging > included in the > *authorizer implementation. > * 3. The default implementation does not support any custom authorizer > configs or other access > *rules apart from ACLs. > * > * @param requestContext Request context including request resourceType, > security protocol and listener name > * @param op The ACL operation to check > * @param resourceType The resource type to check > * @return Return {@link AuthorizationResult#ALLOWED} if > the caller is authorized > * to perform the given ACL operation on at least > one resource of the > * given type. Return {@link > AuthorizationResult#DENIED} otherwise. > */ > default AuthorizationResult > authorizeByResourceType(AuthorizableRequestContext requestContext, > AclOperation op, ResourceType resourceType) { > {code} > In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default > (KAFKA-13598) and the authorization of producer initialization fails, in case > the user doesn't have the deprecated idempotent_write access, as it will call > the {{authorizeByResourceType}} and that calls {{acls}}. > {code} > public Iterable acls(AclBindingFilter filter) { > logger.error("(getting) acls is not supported by Ranger for Kafka"); > throw new UnsupportedOperationException("(getting) acls is not supported > by Ranger for Kafka"); > } > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3790) Ranger tagsync module should not depend on kafka server
[ https://issues.apache.org/jira/browse/RANGER-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3790: -- Fix Version/s: (was: 3.0.0) > Ranger tagsync module should not depend on kafka server > --- > > Key: RANGER-3790 > URL: https://issues.apache.org/jira/browse/RANGER-3790 > Project: Ranger > Issue Type: Bug > Components: tagsync >Affects Versions: 3.0.0 >Reporter: Patrik Márton >Assignee: Patrik Márton >Priority: Major > Attachments: RANGER-3790.patch > > Time Spent: 10m > Remaining Estimate: 0h > > This is a follow-up of RANGER-2426. > Kafka Core dependency has already been removed from tagsync as part of the > mentioned ticket, but since the _distro/src/main/assembly/tagsync.xml_ > contains this dependency, it is included in the distribution. > [https://github.com/apache/ranger/blob/master/distro/src/main/assembly/tagsync.xml#L56] > > The goal is to remove this from the assembly. > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin was:In Kafka 2.8 this new authorization method was introduced mainly to ease authorization setup of idempotent producers. > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Andras Katona >Priority: Major > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. The default implementation > uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *authorizer implementation. * 3. The default implementation does not support any custom authorizer configs or other access *rules apart from ACLs. * * @param requestContext Request context including request resourceType, security protocol and listener name * @param op The ACL operation to check * @param resourceType The resource type to check * @return Return {@link AuthorizationResult#ALLOWED} if the caller is authorized * to perform the given ACL operation on at least one resource of the * given type. Return {@link AuthorizationResult#DENIED} otherwise. */ default AuthorizationResult authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation op, ResourceType resourceType) { {code} In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default (KAFKA-13598) and the authorization of producer initialization fails, in case the user doesn't have the deprecated idempotent_write access, as it will call the {{authorizeByResourceType}} and that calls {{acls}}. {code} public Iterable acls(AclBindingFilter filter) { logger.error("(getting) acls is not supported by Ranger for Kafka"); throw new UnsupportedOperationException("(getting) acls is not supported by Ranger for Kafka"); } {code} was: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *authorizer implementation. * 3. The default implementation does not support any custom authorizer configs or other access *rules apart from ACLs. * * @param requestContext Request context including request resourceType, security protocol and listener name * @param op The ACL operation to check * @param resourceType The resource type to check * @return Return {@link AuthorizationResult#ALLOWED} if the caller is authorized * to perform the given ACL operation on at least one resource of the * given type. Return {@link AuthorizationResult#DENIED} otherwise. */ default AuthorizationResult authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation op, ResourceType resourceType) { {code} In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default (KAFKA-13598) and the authorization of producer initialization fails, in case the user doesn't have the idempotent_write access, as it will call the {{authorizeByResourceType}} and that calls {{acls}}. {code} public Iterable acls(AclBindingFilter filter) { logger.error("(getting) acls is not supported by Ranger for Kafka"); throw new UnsupportedOperationException("(getting) acls is not supported by Ranger for Kafka"); } {code} was: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin {code} /** * Returns ACL bindings which match the provided filter. * * This is a synchronous API designed for use with locally cached ACLs. This method is invoked on the request * thread while processing DescribeAcls requests and should avoid time-consuming remote communication that may * block request threads. * * @return Iterator for ACL bindings, which may be populated lazily. */ Iterable acls(AclBindingFilter filter); {code} {code} /** * Check if the caller is authorized to perform the given ACL operation on at least one * resource of the given type. * * Custom authorizer implementations should consider overriding this default implementation because: * 1. The default implementation iterates all AclBindings multiple times, without any caching *by principal, host, operation, permission types, and resource types. More efficient *implementations may be added in custom authorizers that directly access cached entries. * 2. The default implementation cannot integrate with any audit logging included in the *authorizer implementation. * 3. The default implementation does not support any custom authorizer configs or other access *rules apart from ACLs. * * @param requestContext Request context including request resourceType, security protocol and listener name * @param op The ACL operation to check * @param resourceType The resource type to check * @return Return {@link AuthorizationResult#ALLOWED} if the caller is authorized * to perform the given ACL operation on at least one resource of the * given type. Return {@link AuthorizationResult#DENIED} otherwise. */ default AuthorizationResult authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation op, ResourceType resourceType) { {code} was: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization (setup) of idempotent producers. The default implementation uses [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] which is [not implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] in Kafka Ranger Plugin > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Andras Katona >Priority: Major > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization (setup) of idempotent producers. The default implementation > uses > [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154] > which is [not > implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335] > in Kafka Ranger Plugin > {code} > /** > * Returns ACL bindings which match the provided filter. > * > * This is a synchronous API designed for use with locally cached ACLs. > This method is invoked on the request > * thread while processing DescribeAcls requests and should avoid > time-consuming remote communication that may > * block request threads. > * > * @return Iterator for ACL bindings, which may be populated lazily. > */ > Iterable acls(AclBindingFilter filter); > {code} > {code} > /** > * Check if the caller is authorized to perform the given ACL operation > on at least one >
[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
[ https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3809: -- Description: In Kafka 2.8 this new authorization method was introduced mainly to ease authorization setup of idempotent producers. > Implement authorizeByResourceType method of Kafka Authorizer > > > Key: RANGER-3809 > URL: https://issues.apache.org/jira/browse/RANGER-3809 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Andras Katona >Assignee: Andras Katona >Priority: Major > > In Kafka 2.8 this new authorization method was introduced mainly to ease > authorization setup of idempotent producers. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer
Andras Katona created RANGER-3809: - Summary: Implement authorizeByResourceType method of Kafka Authorizer Key: RANGER-3809 URL: https://issues.apache.org/jira/browse/RANGER-3809 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Andras Katona Assignee: Andras Katona -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527879#comment-17527879 ] Andras Katona commented on RANGER-3231: --- Yes, that unsupported operation exception is intentional. We thought it'd be better to let know the tools what is not supported rather than giving empty response. > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Fix For: 3.0.0 > > Attachments: ranger_kafka.zip > > Time Spent: 2h > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3231: -- Fix Version/s: 3.0.0 > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Fix For: 3.0.0 > > Attachments: ranger_kafka.zip > > Time Spent: 1h 40m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515759#comment-17515759 ] Andras Katona commented on RANGER-3231: --- I think it's not related to this change, it'd happen without my patch too. Would you mind creating a ticket for this error? There you can attach the logs. Thanks for the testing and your findings! > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Attachments: ranger_kafka.zip > > Time Spent: 1h 40m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514468#comment-17514468 ] Andras Katona commented on RANGER-3231: --- I checked that other kafka clients are also sending authz requests with empty actions sometimes, not just the cruisecontrol metrics reporter. It's perfectly normal. I've already updated the PR, handled the scenario. > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Attachments: ranger_kafka.zip > > Time Spent: 1h 40m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512677#comment-17512677 ] Andras Katona commented on RANGER-3231: --- Yup, super helpful, thanks! It seems authorization request comes in with empty actions, coming from cruise control metrics reporter. I'll modify my patch to ignore these and answer an empty response, and also will check why this kind of authorization calls happen. > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Attachments: ranger_kafka.zip > > Time Spent: 1h 40m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511860#comment-17511860 ] Andras Katona commented on RANGER-3231: --- I think something is wrong with your setup, it's hard to tell from these snippets. Yet, if you'd enable debug logging, it would log the ranger request sent to ranger and the answer (which will be denied in our case). And ranger side logs would help you too. I've tried this patch with 2.8.1 kafka and 3.1.0 kafka too and works for me. > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508020#comment-17508020 ] Andras Katona commented on RANGER-3231: --- That's strange, could you download again the patch and recompile and try again? I've improved the logging a bit, if you could attach the logs, I can help investigate. > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/RANGER-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-3459: -- Fix Version/s: 3.0.0 > Upgrade Ranger's Kafka dependency to 2.8 > > > Key: RANGER-3459 > URL: https://issues.apache.org/jira/browse/RANGER-3459 > Project: Ranger > Issue Type: Improvement > Components: plugins, Ranger >Reporter: Andras Katona >Assignee: Andras Katona >Priority: Major > Fix For: 3.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500088#comment-17500088 ] Andras Katona commented on RANGER-3231: --- I've prepared a reviewboard request: https://reviews.apache.org/r/73880/, which is practically the same as the above mentioned PR (https://github.com/apache/ranger/pull/133). It differs from the previous patch: * start method is implemented too as [AclAuthorizer does in Kafka|https://github.com/apache/kafka/blob/f089bea7ed9fee2bad7084e6b40561ff6935ce5f/core/src/main/scala/kafka/security/authorizer/AclAuthorizer.scala#L194-L195] * unsupported methods in Ranger-Kafka plugin are throwing/wrapping UnsupportedOperationException instead of returning empty * calling the base-ranger plugin isAccessAllowed method with a list of requests instead of one-by-one - to support bulk-like authorization better > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Chia-Ping Tsai >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona reassigned RANGER-3231: - Assignee: Andras Katona (was: Chia-Ping Tsai) > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Andras Katona >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504
[ https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498900#comment-17498900 ] Andras Katona commented on RANGER-3231: --- I created a PR to test my solution: https://github.com/apache/ranger/pull/133 I'm going to create a new reviewboard request from it once I have it reviewed by a friend first. [~chia7712], I've added you as co-author of my patch as you started to work on it and used some lines from your patch. > Ranger should use kafka Authorizer from KIP-504 > --- > > Key: RANGER-3231 > URL: https://issues.apache.org/jira/browse/RANGER-3231 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: Ismael Juma >Assignee: Chia-Ping Tsai >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is > an improvement over `kafka.security.auth.Authorizer` and it's a pure Java > interface (instead of Scala). > `kafka.security.auth.Authorizer` has been deprecated since December 2019 and > it will be removed in Apache Kafka 3.0 (roughly planned for July/August). > See the KIP for more details: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/RANGER-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona reassigned RANGER-3459: - Assignee: Andras Katona > Upgrade Ranger's Kafka dependency to 2.8 > > > Key: RANGER-3459 > URL: https://issues.apache.org/jira/browse/RANGER-3459 > Project: Ranger > Issue Type: Improvement > Components: plugins, Ranger >Reporter: Andras Katona >Assignee: Andras Katona >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/RANGER-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498764#comment-17498764 ] Andras Katona commented on RANGER-3459: --- [~rmani], please find the patch below. PR: https://github.com/apache/ranger/pull/134 ReviewBoard: https://reviews.apache.org/r/73877/ > Upgrade Ranger's Kafka dependency to 2.8 > > > Key: RANGER-3459 > URL: https://issues.apache.org/jira/browse/RANGER-3459 > Project: Ranger > Issue Type: Improvement > Components: plugins, Ranger >Reporter: Andras Katona >Assignee: Andras Katona >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-2426) ranger-plugins-audits should depend on kafka-clients not kafka server
[ https://issues.apache.org/jira/browse/RANGER-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497467#comment-17497467 ] Andras Katona commented on RANGER-2426: --- PR: https://github.com/apache/ranger/pull/132 Reviewboard request: https://reviews.apache.org/r/73866/ > ranger-plugins-audits should depend on kafka-clients not kafka server > - > > Key: RANGER-2426 > URL: https://issues.apache.org/jira/browse/RANGER-2426 > Project: Ranger > Issue Type: Bug > Components: audit >Reporter: Fredy Wijaya >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > https://github.com/apache/ranger/blob/f2d310a54ee64a1738a9c1a6afdd93d032e64ddf/agents-audit/pom.xml#L72 > To reduce the transitive dependency, ranger-plugins-audit should depend on > kafka-clients. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8
Andras Katona created RANGER-3459: - Summary: Upgrade Ranger's Kafka dependency to 2.8 Key: RANGER-3459 URL: https://issues.apache.org/jira/browse/RANGER-3459 Project: Ranger Issue Type: Improvement Components: plugins, Ranger Reporter: Andras Katona -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (RANGER-3096) Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger
[ https://issues.apache.org/jira/browse/RANGER-3096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241661#comment-17241661 ] Andras Katona edited comment on RANGER-3096 at 12/1/20, 4:12 PM: - I've uploaded a patch for this: https://reviews.apache.org/r/73054/ was (Author: akatona): https://reviews.apache.org/r/73054/ > Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger > --- > > Key: RANGER-3096 > URL: https://issues.apache.org/jira/browse/RANGER-3096 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Andras Katona >Priority: Major > > Kafka is removing kafka_2.11 artifact from 2.5.0. > Upgrading kafka version requires another Kafka core artifact to be pulled in. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (RANGER-3096) Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger
Andras Katona created RANGER-3096: - Summary: Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger Key: RANGER-3096 URL: https://issues.apache.org/jira/browse/RANGER-3096 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Andras Katona Kafka is removing kafka_2.11 artifact from 2.5.0. Upgrading kafka version requires another Kafka core artifact to be pulled in. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077166#comment-17077166 ] Andras Katona commented on RANGER-2702: --- This ticket upgraded kafka dependency to 2.4.0, not to kafka 2.0.0 > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0, 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Labels: security > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-2702: -- Attachment: (was: 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch) > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024971#comment-17024971 ] Andras Katona commented on RANGER-2702: --- Ok. Thanks! > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated RANGER-2702: -- Attachment: 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Fix For: 2.1.0 > > Attachments: > 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4
[ https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021005#comment-17021005 ] Andras Katona commented on RANGER-2702: --- [~rmani], uploaded my patch to here: https://reviews.apache.org/r/72037/ > Upgrade Kafka Version in Ranger to 2.4 > -- > > Key: RANGER-2702 > URL: https://issues.apache.org/jira/browse/RANGER-2702 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Ramesh Mani >Assignee: Ramesh Mani >Priority: Major > Fix For: 2.1.0 > > > Upgrade Kafka Version in Ranger to 2.4. > -- This message was sent by Atlassian Jira (v8.3.4#803005)