[jira] [Commented] (RANGER-3842) Ranger Kafka test connection timeout in Kerberos environment

2022-07-27 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571925#comment-17571925
 ] 

Andras Katona commented on RANGER-3842:
---

It seems to be a configuration issue (and not a bug), I don't know how ranger 
should be configured, yet just tried out this test connection function 
providing the params (sasl.mechanism, security.protocol, bootstrap.servers, 
kafka.keytab and kafka.principal) and worked for me.
On the other hand, you have to configure kafka broker (the kafka plugin in it) 
to be able to connect to ranger in {{ranger-kafka-security.xml}}.

> Ranger Kafka test connection timeout in Kerberos environment
> 
>
> Key: RANGER-3842
> URL: https://issues.apache.org/jira/browse/RANGER-3842
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins, Ranger
>Affects Versions: 2.3.0
> Environment: linux 
>Reporter: kejie
>Priority: Major
> Attachments: image-2022-07-26-11-38-37-426.png, 
> image-2022-07-26-11-38-55-377.png, image-2022-07-26-11-42-13-846.png
>
>
> Excuse me, how can I configure my Kafka to work normally?
> The following is my related configuration and log information! 
>  
> ranger-admin.log
>  
> {code:java}
> 2022-07-26 02:52:13,658 [http-nio-6080-exec-9] ERROR [ServiceMgr.java:198] 
> ==> ServiceMgr.validateConfig Error:java.util.concurrent.TimeoutException
> ...
> ...
> ...
> 2022-07-26 00:00:28,532 [http-nio-6080-exec-5] INFO [RESTErrorUtil.java:346] 
> Request failed. loginId=null, logMessage=Unauthenticated access not allowed
> javax.ws.rs.WebApplicationException: null{code}
>  
>  
>  
>  
> This is my Kafka server.log 
>  
> {code:java}
> [2022-07-26 11:27:05,359] WARN Error getting Roles. secureMode=false, 
> user=root (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, 
> serviceName=kmr_kafka 
> (org.apache.ranger.admin.client.RangerAdminRESTClient){code}
>  
>  
>  
> ranger-admin ui
> !image-2022-07-26-11-38-37-426.png!
> !image-2022-07-26-11-42-13-846.png!
> !image-2022-07-26-11-38-55-377.png!
> ranger-admin  install.properties kerberos config
> {code:java}
> # Kerberos Config -
> spnego_principal=HTTP/kmr-58106bd2-gn-127160a8-master-1-1.ksc@kingsoft.com
> spnego_keytab=/etc/kmr/krb5/data/keytabs/kmr.keytab
> token_valid=30
> cookie_domain=
> cookie_path=/
> admin_principal=hadoop/kmr-58106bd2-gn-127160a8-master-1-1.ksc@kingsoft.com
> admin_keytab=/etc/kmr/krb5/data/keytabs/kmr.keytab
> lookup_principal=hadoop/kmr-58106bd2-gn-127160a8-master-1-1.ksc@kingsoft.com
> lookup_keytab=/etc/kmr/krb5/data/keytabs/kmr.keytab
> hadoop_conf=/mnt/kmr/hadoop/1/hadoop-3.1.1/etc/hadoop {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-23 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570278#comment-17570278
 ] 

Andras Katona commented on RANGER-3809:
---

Thank you!

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation of {{authorizeByResourceType}} uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin, but it is not really efficient and it is recommended 
> to implement/override it in custom authorizer implementation (meaning Ranger 
> in our case).
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}
> With [this 
> commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1]
>  {{authorizeByResourceType}} got an implementation with a basic validation 
> check and a (constant) denial response. It's just making 
> UnsupportedOperationException disappear and having an expected response for 
> initProducer authorization.
> Until the proper implementation is done, the idempotent_write access should 
> be granted for the producer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-22 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation of {{authorizeByResourceType}} uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin, but it is not really efficient and it is recommended 
to implement/override it in custom authorizer implementation (meaning Ranger in 
our case).

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 *authorizer implementation.
 * 3. The default implementation does not support any custom authorizer 
configs or other access
 *rules apart from ACLs.
 *
 * @param requestContext Request context including request resourceType, 
security protocol and listener name
 * @param op The ACL operation to check
 * @param resourceType   The resource type to check
 * @return   Return {@link AuthorizationResult#ALLOWED} if the 
caller is authorized
 *   to perform the given ACL operation on at least one 
resource of the
 *   given type. Return {@link 
AuthorizationResult#DENIED} otherwise.
 */
default AuthorizationResult 
authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation 
op, ResourceType resourceType) {
{code}

In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
(KAFKA-13598) and the authorization of producer initialization fails, in case 
the user doesn't have the deprecated idempotent_write access, as it will call 
the {{authorizeByResourceType}} and that calls {{acls}}.

{code}
  public Iterable acls(AclBindingFilter filter) {
logger.error("(getting) acls is not supported by Ranger for Kafka");
throw new UnsupportedOperationException("(getting) acls is not supported by 
Ranger for Kafka");
  }
{code}

With [this 
commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1]
 {{authorizeByResourceType}} got an implementation with a basic validation 
check and a (constant) denial response. It's just making 
UnsupportedOperationException disappear and having an expected response for 
initProducer authorization.

Until the proper implementation is done, the idempotent_write access should be 
granted for the producer.

  was:
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation of  uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in c

[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-22 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation of  uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 *authorizer implementation.
 * 3. The default implementation does not support any custom authorizer 
configs or other access
 *rules apart from ACLs.
 *
 * @param requestContext Request context including request resourceType, 
security protocol and listener name
 * @param op The ACL operation to check
 * @param resourceType   The resource type to check
 * @return   Return {@link AuthorizationResult#ALLOWED} if the 
caller is authorized
 *   to perform the given ACL operation on at least one 
resource of the
 *   given type. Return {@link 
AuthorizationResult#DENIED} otherwise.
 */
default AuthorizationResult 
authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation 
op, ResourceType resourceType) {
{code}

In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
(KAFKA-13598) and the authorization of producer initialization fails, in case 
the user doesn't have the deprecated idempotent_write access, as it will call 
the {{authorizeByResourceType}} and that calls {{acls}}.

{code}
  public Iterable acls(AclBindingFilter filter) {
logger.error("(getting) acls is not supported by Ranger for Kafka");
throw new UnsupportedOperationException("(getting) acls is not supported by 
Ranger for Kafka");
  }
{code}

With [this 
commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1]
 {{authorizeByResourceType}} got an implementation with a basic validation 
check and a (constant) denial response. It's just making 
UnsupportedOperationException disappear and having an expected response for 
initProducer authorization.

Until the proper implementation is done, the idempotent_write access should be 
granted for the producer.

  was:
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementat

[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-22 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570039#comment-17570039
 ] 

Andras Katona edited comment on RANGER-3809 at 7/22/22 1:57 PM:


Last time we haven't changed the shim, so the dummy implementation is still not 
called by Kafka... 
I'm fixing it, could you pls take a look?
https://reviews.apache.org/r/74066/

This shim modification will be good for the real implementation of 
authorizeByResourceType method as well.


was (Author: akatona):
Last time we haven't changed the shim, so the dummy implementation is still not 
called by Kafka... 
https://reviews.apache.org/r/74066/

This shim modification will be good for the real implementation of 
authorizeByResourceType method as well.

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}
> With [this 
> commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1]
>  {{authorizeByResourceType}} got an implementation with a basic validation 
> check and a (constant) denial response. It's just making 
> UnsupportedOperationException disappear and having an expected response for 
> initProducer authorization.
> Until the proper implementation is done, the id

[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-22 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570039#comment-17570039
 ] 

Andras Katona commented on RANGER-3809:
---

Last time we haven't changed the shim, so the dummy implementation is still not 
called by Kafka... 
https://reviews.apache.org/r/74066/

This shim modification will be good for the real implementation of 
authorizeByResourceType method as well.

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}
> With [this 
> commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1]
>  {{authorizeByResourceType}} got an implementation with a basic validation 
> check and a (constant) denial response. It's just making 
> UnsupportedOperationException disappear and having an expected response for 
> initProducer authorization.
> Until the proper implementation is done, the idempotent_write access should 
> be granted for the producer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-14 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 *authorizer implementation.
 * 3. The default implementation does not support any custom authorizer 
configs or other access
 *rules apart from ACLs.
 *
 * @param requestContext Request context including request resourceType, 
security protocol and listener name
 * @param op The ACL operation to check
 * @param resourceType   The resource type to check
 * @return   Return {@link AuthorizationResult#ALLOWED} if the 
caller is authorized
 *   to perform the given ACL operation on at least one 
resource of the
 *   given type. Return {@link 
AuthorizationResult#DENIED} otherwise.
 */
default AuthorizationResult 
authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation 
op, ResourceType resourceType) {
{code}

In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
(KAFKA-13598) and the authorization of producer initialization fails, in case 
the user doesn't have the deprecated idempotent_write access, as it will call 
the {{authorizeByResourceType}} and that calls {{acls}}.

{code}
  public Iterable acls(AclBindingFilter filter) {
logger.error("(getting) acls is not supported by Ranger for Kafka");
throw new UnsupportedOperationException("(getting) acls is not supported by 
Ranger for Kafka");
  }
{code}

With [this 
commit|https://github.com/apache/ranger/commit/0ec279474e0439f6c5e7d4497db191fb7cc99bc1]
 {{authorizeByResourceType}} got an implementation with a basic validation 
check and a (constant) denial response. It's just making 
UnsupportedOperationException disappear and having an expected response for 
initProducer authorization.

Until the proper implementation is done, the idempotent_write access should be 
granted for the producer.

  was:
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations

[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-14 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566679#comment-17566679
 ] 

Andras Katona commented on RANGER-3809:
---

Sorry for reopening, but the implementation should be done under this ticket. 
Viktor put this ticket number into the code:
https://github.com/apache/ranger/blob/3ce8112e723e65c93de092f6a14a418383abc8d0/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332
What changed is that the acls() operation implementation is no longer an 
option, adjusting  the description.

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-14 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Fix Version/s: (was: 3.0.0)

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Reopened] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-14 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona reopened RANGER-3809:
---

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 3.0.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-13 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566428#comment-17566428
 ] 

Andras Katona commented on RANGER-3809:
---

Could we keep this ticket open, as for the proper authorizeByResourceType 
implementation too? We just made the plugin less ugly.. :)

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 3.0.0
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 3.0.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-13 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407
 ] 

Andras Katona edited comment on RANGER-3809 at 7/13/22 4:34 PM:


First we'll do a dummy impl for the {{authorizeByResourceType }} method and 
asking to still use deprecated idempotent_write cluster operation/right.
https://reviews.apache.org/r/74056/


was (Author: akatona):
First we'll do a dummy impl for the method and asking to still use deprecated 
idempotent_write cluster operation/right.
https://reviews.apache.org/r/74056/

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-13 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407
 ] 

Andras Katona edited comment on RANGER-3809 at 7/13/22 4:34 PM:


First we'll do a dummy impl for the {{authorizeByResourceType}} method and 
asking to still use deprecated idempotent_write cluster operation/right.
https://reviews.apache.org/r/74056/


was (Author: akatona):
First we'll do a dummy impl for the {{authorizeByResourceType }} method and 
asking to still use deprecated idempotent_write cluster operation/right.
https://reviews.apache.org/r/74056/

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-07-13 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566407#comment-17566407
 ] 

Andras Katona commented on RANGER-3809:
---

First we'll do a dummy impl for the method and asking to still use deprecated 
idempotent_write cluster operation/right.
https://reviews.apache.org/r/74056/

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-30 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona reassigned RANGER-3809:
-

Assignee: Ramesh Mani  (was: Andras Katona)

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Ramesh Mani
>Priority: Major
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers.
> The default implementation uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
>  * resource of the given type.
>  *
>  * Custom authorizer implementations should consider overriding this 
> default implementation because:
>  * 1. The default implementation iterates all AclBindings multiple times, 
> without any caching
>  *by principal, host, operation, permission types, and resource 
> types. More efficient
>  *implementations may be added in custom authorizers that directly 
> access cached entries.
>  * 2. The default implementation cannot integrate with any audit logging 
> included in the
>  *authorizer implementation.
>  * 3. The default implementation does not support any custom authorizer 
> configs or other access
>  *rules apart from ACLs.
>  *
>  * @param requestContext Request context including request resourceType, 
> security protocol and listener name
>  * @param op The ACL operation to check
>  * @param resourceType   The resource type to check
>  * @return   Return {@link AuthorizationResult#ALLOWED} if 
> the caller is authorized
>  *   to perform the given ACL operation on at least 
> one resource of the
>  *   given type. Return {@link 
> AuthorizationResult#DENIED} otherwise.
>  */
> default AuthorizationResult 
> authorizeByResourceType(AuthorizableRequestContext requestContext, 
> AclOperation op, ResourceType resourceType) {
> {code}
> In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
> (KAFKA-13598) and the authorization of producer initialization fails, in case 
> the user doesn't have the deprecated idempotent_write access, as it will call 
> the {{authorizeByResourceType}} and that calls {{acls}}.
> {code}
>   public Iterable acls(AclBindingFilter filter) {
> logger.error("(getting) acls is not supported by Ranger for Kafka");
> throw new UnsupportedOperationException("(getting) acls is not supported 
> by Ranger for Kafka");
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (RANGER-3790) Ranger tagsync module should not depend on kafka server

2022-06-27 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3790:
--
Fix Version/s: (was: 3.0.0)

> Ranger tagsync module should not depend on kafka server
> ---
>
> Key: RANGER-3790
> URL: https://issues.apache.org/jira/browse/RANGER-3790
> Project: Ranger
>  Issue Type: Bug
>  Components: tagsync
>Affects Versions: 3.0.0
>Reporter: Patrik Márton
>Assignee: Patrik Márton
>Priority: Major
> Attachments: RANGER-3790.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This is a follow-up of RANGER-2426.
> Kafka Core dependency has already been removed from tagsync as part of the 
> mentioned ticket, but since the _distro/src/main/assembly/tagsync.xml_ 
> contains this dependency, it is included in the distribution. 
> [https://github.com/apache/ranger/blob/master/distro/src/main/assembly/tagsync.xml#L56]
>  
> The goal is to remove this from the assembly.
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-27 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers. The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin



  was:In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization setup of idempotent producers.


> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Andras Katona
>Priority: Major
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers. The default implementation 
> uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-27 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 *authorizer implementation.
 * 3. The default implementation does not support any custom authorizer 
configs or other access
 *rules apart from ACLs.
 *
 * @param requestContext Request context including request resourceType, 
security protocol and listener name
 * @param op The ACL operation to check
 * @param resourceType   The resource type to check
 * @return   Return {@link AuthorizationResult#ALLOWED} if the 
caller is authorized
 *   to perform the given ACL operation on at least one 
resource of the
 *   given type. Return {@link 
AuthorizationResult#DENIED} otherwise.
 */
default AuthorizationResult 
authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation 
op, ResourceType resourceType) {
{code}

In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
(KAFKA-13598) and the authorization of producer initialization fails, in case 
the user doesn't have the deprecated idempotent_write access, as it will call 
the {{authorizeByResourceType}} and that calls {{acls}}.

{code}
  public Iterable acls(AclBindingFilter filter) {
logger.error("(getting) acls is not supported by Ranger for Kafka");
throw new UnsupportedOperationException("(getting) acls is not supported by 
Ranger for Kafka");
  }
{code}

  was:
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in

[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-27 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers.

The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 *authorizer implementation.
 * 3. The default implementation does not support any custom authorizer 
configs or other access
 *rules apart from ACLs.
 *
 * @param requestContext Request context including request resourceType, 
security protocol and listener name
 * @param op The ACL operation to check
 * @param resourceType   The resource type to check
 * @return   Return {@link AuthorizationResult#ALLOWED} if the 
caller is authorized
 *   to perform the given ACL operation on at least one 
resource of the
 *   given type. Return {@link 
AuthorizationResult#DENIED} otherwise.
 */
default AuthorizationResult 
authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation 
op, ResourceType resourceType) {
{code}

In Kafka 3.0.1, 3.1.1 and 3.2.0, producers are idempotent by default 
(KAFKA-13598) and the authorization of producer initialization fails, in case 
the user doesn't have the idempotent_write access, as it will call the 
{{authorizeByResourceType}} and that calls {{acls}}.

{code}
  public Iterable acls(AclBindingFilter filter) {
logger.error("(getting) acls is not supported by Ranger for Kafka");
throw new UnsupportedOperationException("(getting) acls is not supported by 
Ranger for Kafka");
  }
{code}

  was:
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers. The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 * 

[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-27 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: 
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers. The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin

{code}
/**
 * Returns ACL bindings which match the provided filter.
 * 
 * This is a synchronous API designed for use with locally cached ACLs. 
This method is invoked on the request
 * thread while processing DescribeAcls requests and should avoid 
time-consuming remote communication that may
 * block request threads.
 *
 * @return Iterator for ACL bindings, which may be populated lazily.
 */
Iterable acls(AclBindingFilter filter);
{code}

{code}
/**
 * Check if the caller is authorized to perform the given ACL operation on 
at least one
 * resource of the given type.
 *
 * Custom authorizer implementations should consider overriding this 
default implementation because:
 * 1. The default implementation iterates all AclBindings multiple times, 
without any caching
 *by principal, host, operation, permission types, and resource types. 
More efficient
 *implementations may be added in custom authorizers that directly 
access cached entries.
 * 2. The default implementation cannot integrate with any audit logging 
included in the
 *authorizer implementation.
 * 3. The default implementation does not support any custom authorizer 
configs or other access
 *rules apart from ACLs.
 *
 * @param requestContext Request context including request resourceType, 
security protocol and listener name
 * @param op The ACL operation to check
 * @param resourceType   The resource type to check
 * @return   Return {@link AuthorizationResult#ALLOWED} if the 
caller is authorized
 *   to perform the given ACL operation on at least one 
resource of the
 *   given type. Return {@link 
AuthorizationResult#DENIED} otherwise.
 */
default AuthorizationResult 
authorizeByResourceType(AuthorizableRequestContext requestContext, AclOperation 
op, ResourceType resourceType) {
{code}

  was:
In Kafka 2.8 this new authorization method was introduced mainly to ease 
authorization (setup) of idempotent producers. The default implementation uses 
[acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
  which is [not 
implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
 in Kafka Ranger Plugin




> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Andras Katona
>Priority: Major
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization (setup) of idempotent producers. The default implementation 
> uses 
> [acls()|https://github.com/apache/kafka/blob/a3c7017ff7e543b50f84110195690a253f19d9cf/clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java#L154]
>   which is [not 
> implemented|https://github.com/apache/ranger/blob/fc7ad98fbb2ee7bb7d4cd3329abc438a73e0444a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L332-L335]
>  in Kafka Ranger Plugin
> {code}
> /**
>  * Returns ACL bindings which match the provided filter.
>  * 
>  * This is a synchronous API designed for use with locally cached ACLs. 
> This method is invoked on the request
>  * thread while processing DescribeAcls requests and should avoid 
> time-consuming remote communication that may
>  * block request threads.
>  *
>  * @return Iterator for ACL bindings, which may be populated lazily.
>  */
> Iterable acls(AclBindingFilter filter);
> {code}
> {code}
> /**
>  * Check if the caller is authorized to perform the given ACL operation 
> on at least one
> 

[jira] [Updated] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-27 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3809:
--
Description: In Kafka 2.8 this new authorization method was introduced 
mainly to ease authorization setup of idempotent producers.

> Implement authorizeByResourceType method of Kafka Authorizer
> 
>
> Key: RANGER-3809
> URL: https://issues.apache.org/jira/browse/RANGER-3809
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andras Katona
>Assignee: Andras Katona
>Priority: Major
>
> In Kafka 2.8 this new authorization method was introduced mainly to ease 
> authorization setup of idempotent producers.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Created] (RANGER-3809) Implement authorizeByResourceType method of Kafka Authorizer

2022-06-27 Thread Andras Katona (Jira)
Andras Katona created RANGER-3809:
-

 Summary: Implement authorizeByResourceType method of Kafka 
Authorizer
 Key: RANGER-3809
 URL: https://issues.apache.org/jira/browse/RANGER-3809
 Project: Ranger
  Issue Type: Improvement
  Components: plugins
Reporter: Andras Katona
Assignee: Andras Katona






--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-04-25 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527879#comment-17527879
 ] 

Andras Katona commented on RANGER-3231:
---

Yes, that unsupported operation exception is intentional. We thought it'd be 
better to let know the tools what is not supported rather than giving empty 
response.

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ranger_kafka.zip
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-04-06 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3231:
--
Fix Version/s: 3.0.0

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ranger_kafka.zip
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-04-01 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515759#comment-17515759
 ] 

Andras Katona commented on RANGER-3231:
---

I think it's not related to this change, it'd happen without my patch too. 
Would you mind creating a ticket for this error? There you can attach the logs.
Thanks for the testing and your findings! 

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
> Attachments: ranger_kafka.zip
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-03-29 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514468#comment-17514468
 ] 

Andras Katona commented on RANGER-3231:
---

I checked that other kafka clients are also sending authz requests with empty 
actions sometimes, not just the cruisecontrol metrics reporter. It's perfectly 
normal. I've already updated the PR, handled the scenario.


> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
> Attachments: ranger_kafka.zip
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-03-25 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512677#comment-17512677
 ] 

Andras Katona commented on RANGER-3231:
---

Yup, super helpful, thanks! It seems authorization request comes in with empty 
actions, coming from cruise control metrics reporter. I'll modify my patch to 
ignore these and answer an empty response, and also will check why this kind of 
authorization calls happen.

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
> Attachments: ranger_kafka.zip
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-03-24 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511860#comment-17511860
 ] 

Andras Katona commented on RANGER-3231:
---

I think something is wrong with your setup, it's hard to tell from these 
snippets. Yet, if you'd enable debug logging, it would log the ranger request 
sent to ranger and the answer (which will be denied in our case). And ranger 
side logs would help you too.

I've tried this patch with 2.8.1 kafka and 3.1.0 kafka too and works for me.

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-03-17 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508020#comment-17508020
 ] 

Andras Katona commented on RANGER-3231:
---

That's strange, could you download again the patch and recompile and try again? 
I've improved the logging a bit, if you could attach the logs, I can help 
investigate.

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Updated] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8

2022-03-02 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-3459:
--
Fix Version/s: 3.0.0

> Upgrade Ranger's Kafka dependency to 2.8
> 
>
> Key: RANGER-3459
> URL: https://issues.apache.org/jira/browse/RANGER-3459
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins, Ranger
>Reporter: Andras Katona
>Assignee: Andras Katona
>Priority: Major
> Fix For: 3.0.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-03-02 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500088#comment-17500088
 ] 

Andras Katona commented on RANGER-3231:
---

I've prepared a reviewboard request: https://reviews.apache.org/r/73880/, which 
is practically the same as the above mentioned PR 
(https://github.com/apache/ranger/pull/133).

It differs from the previous patch:
* start method is implemented too as [AclAuthorizer does in 
Kafka|https://github.com/apache/kafka/blob/f089bea7ed9fee2bad7084e6b40561ff6935ce5f/core/src/main/scala/kafka/security/authorizer/AclAuthorizer.scala#L194-L195]
* unsupported methods in Ranger-Kafka plugin are throwing/wrapping 
UnsupportedOperationException instead of returning empty
* calling the base-ranger plugin isAccessAllowed method with a list of requests 
instead of one-by-one - to support bulk-like authorization better


> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Chia-Ping Tsai
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Assigned] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-03-02 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona reassigned RANGER-3231:
-

Assignee: Andras Katona  (was: Chia-Ping Tsai)

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Andras Katona
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3231) Ranger should use kafka Authorizer from KIP-504

2022-02-28 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498900#comment-17498900
 ] 

Andras Katona commented on RANGER-3231:
---

I created a PR to test my solution: https://github.com/apache/ranger/pull/133 
I'm going to create a new reviewboard request from it once I have it reviewed 
by a friend first. 
[~chia7712], I've added you as co-author of my patch as you started to work on 
it and used some lines from your patch.

> Ranger should use kafka Authorizer from KIP-504
> ---
>
> Key: RANGER-3231
> URL: https://issues.apache.org/jira/browse/RANGER-3231
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Ismael Juma
>Assignee: Chia-Ping Tsai
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> As described in the KIP, `org.apache.kafka.server.authorizer.Authorizer` is 
> an improvement over `kafka.security.auth.Authorizer` and it's a pure Java 
> interface (instead of Scala).
> `kafka.security.auth.Authorizer` has been deprecated since December 2019 and 
> it will be removed in Apache Kafka 3.0 (roughly planned for July/August).
> See the KIP for more details:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Assigned] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8

2022-02-28 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona reassigned RANGER-3459:
-

Assignee: Andras Katona

> Upgrade Ranger's Kafka dependency to 2.8
> 
>
> Key: RANGER-3459
> URL: https://issues.apache.org/jira/browse/RANGER-3459
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins, Ranger
>Reporter: Andras Katona
>Assignee: Andras Katona
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8

2022-02-28 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498764#comment-17498764
 ] 

Andras Katona commented on RANGER-3459:
---

[~rmani], please find the patch below.

PR:
https://github.com/apache/ranger/pull/134
ReviewBoard:
https://reviews.apache.org/r/73877/


> Upgrade Ranger's Kafka dependency to 2.8
> 
>
> Key: RANGER-3459
> URL: https://issues.apache.org/jira/browse/RANGER-3459
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins, Ranger
>Reporter: Andras Katona
>Assignee: Andras Katona
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Commented] (RANGER-2426) ranger-plugins-audits should depend on kafka-clients not kafka server

2022-02-24 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497467#comment-17497467
 ] 

Andras Katona commented on RANGER-2426:
---

PR:
https://github.com/apache/ranger/pull/132
Reviewboard request:
https://reviews.apache.org/r/73866/

> ranger-plugins-audits should depend on kafka-clients not kafka server
> -
>
> Key: RANGER-2426
> URL: https://issues.apache.org/jira/browse/RANGER-2426
> Project: Ranger
>  Issue Type: Bug
>  Components: audit
>Reporter: Fredy Wijaya
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://github.com/apache/ranger/blob/f2d310a54ee64a1738a9c1a6afdd93d032e64ddf/agents-audit/pom.xml#L72
> To reduce the transitive dependency, ranger-plugins-audit should depend on 
> kafka-clients.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (RANGER-3459) Upgrade Ranger's Kafka dependency to 2.8

2021-09-30 Thread Andras Katona (Jira)
Andras Katona created RANGER-3459:
-

 Summary: Upgrade Ranger's Kafka dependency to 2.8
 Key: RANGER-3459
 URL: https://issues.apache.org/jira/browse/RANGER-3459
 Project: Ranger
  Issue Type: Improvement
  Components: plugins, Ranger
Reporter: Andras Katona






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (RANGER-3096) Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger

2020-12-01 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-3096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241661#comment-17241661
 ] 

Andras Katona edited comment on RANGER-3096 at 12/1/20, 4:12 PM:
-

I've uploaded a patch for this:
https://reviews.apache.org/r/73054/


was (Author: akatona):
https://reviews.apache.org/r/73054/

> Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger
> ---
>
> Key: RANGER-3096
> URL: https://issues.apache.org/jira/browse/RANGER-3096
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Andras Katona
>Priority: Major
>
> Kafka is removing kafka_2.11 artifact from 2.5.0.
> Upgrading kafka version requires another Kafka core artifact to be pulled in.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (RANGER-3096) Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger

2020-12-01 Thread Andras Katona (Jira)
Andras Katona created RANGER-3096:
-

 Summary: Upgrade to Kafka 2.5 with kafka_2.12 artifact in Ranger
 Key: RANGER-3096
 URL: https://issues.apache.org/jira/browse/RANGER-3096
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Andras Katona


Kafka is removing kafka_2.11 artifact from 2.5.0.
Upgrading kafka version requires another Kafka core artifact to be pulled in.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4

2020-04-07 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077166#comment-17077166
 ] 

Andras Katona commented on RANGER-2702:
---

This ticket upgraded kafka dependency to 2.4.0, not to kafka 2.0.0

> Upgrade Kafka Version in Ranger to 2.4
> --
>
> Key: RANGER-2702
> URL: https://issues.apache.org/jira/browse/RANGER-2702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
>  Labels: security
> Fix For: 2.1.0
>
>
> Upgrade Kafka Version in Ranger to 2.4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4

2020-03-05 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-2702:
--
Attachment: (was: 
0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch)

> Upgrade Kafka Version in Ranger to 2.4
> --
>
> Key: RANGER-2702
> URL: https://issues.apache.org/jira/browse/RANGER-2702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.1.0
>
>
> Upgrade Kafka Version in Ranger to 2.4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4

2020-01-28 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024971#comment-17024971
 ] 

Andras Katona commented on RANGER-2702:
---

Ok. Thanks!

> Upgrade Kafka Version in Ranger to 2.4
> --
>
> Key: RANGER-2702
> URL: https://issues.apache.org/jira/browse/RANGER-2702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 
> 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch
>
>
> Upgrade Kafka Version in Ranger to 2.4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4

2020-01-24 Thread Andras Katona (Jira)


 [ 
https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Katona updated RANGER-2702:
--
Attachment: 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch

> Upgrade Kafka Version in Ranger to 2.4
> --
>
> Key: RANGER-2702
> URL: https://issues.apache.org/jira/browse/RANGER-2702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 
> 0001-RANGER-2702-Upgrade-Kafka-Version-in-Ranger-to-2.4.patch
>
>
> Upgrade Kafka Version in Ranger to 2.4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4

2020-01-22 Thread Andras Katona (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021005#comment-17021005
 ] 

Andras Katona commented on RANGER-2702:
---

[~rmani], uploaded my patch to here: https://reviews.apache.org/r/72037/

> Upgrade Kafka Version in Ranger to 2.4
> --
>
> Key: RANGER-2702
> URL: https://issues.apache.org/jira/browse/RANGER-2702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.1.0
>
>
> Upgrade Kafka Version in Ranger to 2.4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)