Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, Is this KIP still active? In case you have no time for it, I'd like to continue this. Thanks, Viktor On Thu, May 23, 2019 at 6:37 PM Yaodong Yang wrote: > friendly ping... > > On Wed, May 15, 2019 at 10:57 AM Yaodong Yang > wrote: > > > Hello Colin, Rajini and Jun, > > > > I have

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

friendly ping... On Wed, May 15, 2019 at 10:57 AM Yaodong Yang wrote: > Hello Colin, Rajini and Jun, > > I have updated the KIP 422, please take another look! > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97555704 > > Thanks! > Yaodong > > On Tue, May 7, 2019 at 11:54 AM

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hello Colin, Rajini and Jun, I have updated the KIP 422, please take another look! https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97555704 Thanks! Yaodong On Tue, May 7, 2019 at 11:54 AM Yaodong Yang wrote: > Hello Colin, Rajini and Jun, > > I think it makes sense to have

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hello Colin, Rajini and Jun, I think it makes sense to have new APIs defined in the AdminClient for quota management only, as Colin described above. For the request and response protocol, It seems like we can leverage the existing requests: *IncrementalAlterConfigsRequest* and

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi all, In KIP-133: Describe and Alter Configs Admin APIs, there is "future work" section that explains: > Future Work > ... > > 2. Support for reading and updating client, user and replication quotas. We > initially included that in the KIP, but it subsequently became apparent > that a

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Jun, Yaodong, 21. The proposed approach sounds very hacky. User principals can contain arbitrary characters. So we can't simply split `user1/clients/clientA` into tokens using '/' as delimiter. Internally, we sanitize names before storing in ZK, but the names provided by the user are actual

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi, Yaodong, Yes, what you proposed makes sense. A couple of more comments. 21. Could you add those examples to the KIP wiki? It would also be useful to know how to set the ConfigEntry value for quotas at DefaultClientInUser, DefaultClientDefaultUser and DefaultUser level. 22. To support

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Jun, The proposal we have right now is to directly set the quota through existing admin client APIs. See following examples: Example 1: set a user quota AdminClient adminClient = mock(AdminClient.class); Map configs = new HashMap(); Config config = new Config(Arrays.asList(new

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi, Yaodong, Thanks for the updated KIP. A few more comments below. 11. For quotas, in addition to user and client, we need to be able to set a quota for combination. Would that be a new resource type? How would the name look like for this resource? 12. Similarly, to support customizable quota

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

ping... On Thu, Feb 28, 2019 at 10:23 AM Yaodong Yang wrote: > Hello folks, > > Please share your comments for this KIP  > > Thanks! > Yaodong > > On Tue, Feb 26, 2019 at 6:53 PM Yaodong Yang > wrote: > >> Hello Colin, >> >> There is a POC PR for this KIP, and it contains most changes we are

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hello folks, Please share your comments for this KIP  Thanks! Yaodong On Tue, Feb 26, 2019 at 6:53 PM Yaodong Yang wrote: > Hello Colin, > > There is a POC PR for this KIP, and it contains most changes we are > proposing now. > > Best, > Yaodong > > On Tue, Feb 26, 2019 at 6:51 PM Yaodong

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hello Colin, There is a POC PR for this KIP, and it contains most changes we are proposing now. Best, Yaodong On Tue, Feb 26, 2019 at 6:51 PM Yaodong Yang wrote: > Hello Colin, > > CIL, > > Thanks! > Yaodong > > > On Tue, Feb 26, 2019 at 9:59 AM Colin McCabe wrote: > >> Hi Yaodong, >> >> I

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hello Colin, CIL, Thanks! Yaodong On Tue, Feb 26, 2019 at 9:59 AM Colin McCabe wrote: > Hi Yaodong, > > I don't understand how the proposed API would be used. It talks about > adding a ConfigResource type for clients and users, but doesn't explain > what can be done with these. > Sorry for

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, I don't understand how the proposed API would be used. It talks about adding a ConfigResource type for clients and users, but doesn't explain what can be done with these. In the compatibility section (?) it says "We only add a new way to configure the quotas" which suggests that

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Colin, CIL, Thanks! Yaodong On Fri, Feb 22, 2019 at 10:56 AM Colin McCabe wrote: > Hi Yaodong, > > KIP-422 says that it would be good if "applications [could] leverage the > unified KafkaAdminClient to manage their user/client configurations, > instead of the direct dependency on

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, KIP-422 says that it would be good if "applications [could] leverage the unified KafkaAdminClient to manage their user/client configurations, instead of the direct dependency on Zookeeper." But the KIP doesn't talk about any changes to KafkaAdminClient. Instead, the only changes

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hello Jun, Viktor, Snoke and Stan, Thanks for taking time to look at this KIP-422! For some reason, this email was put in my spam folder. Sorry about that. Jun is right, the main motivation for this KIP-422 is to allow users to config user/clientId quota through AdminClient. In addition, this

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Guys, I wanted to reject that KIP, split it up and revamp it as in the meantime there were some overlapping works I just didn't get to it due to other higher priority work. One of the splitted KIPs would have been the quota part of that and I'd be happy if that lived in this KIP if Yaodong

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi, Yaodong, Thanks for the KIP. As Stan mentioned earlier, it seems that this is mostly covered by KIP-248, which was originally proposed by Victor. Hi, Victor, Do you still plan to work on KIP-248? It seems that you already got pretty far on that. If not, would you mind letting Yaodong take

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, thanks for the KIP! If I understand your intentions correctly then this KIP would only address a fairly specific use case, namely SASL-PLAIN with users defined in Zookeeper. For all other authentication mechanisms like SSL, SASL-GSSAPI or SASL-PLAIN with users defined in jaas files I

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

This KIP seems to duplicate some of the functionality proposed in KIP-248 . KIP-248 has been stuck in a vote thread since July 2018. Viktor, do you plan on working on the KIP? On

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hey there Yaodong, thanks for the KIP! I'm not too familiar with the user/client configurations we currently allow, is there a KIP describing the initial feature? If there is, it would be useful to include in KIP-422. I also didn't see any authorization in the PR, have we thought about needing

[DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi folks, I've published KIP-422 which is about adding support for user/client configurations in the Kafka Admin Client. Basically the story here is to allow KafkaAdminClient to configure the user or client configurations for users, instead of requiring users to directly talk to ZK. The link