RE: [VOTE] KIP-21 Dynamic Configuration
Thanks Jay. Since we have enough votes now, I'll mark this as adopted. Aditya From: Jay Kreps [jay.kr...@gmail.com] Sent: Monday, June 01, 2015 8:42 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration Awesome. +1 On Sunday, May 31, 2015, Aditya Auradkar wrote: > 2. There was a typo in my previous email. I meant to say that we should > use snake case because it's more consistent. I couldn't find any examples > of camel case but did find some snake case (jmx_port). Other than that, > most other entries are single word keys. > > 3. The purge frequency is short (15 minutes). So it should be safe to > ignore older notifications. Add a story here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration#KIP-21-DynamicConfiguration-Migrationplanfornotifications > > Thanks, > Aditya > > From: Jay Kreps [jay.kr...@gmail.com ] > Sent: Saturday, May 30, 2015 9:51 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > 1. Great. > 2. I don't have a preference as to the casing, but I really appreciate > consistency. Is everything using underscores today? If so let's stick with > that. If we are already inconsistent then I guess it's too late and we can > do whatever. Let me know and I'll update the coding standard. > 3. Not sure what the default purge frequency is. I don't think we need to > work the details of this out in the KIP, but we need a story for the > upgrade path so people don't get bitten. > > -Jay > > On Thu, May 28, 2015 at 11:22 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Yeah, the same cleaning mechanism will be carried over. > > > > > 1. Are we introducing a new Java API for the config change protocol and > > if > > > so where will that appear? Is that going to be part of the java api in > > the > > > admin api kip? Let's document that. > > Yeah, we need to introduce a new Java API for the config change protocol. > > It should be a part of the AdminClient API. I'll alter KIP-4 to reflect > > that since the API is being introduced there. > > > > > 2. The proposed JSON format uses camel case for field names, is that > what > > > we've used for other JSON in zookeeper? > > I think camel case is more appropriate for the JSON format. For example, > > under the "brokers" znode, I found "jmx_port". > > > > > 3. This changes the format of the notifications, right? How will we > > > grandfather in the old format? Clusters will have existing change > > > notifications in the old format so I think the new code will need to be > > > able to read those? > > Interesting, I figured the existing notifications were purged by a > cleaner > > thread frequently. In that case, we wouldn't need to grandfather any > > notifications since we would only need to not make any config changes > for X > > minutes for there to be no changes in ZK. But the old notifications are > > actually removed when a new notification is received or the broker is > > bounced. So we do need to handle notifications in the old format. Should > we > > simply ignore older changes since they are only valid for a short period > of > > time? > > > > Thanks, > > Aditya > > > > From: Jay Kreps [jay.kr...@gmail.com ] > > Sent: Thursday, May 28, 2015 5:25 PM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > That is handled now so I am assuming the same mechanism carries over? > > > > -Jay > > > > On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang > wrote: > > > > > For the sequential config/changes/config_change_XX znode, do we have > any > > > manners to do cleaning in order to avoid the change-log from growing > > > indefinitely? > > > > > > Guozhang > > > > > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps > wrote: > > > > > > > I still have a couple of questions: > > > > 1. Are we introducing a new Java API for the config change protocol > and > > > if > > > > so where will that appear? Is that going to be part of the java api > in > > > the > > > > admin api kip? Let's document that. > > > > 2. The proposed JSON format uses camel case for field names, is that > > what > > > > we've used for other JSON in zookeep
Re: [VOTE] KIP-21 Dynamic Configuration
Awesome. +1 On Sunday, May 31, 2015, Aditya Auradkar wrote: > 2. There was a typo in my previous email. I meant to say that we should > use snake case because it's more consistent. I couldn't find any examples > of camel case but did find some snake case (jmx_port). Other than that, > most other entries are single word keys. > > 3. The purge frequency is short (15 minutes). So it should be safe to > ignore older notifications. Add a story here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration#KIP-21-DynamicConfiguration-Migrationplanfornotifications > > Thanks, > Aditya > > From: Jay Kreps [jay.kr...@gmail.com ] > Sent: Saturday, May 30, 2015 9:51 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > 1. Great. > 2. I don't have a preference as to the casing, but I really appreciate > consistency. Is everything using underscores today? If so let's stick with > that. If we are already inconsistent then I guess it's too late and we can > do whatever. Let me know and I'll update the coding standard. > 3. Not sure what the default purge frequency is. I don't think we need to > work the details of this out in the KIP, but we need a story for the > upgrade path so people don't get bitten. > > -Jay > > On Thu, May 28, 2015 at 11:22 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Yeah, the same cleaning mechanism will be carried over. > > > > > 1. Are we introducing a new Java API for the config change protocol and > > if > > > so where will that appear? Is that going to be part of the java api in > > the > > > admin api kip? Let's document that. > > Yeah, we need to introduce a new Java API for the config change protocol. > > It should be a part of the AdminClient API. I'll alter KIP-4 to reflect > > that since the API is being introduced there. > > > > > 2. The proposed JSON format uses camel case for field names, is that > what > > > we've used for other JSON in zookeeper? > > I think camel case is more appropriate for the JSON format. For example, > > under the "brokers" znode, I found "jmx_port". > > > > > 3. This changes the format of the notifications, right? How will we > > > grandfather in the old format? Clusters will have existing change > > > notifications in the old format so I think the new code will need to be > > > able to read those? > > Interesting, I figured the existing notifications were purged by a > cleaner > > thread frequently. In that case, we wouldn't need to grandfather any > > notifications since we would only need to not make any config changes > for X > > minutes for there to be no changes in ZK. But the old notifications are > > actually removed when a new notification is received or the broker is > > bounced. So we do need to handle notifications in the old format. Should > we > > simply ignore older changes since they are only valid for a short period > of > > time? > > > > Thanks, > > Aditya > > > > From: Jay Kreps [jay.kr...@gmail.com ] > > Sent: Thursday, May 28, 2015 5:25 PM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > That is handled now so I am assuming the same mechanism carries over? > > > > -Jay > > > > On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang > wrote: > > > > > For the sequential config/changes/config_change_XX znode, do we have > any > > > manners to do cleaning in order to avoid the change-log from growing > > > indefinitely? > > > > > > Guozhang > > > > > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps > wrote: > > > > > > > I still have a couple of questions: > > > > 1. Are we introducing a new Java API for the config change protocol > and > > > if > > > > so where will that appear? Is that going to be part of the java api > in > > > the > > > > admin api kip? Let's document that. > > > > 2. The proposed JSON format uses camel case for field names, is that > > what > > > > we've used for other JSON in zookeeper? > > > > 3. This changes the format of the notifications, right? How will we > > > > grandfather in the old format? Clusters will have existing change > > > > notifications in the old format so I think the new code will need to >
RE: [VOTE] KIP-21 Dynamic Configuration
2. There was a typo in my previous email. I meant to say that we should use snake case because it's more consistent. I couldn't find any examples of camel case but did find some snake case (jmx_port). Other than that, most other entries are single word keys. 3. The purge frequency is short (15 minutes). So it should be safe to ignore older notifications. Add a story here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration#KIP-21-DynamicConfiguration-Migrationplanfornotifications Thanks, Aditya From: Jay Kreps [jay.kr...@gmail.com] Sent: Saturday, May 30, 2015 9:51 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration 1. Great. 2. I don't have a preference as to the casing, but I really appreciate consistency. Is everything using underscores today? If so let's stick with that. If we are already inconsistent then I guess it's too late and we can do whatever. Let me know and I'll update the coding standard. 3. Not sure what the default purge frequency is. I don't think we need to work the details of this out in the KIP, but we need a story for the upgrade path so people don't get bitten. -Jay On Thu, May 28, 2015 at 11:22 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Yeah, the same cleaning mechanism will be carried over. > > > 1. Are we introducing a new Java API for the config change protocol and > if > > so where will that appear? Is that going to be part of the java api in > the > > admin api kip? Let's document that. > Yeah, we need to introduce a new Java API for the config change protocol. > It should be a part of the AdminClient API. I'll alter KIP-4 to reflect > that since the API is being introduced there. > > > 2. The proposed JSON format uses camel case for field names, is that what > > we've used for other JSON in zookeeper? > I think camel case is more appropriate for the JSON format. For example, > under the "brokers" znode, I found "jmx_port". > > > 3. This changes the format of the notifications, right? How will we > > grandfather in the old format? Clusters will have existing change > > notifications in the old format so I think the new code will need to be > > able to read those? > Interesting, I figured the existing notifications were purged by a cleaner > thread frequently. In that case, we wouldn't need to grandfather any > notifications since we would only need to not make any config changes for X > minutes for there to be no changes in ZK. But the old notifications are > actually removed when a new notification is received or the broker is > bounced. So we do need to handle notifications in the old format. Should we > simply ignore older changes since they are only valid for a short period of > time? > > Thanks, > Aditya > > From: Jay Kreps [jay.kr...@gmail.com] > Sent: Thursday, May 28, 2015 5:25 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > That is handled now so I am assuming the same mechanism carries over? > > -Jay > > On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang wrote: > > > For the sequential config/changes/config_change_XX znode, do we have any > > manners to do cleaning in order to avoid the change-log from growing > > indefinitely? > > > > Guozhang > > > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps wrote: > > > > > I still have a couple of questions: > > > 1. Are we introducing a new Java API for the config change protocol and > > if > > > so where will that appear? Is that going to be part of the java api in > > the > > > admin api kip? Let's document that. > > > 2. The proposed JSON format uses camel case for field names, is that > what > > > we've used for other JSON in zookeeper? > > > 3. This changes the format of the notifications, right? How will we > > > grandfather in the old format? Clusters will have existing change > > > notifications in the old format so I think the new code will need to be > > > able to read those? > > > > > > -Jay > > > > > > On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > bump > > > > > > > > > > > > From: Aditya Auradkar > > > > Sent: Tuesday, May 26, 2015 1:16 PM > > > > To: dev@kafka.apache.org > > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > &g
Re: [VOTE] KIP-21 Dynamic Configuration
1. Great. 2. I don't have a preference as to the casing, but I really appreciate consistency. Is everything using underscores today? If so let's stick with that. If we are already inconsistent then I guess it's too late and we can do whatever. Let me know and I'll update the coding standard. 3. Not sure what the default purge frequency is. I don't think we need to work the details of this out in the KIP, but we need a story for the upgrade path so people don't get bitten. -Jay On Thu, May 28, 2015 at 11:22 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Yeah, the same cleaning mechanism will be carried over. > > > 1. Are we introducing a new Java API for the config change protocol and > if > > so where will that appear? Is that going to be part of the java api in > the > > admin api kip? Let's document that. > Yeah, we need to introduce a new Java API for the config change protocol. > It should be a part of the AdminClient API. I'll alter KIP-4 to reflect > that since the API is being introduced there. > > > 2. The proposed JSON format uses camel case for field names, is that what > > we've used for other JSON in zookeeper? > I think camel case is more appropriate for the JSON format. For example, > under the "brokers" znode, I found "jmx_port". > > > 3. This changes the format of the notifications, right? How will we > > grandfather in the old format? Clusters will have existing change > > notifications in the old format so I think the new code will need to be > > able to read those? > Interesting, I figured the existing notifications were purged by a cleaner > thread frequently. In that case, we wouldn't need to grandfather any > notifications since we would only need to not make any config changes for X > minutes for there to be no changes in ZK. But the old notifications are > actually removed when a new notification is received or the broker is > bounced. So we do need to handle notifications in the old format. Should we > simply ignore older changes since they are only valid for a short period of > time? > > Thanks, > Aditya > > From: Jay Kreps [jay.kr...@gmail.com] > Sent: Thursday, May 28, 2015 5:25 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > That is handled now so I am assuming the same mechanism carries over? > > -Jay > > On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang wrote: > > > For the sequential config/changes/config_change_XX znode, do we have any > > manners to do cleaning in order to avoid the change-log from growing > > indefinitely? > > > > Guozhang > > > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps wrote: > > > > > I still have a couple of questions: > > > 1. Are we introducing a new Java API for the config change protocol and > > if > > > so where will that appear? Is that going to be part of the java api in > > the > > > admin api kip? Let's document that. > > > 2. The proposed JSON format uses camel case for field names, is that > what > > > we've used for other JSON in zookeeper? > > > 3. This changes the format of the notifications, right? How will we > > > grandfather in the old format? Clusters will have existing change > > > notifications in the old format so I think the new code will need to be > > > able to read those? > > > > > > -Jay > > > > > > On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > bump > > > > > > > > > > > > From: Aditya Auradkar > > > > Sent: Tuesday, May 26, 2015 1:16 PM > > > > To: dev@kafka.apache.org > > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > > > Hey everyone, > > > > > > > > Completed the changes to KIP-4. After today's hangout, there doesn't > > > > appear to be anything remaining to discuss on this KIP. > > > > Please vote so we can formally close this. > > > > > > > > Thanks, > > > > Aditya > > > > > > > > > > > > From: Aditya Auradkar > > > > Sent: Thursday, May 21, 2015 11:26 AM > > > > To: dev@kafka.apache.org > > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > > > I think we should remove th
RE: [VOTE] KIP-21 Dynamic Configuration
Minor edit: I meant that we should expect change notifications in the old format made earlier, but should perhaps ignore them. After the upgrade is done, older versions of AdminTools can no longer be used to make config changes. Aditya From: Aditya Auradkar Sent: Thursday, May 28, 2015 11:22 PM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-21 Dynamic Configuration Yeah, the same cleaning mechanism will be carried over. > 1. Are we introducing a new Java API for the config change protocol and if > so where will that appear? Is that going to be part of the java api in the > admin api kip? Let's document that. Yeah, we need to introduce a new Java API for the config change protocol. It should be a part of the AdminClient API. I'll alter KIP-4 to reflect that since the API is being introduced there. > 2. The proposed JSON format uses camel case for field names, is that what > we've used for other JSON in zookeeper? I think camel case is more appropriate for the JSON format. For example, under the "brokers" znode, I found "jmx_port". > 3. This changes the format of the notifications, right? How will we > grandfather in the old format? Clusters will have existing change > notifications in the old format so I think the new code will need to be > able to read those? Interesting, I figured the existing notifications were purged by a cleaner thread frequently. In that case, we wouldn't need to grandfather any notifications since we would only need to not make any config changes for X minutes for there to be no changes in ZK. But the old notifications are actually removed when a new notification is received or the broker is bounced. So we do need to handle notifications in the old format. Should we simply ignore older changes since they are only valid for a short period of time? Thanks, Aditya From: Jay Kreps [jay.kr...@gmail.com] Sent: Thursday, May 28, 2015 5:25 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration That is handled now so I am assuming the same mechanism carries over? -Jay On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang wrote: > For the sequential config/changes/config_change_XX znode, do we have any > manners to do cleaning in order to avoid the change-log from growing > indefinitely? > > Guozhang > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps wrote: > > > I still have a couple of questions: > > 1. Are we introducing a new Java API for the config change protocol and > if > > so where will that appear? Is that going to be part of the java api in > the > > admin api kip? Let's document that. > > 2. The proposed JSON format uses camel case for field names, is that what > > we've used for other JSON in zookeeper? > > 3. This changes the format of the notifications, right? How will we > > grandfather in the old format? Clusters will have existing change > > notifications in the old format so I think the new code will need to be > > able to read those? > > > > -Jay > > > > On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > bump > > > > > > > > > From: Aditya Auradkar > > > Sent: Tuesday, May 26, 2015 1:16 PM > > > To: dev@kafka.apache.org > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hey everyone, > > > > > > Completed the changes to KIP-4. After today's hangout, there doesn't > > > appear to be anything remaining to discuss on this KIP. > > > Please vote so we can formally close this. > > > > > > Thanks, > > > Aditya > > > > > > > > > From: Aditya Auradkar > > > Sent: Thursday, May 21, 2015 11:26 AM > > > To: dev@kafka.apache.org > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > I think we should remove the config part in TopicMetadataResponse. It's > > > probably cleaner if Alter and Describe are the only way to view and > > modify > > > configs but I don't feel very strongly about it. > > > > > > Re-summarizing the proposed changes to KIP-4: > > > - Change AlterTopic to not allow setting configs. Config changes will > > flow > > > through AlterConfig. CreateTopic will still allow setting configs as it > > is > > > nice to be able to specify configs while creating the topic. > > > - TopicMetadataResponse shoudn't return config for the topic. > > > DescribeConfig
RE: [VOTE] KIP-21 Dynamic Configuration
Yeah, the same cleaning mechanism will be carried over. > 1. Are we introducing a new Java API for the config change protocol and if > so where will that appear? Is that going to be part of the java api in the > admin api kip? Let's document that. Yeah, we need to introduce a new Java API for the config change protocol. It should be a part of the AdminClient API. I'll alter KIP-4 to reflect that since the API is being introduced there. > 2. The proposed JSON format uses camel case for field names, is that what > we've used for other JSON in zookeeper? I think camel case is more appropriate for the JSON format. For example, under the "brokers" znode, I found "jmx_port". > 3. This changes the format of the notifications, right? How will we > grandfather in the old format? Clusters will have existing change > notifications in the old format so I think the new code will need to be > able to read those? Interesting, I figured the existing notifications were purged by a cleaner thread frequently. In that case, we wouldn't need to grandfather any notifications since we would only need to not make any config changes for X minutes for there to be no changes in ZK. But the old notifications are actually removed when a new notification is received or the broker is bounced. So we do need to handle notifications in the old format. Should we simply ignore older changes since they are only valid for a short period of time? Thanks, Aditya From: Jay Kreps [jay.kr...@gmail.com] Sent: Thursday, May 28, 2015 5:25 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration That is handled now so I am assuming the same mechanism carries over? -Jay On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang wrote: > For the sequential config/changes/config_change_XX znode, do we have any > manners to do cleaning in order to avoid the change-log from growing > indefinitely? > > Guozhang > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps wrote: > > > I still have a couple of questions: > > 1. Are we introducing a new Java API for the config change protocol and > if > > so where will that appear? Is that going to be part of the java api in > the > > admin api kip? Let's document that. > > 2. The proposed JSON format uses camel case for field names, is that what > > we've used for other JSON in zookeeper? > > 3. This changes the format of the notifications, right? How will we > > grandfather in the old format? Clusters will have existing change > > notifications in the old format so I think the new code will need to be > > able to read those? > > > > -Jay > > > > On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > bump > > > > > > > > > From: Aditya Auradkar > > > Sent: Tuesday, May 26, 2015 1:16 PM > > > To: dev@kafka.apache.org > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hey everyone, > > > > > > Completed the changes to KIP-4. After today's hangout, there doesn't > > > appear to be anything remaining to discuss on this KIP. > > > Please vote so we can formally close this. > > > > > > Thanks, > > > Aditya > > > > > > > > > From: Aditya Auradkar > > > Sent: Thursday, May 21, 2015 11:26 AM > > > To: dev@kafka.apache.org > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > I think we should remove the config part in TopicMetadataResponse. It's > > > probably cleaner if Alter and Describe are the only way to view and > > modify > > > configs but I don't feel very strongly about it. > > > > > > Re-summarizing the proposed changes to KIP-4: > > > - Change AlterTopic to not allow setting configs. Config changes will > > flow > > > through AlterConfig. CreateTopic will still allow setting configs as it > > is > > > nice to be able to specify configs while creating the topic. > > > - TopicMetadataResponse shoudn't return config for the topic. > > > DescribeConfig is the way to go. > > > - Change "InvalidTopicConfiguration" error code to > "InvalidEntityConfig" > > > as proposed in KIP-21. > > > > > > Aditya > > > > > > > > > From: Jun Rao [j...@confluent.io] > > > Sent: Thursday, May 21, 2015 10:50 AM > > > To: dev@kafka.apach
Re: [VOTE] KIP-21 Dynamic Configuration
That is handled now so I am assuming the same mechanism carries over? -Jay On Thu, May 28, 2015 at 5:12 PM, Guozhang Wang wrote: > For the sequential config/changes/config_change_XX znode, do we have any > manners to do cleaning in order to avoid the change-log from growing > indefinitely? > > Guozhang > > On Thu, May 28, 2015 at 5:02 PM, Jay Kreps wrote: > > > I still have a couple of questions: > > 1. Are we introducing a new Java API for the config change protocol and > if > > so where will that appear? Is that going to be part of the java api in > the > > admin api kip? Let's document that. > > 2. The proposed JSON format uses camel case for field names, is that what > > we've used for other JSON in zookeeper? > > 3. This changes the format of the notifications, right? How will we > > grandfather in the old format? Clusters will have existing change > > notifications in the old format so I think the new code will need to be > > able to read those? > > > > -Jay > > > > On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > bump > > > > > > ____________ > > > From: Aditya Auradkar > > > Sent: Tuesday, May 26, 2015 1:16 PM > > > To: dev@kafka.apache.org > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hey everyone, > > > > > > Completed the changes to KIP-4. After today's hangout, there doesn't > > > appear to be anything remaining to discuss on this KIP. > > > Please vote so we can formally close this. > > > > > > Thanks, > > > Aditya > > > > > > > > > From: Aditya Auradkar > > > Sent: Thursday, May 21, 2015 11:26 AM > > > To: dev@kafka.apache.org > > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > > > I think we should remove the config part in TopicMetadataResponse. It's > > > probably cleaner if Alter and Describe are the only way to view and > > modify > > > configs but I don't feel very strongly about it. > > > > > > Re-summarizing the proposed changes to KIP-4: > > > - Change AlterTopic to not allow setting configs. Config changes will > > flow > > > through AlterConfig. CreateTopic will still allow setting configs as it > > is > > > nice to be able to specify configs while creating the topic. > > > - TopicMetadataResponse shoudn't return config for the topic. > > > DescribeConfig is the way to go. > > > - Change "InvalidTopicConfiguration" error code to > "InvalidEntityConfig" > > > as proposed in KIP-21. > > > > > > Aditya > > > > > > > > > From: Jun Rao [j...@confluent.io] > > > Sent: Thursday, May 21, 2015 10:50 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > What about TopicMetadataResponse in KIP-4? Do we remove the config part > > in > > > it? > > > > > > Thanks, > > > > > > Jun > > > > > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > Hey Jun, > > > > > > > > I've added a section on error codes on the KIP-21 wiki. > > > > > > > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > > > > - Change AlterTopic to not allow setting configs. Config changes will > > > flow > > > > through AlterConfig. CreateTopic will still allow setting configs as > it > > > is > > > > nice to be able to specify configs while creating the topic. > > > > - Change "InvalidTopicConfiguration" error code to > > "InvalidEntityConfig" > > > > as proposed in KIP-21. > > > > > > > > > > > > Thanks, > > > > Aditya > > > > > > > > > > > > From: Jun Rao [j...@confluent.io] > > > > Sent: Thursday, May 21, 2015 8:41 AM > > > > To: dev@kafka.apache.org > > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > > > Aditya, > > > > > > > > For completeness, could you list the set of error codes in the wiki? > >
Re: [VOTE] KIP-21 Dynamic Configuration
For the sequential config/changes/config_change_XX znode, do we have any manners to do cleaning in order to avoid the change-log from growing indefinitely? Guozhang On Thu, May 28, 2015 at 5:02 PM, Jay Kreps wrote: > I still have a couple of questions: > 1. Are we introducing a new Java API for the config change protocol and if > so where will that appear? Is that going to be part of the java api in the > admin api kip? Let's document that. > 2. The proposed JSON format uses camel case for field names, is that what > we've used for other JSON in zookeeper? > 3. This changes the format of the notifications, right? How will we > grandfather in the old format? Clusters will have existing change > notifications in the old format so I think the new code will need to be > able to read those? > > -Jay > > On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > bump > > > > > > From: Aditya Auradkar > > Sent: Tuesday, May 26, 2015 1:16 PM > > To: dev@kafka.apache.org > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > Hey everyone, > > > > Completed the changes to KIP-4. After today's hangout, there doesn't > > appear to be anything remaining to discuss on this KIP. > > Please vote so we can formally close this. > > > > Thanks, > > Aditya > > > > > > From: Aditya Auradkar > > Sent: Thursday, May 21, 2015 11:26 AM > > To: dev@kafka.apache.org > > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > > > I think we should remove the config part in TopicMetadataResponse. It's > > probably cleaner if Alter and Describe are the only way to view and > modify > > configs but I don't feel very strongly about it. > > > > Re-summarizing the proposed changes to KIP-4: > > - Change AlterTopic to not allow setting configs. Config changes will > flow > > through AlterConfig. CreateTopic will still allow setting configs as it > is > > nice to be able to specify configs while creating the topic. > > - TopicMetadataResponse shoudn't return config for the topic. > > DescribeConfig is the way to go. > > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > > as proposed in KIP-21. > > > > Aditya > > > > > > From: Jun Rao [j...@confluent.io] > > Sent: Thursday, May 21, 2015 10:50 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > What about TopicMetadataResponse in KIP-4? Do we remove the config part > in > > it? > > > > Thanks, > > > > Jun > > > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Hey Jun, > > > > > > I've added a section on error codes on the KIP-21 wiki. > > > > > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > > > - Change AlterTopic to not allow setting configs. Config changes will > > flow > > > through AlterConfig. CreateTopic will still allow setting configs as it > > is > > > nice to be able to specify configs while creating the topic. > > > - Change "InvalidTopicConfiguration" error code to > "InvalidEntityConfig" > > > as proposed in KIP-21. > > > > > > > > > Thanks, > > > Aditya > > > > > > > > > From: Jun Rao [j...@confluent.io] > > > Sent: Thursday, May 21, 2015 8:41 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Aditya, > > > > > > For completeness, could you list the set of error codes in the wiki? > > Also, > > > could you summarize the changes that are needed for the requests listed > > in > > > KIP-4 and update the wiki accordingly? > > > > > > Thanks, > > > > > > Jun > > > > > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > Thanks Andrii. I'll make the changes. > > > > > > > > I've also updated KIP-21 to include the new config requests. Take a > > look > > > > and vote. > > > > > > > > > > > > > > https://cwiki.
Re: [VOTE] KIP-21 Dynamic Configuration
I still have a couple of questions: 1. Are we introducing a new Java API for the config change protocol and if so where will that appear? Is that going to be part of the java api in the admin api kip? Let's document that. 2. The proposed JSON format uses camel case for field names, is that what we've used for other JSON in zookeeper? 3. This changes the format of the notifications, right? How will we grandfather in the old format? Clusters will have existing change notifications in the old format so I think the new code will need to be able to read those? -Jay On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > bump > > > From: Aditya Auradkar > Sent: Tuesday, May 26, 2015 1:16 PM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > Hey everyone, > > Completed the changes to KIP-4. After today's hangout, there doesn't > appear to be anything remaining to discuss on this KIP. > Please vote so we can formally close this. > > Thanks, > Aditya > > > From: Aditya Auradkar > Sent: Thursday, May 21, 2015 11:26 AM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > I think we should remove the config part in TopicMetadataResponse. It's > probably cleaner if Alter and Describe are the only way to view and modify > configs but I don't feel very strongly about it. > > Re-summarizing the proposed changes to KIP-4: > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - TopicMetadataResponse shoudn't return config for the topic. > DescribeConfig is the way to go. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > Aditya > > ____ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 10:50 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > What about TopicMetadataResponse in KIP-4? Do we remove the config part in > it? > > Thanks, > > Jun > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Hey Jun, > > > > I've added a section on error codes on the KIP-21 wiki. > > > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > > - Change AlterTopic to not allow setting configs. Config changes will > flow > > through AlterConfig. CreateTopic will still allow setting configs as it > is > > nice to be able to specify configs while creating the topic. > > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > > as proposed in KIP-21. > > > > > > Thanks, > > Aditya > > > > > > From: Jun Rao [j...@confluent.io] > > Sent: Thursday, May 21, 2015 8:41 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Aditya, > > > > For completeness, could you list the set of error codes in the wiki? > Also, > > could you summarize the changes that are needed for the requests listed > in > > KIP-4 and update the wiki accordingly? > > > > Thanks, > > > > Jun > > > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Thanks Andrii. I'll make the changes. > > > > > > I've also updated KIP-21 to include the new config requests. Take a > look > > > and vote. > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > > > Aditya > > > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > > Sent: Tuesday, May 19, 2015 2:26 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hi, > > > > > > Sorry I wasn't able to participate. I don't have objections about > > removing > > > config changes from AlterTopic (as I understand both AddedConfig and > > > DeletedConfig) - you are welcome to update the KIP page. > > > > > > Thanks, > > > Andrii Biletskyi > > > > > > On Tue, May 19, 2015 at 11:40 PM
Re: [VOTE] KIP-21 Dynamic Configuration
I still have a couple of questions: 1. Are we introducing a new Java API for the config change protocol and if so where will that appear? Is that going to be part of the java api in the admin api kip? Let's document that. 2. The proposed JSON format uses camel case for field names, is that what we've used for other JSON in zookeeper? 3. This changes the format of the notifications, right? How will we grandfather in the old format? Clusters will have existing change notifications in the old format so I think the new code will need to be able to read those? -Jay On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > bump > > > From: Aditya Auradkar > Sent: Tuesday, May 26, 2015 1:16 PM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > Hey everyone, > > Completed the changes to KIP-4. After today's hangout, there doesn't > appear to be anything remaining to discuss on this KIP. > Please vote so we can formally close this. > > Thanks, > Aditya > > > From: Aditya Auradkar > Sent: Thursday, May 21, 2015 11:26 AM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > I think we should remove the config part in TopicMetadataResponse. It's > probably cleaner if Alter and Describe are the only way to view and modify > configs but I don't feel very strongly about it. > > Re-summarizing the proposed changes to KIP-4: > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - TopicMetadataResponse shoudn't return config for the topic. > DescribeConfig is the way to go. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > Aditya > > ____ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 10:50 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > What about TopicMetadataResponse in KIP-4? Do we remove the config part in > it? > > Thanks, > > Jun > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Hey Jun, > > > > I've added a section on error codes on the KIP-21 wiki. > > > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > > - Change AlterTopic to not allow setting configs. Config changes will > flow > > through AlterConfig. CreateTopic will still allow setting configs as it > is > > nice to be able to specify configs while creating the topic. > > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > > as proposed in KIP-21. > > > > > > Thanks, > > Aditya > > > > > > From: Jun Rao [j...@confluent.io] > > Sent: Thursday, May 21, 2015 8:41 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Aditya, > > > > For completeness, could you list the set of error codes in the wiki? > Also, > > could you summarize the changes that are needed for the requests listed > in > > KIP-4 and update the wiki accordingly? > > > > Thanks, > > > > Jun > > > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Thanks Andrii. I'll make the changes. > > > > > > I've also updated KIP-21 to include the new config requests. Take a > look > > > and vote. > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > > > Aditya > > > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > > Sent: Tuesday, May 19, 2015 2:26 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hi, > > > > > > Sorry I wasn't able to participate. I don't have objections about > > removing > > > config changes from AlterTopic (as I understand both AddedConfig and > > > DeletedConfig) - you are welcome to update the KIP page. > > > > > > Thanks, > > > Andrii Biletskyi > > > > > > On Tue, May 19, 2015 at 11:40 PM
Re: [VOTE] KIP-21 Dynamic Configuration
+1. Thanks, Jun On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > bump > > > From: Aditya Auradkar > Sent: Tuesday, May 26, 2015 1:16 PM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > Hey everyone, > > Completed the changes to KIP-4. After today's hangout, there doesn't > appear to be anything remaining to discuss on this KIP. > Please vote so we can formally close this. > > Thanks, > Aditya > > > From: Aditya Auradkar > Sent: Thursday, May 21, 2015 11:26 AM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > I think we should remove the config part in TopicMetadataResponse. It's > probably cleaner if Alter and Describe are the only way to view and modify > configs but I don't feel very strongly about it. > > Re-summarizing the proposed changes to KIP-4: > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - TopicMetadataResponse shoudn't return config for the topic. > DescribeConfig is the way to go. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > Aditya > > ____________ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 10:50 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > What about TopicMetadataResponse in KIP-4? Do we remove the config part in > it? > > Thanks, > > Jun > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Hey Jun, > > > > I've added a section on error codes on the KIP-21 wiki. > > > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > > - Change AlterTopic to not allow setting configs. Config changes will > flow > > through AlterConfig. CreateTopic will still allow setting configs as it > is > > nice to be able to specify configs while creating the topic. > > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > > as proposed in KIP-21. > > > > > > Thanks, > > Aditya > > > > > > From: Jun Rao [j...@confluent.io] > > Sent: Thursday, May 21, 2015 8:41 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Aditya, > > > > For completeness, could you list the set of error codes in the wiki? > Also, > > could you summarize the changes that are needed for the requests listed > in > > KIP-4 and update the wiki accordingly? > > > > Thanks, > > > > Jun > > > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Thanks Andrii. I'll make the changes. > > > > > > I've also updated KIP-21 to include the new config requests. Take a > look > > > and vote. > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > > > Aditya > > > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > > Sent: Tuesday, May 19, 2015 2:26 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hi, > > > > > > Sorry I wasn't able to participate. I don't have objections about > > removing > > > config changes from AlterTopic (as I understand both AddedConfig and > > > DeletedConfig) - you are welcome to update the KIP page. > > > > > > Thanks, > > > Andrii Biletskyi > > > > > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > Updating the discussion with the latest comments. > > > > > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). > > I'll > > > > update KIP-21 with details on these. > > > > 2. Discussed during the KIP hangout. We are in agreement. > > > > > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in > the > > &g
Re: [VOTE] KIP-21 Dynamic Configuration
+1 (non binding) > On May 28, 2015, at 11:41 AM, Aditya Auradkar > wrote: > > bump > > > From: Aditya Auradkar > Sent: Tuesday, May 26, 2015 1:16 PM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > Hey everyone, > > Completed the changes to KIP-4. After today's hangout, there doesn't appear > to be anything remaining to discuss on this KIP. > Please vote so we can formally close this. > > Thanks, > Aditya > > > From: Aditya Auradkar > Sent: Thursday, May 21, 2015 11:26 AM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > I think we should remove the config part in TopicMetadataResponse. It's > probably cleaner if Alter and Describe are the only way to view and modify > configs but I don't feel very strongly about it. > > Re-summarizing the proposed changes to KIP-4: > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - TopicMetadataResponse shoudn't return config for the topic. DescribeConfig > is the way to go. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" as > proposed in KIP-21. > > Aditya > > ________ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 10:50 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > What about TopicMetadataResponse in KIP-4? Do we remove the config part in > it? > > Thanks, > > Jun > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > >> Hey Jun, >> >> I've added a section on error codes on the KIP-21 wiki. >> >> Here are the proposed changes to KIP-4. I'll update the wiki shortly. >> - Change AlterTopic to not allow setting configs. Config changes will flow >> through AlterConfig. CreateTopic will still allow setting configs as it is >> nice to be able to specify configs while creating the topic. >> - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" >> as proposed in KIP-21. >> >> >> Thanks, >> Aditya >> >> >> From: Jun Rao [j...@confluent.io] >> Sent: Thursday, May 21, 2015 8:41 AM >> To: dev@kafka.apache.org >> Subject: Re: [VOTE] KIP-21 Dynamic Configuration >> >> Aditya, >> >> For completeness, could you list the set of error codes in the wiki? Also, >> could you summarize the changes that are needed for the requests listed in >> KIP-4 and update the wiki accordingly? >> >> Thanks, >> >> Jun >> >> On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < >> aaurad...@linkedin.com.invalid> wrote: >> >>> Thanks Andrii. I'll make the changes. >>> >>> I've also updated KIP-21 to include the new config requests. Take a look >>> and vote. >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration >>> >>> Aditya >>> >>> From: Andrii Biletskyi [andrii.bilets...@stealth.ly] >>> Sent: Tuesday, May 19, 2015 2:26 PM >>> To: dev@kafka.apache.org >>> Subject: Re: [VOTE] KIP-21 Dynamic Configuration >>> >>> Hi, >>> >>> Sorry I wasn't able to participate. I don't have objections about >> removing >>> config changes from AlterTopic (as I understand both AddedConfig and >>> DeletedConfig) - you are welcome to update the KIP page. >>> >>> Thanks, >>> Andrii Biletskyi >>> >>> On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < >>> aaurad...@linkedin.com.invalid> wrote: >>> >>>> Updating the discussion with the latest comments. >>>> >>>> 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). >> I'll >>>> update KIP-21 with details on these. >>>> 2. Discussed during the KIP hangout. We are in agreement. >>>> >>>> (1) has a dependency on KIP-4 being completed. Rest of the work in the >>> KIP >>>> can be implemented independently. Any concerns if we tackle it as two >>>> separate work items implementation wise? >>
RE: [VOTE] KIP-21 Dynamic Configuration
bump From: Aditya Auradkar Sent: Tuesday, May 26, 2015 1:16 PM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-21 Dynamic Configuration Hey everyone, Completed the changes to KIP-4. After today's hangout, there doesn't appear to be anything remaining to discuss on this KIP. Please vote so we can formally close this. Thanks, Aditya From: Aditya Auradkar Sent: Thursday, May 21, 2015 11:26 AM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-21 Dynamic Configuration I think we should remove the config part in TopicMetadataResponse. It's probably cleaner if Alter and Describe are the only way to view and modify configs but I don't feel very strongly about it. Re-summarizing the proposed changes to KIP-4: - Change AlterTopic to not allow setting configs. Config changes will flow through AlterConfig. CreateTopic will still allow setting configs as it is nice to be able to specify configs while creating the topic. - TopicMetadataResponse shoudn't return config for the topic. DescribeConfig is the way to go. - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" as proposed in KIP-21. Aditya From: Jun Rao [j...@confluent.io] Sent: Thursday, May 21, 2015 10:50 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration What about TopicMetadataResponse in KIP-4? Do we remove the config part in it? Thanks, Jun On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hey Jun, > > I've added a section on error codes on the KIP-21 wiki. > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > > Thanks, > Aditya > > ____________ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 8:41 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Aditya, > > For completeness, could you list the set of error codes in the wiki? Also, > could you summarize the changes that are needed for the requests listed in > KIP-4 and update the wiki accordingly? > > Thanks, > > Jun > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Thanks Andrii. I'll make the changes. > > > > I've also updated KIP-21 to include the new config requests. Take a look > > and vote. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > Aditya > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > Sent: Tuesday, May 19, 2015 2:26 PM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hi, > > > > Sorry I wasn't able to participate. I don't have objections about > removing > > config changes from AlterTopic (as I understand both AddedConfig and > > DeletedConfig) - you are welcome to update the KIP page. > > > > Thanks, > > Andrii Biletskyi > > > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Updating the discussion with the latest comments. > > > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). > I'll > > > update KIP-21 with details on these. > > > 2. Discussed during the KIP hangout. We are in agreement. > > > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the > > KIP > > > can be implemented independently. Any concerns if we tackle it as two > > > separate work items implementation wise? > > > > > > We also discussed changing the AlterTopic command in KIP-4 to not > include > > > config changes. Instead, all config changes will pass through the newly > > > proposed AlterConfig. If no-one objects, I can make some changes to > KIP-4 > > > to reflect this. > > > > > > Aditya > > > > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > > Sent: Tuesday, May 19, 2015 10:51 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE]
RE: [VOTE] KIP-21 Dynamic Configuration
Hey everyone, Completed the changes to KIP-4. After today's hangout, there doesn't appear to be anything remaining to discuss on this KIP. Please vote so we can formally close this. Thanks, Aditya From: Aditya Auradkar Sent: Thursday, May 21, 2015 11:26 AM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-21 Dynamic Configuration I think we should remove the config part in TopicMetadataResponse. It's probably cleaner if Alter and Describe are the only way to view and modify configs but I don't feel very strongly about it. Re-summarizing the proposed changes to KIP-4: - Change AlterTopic to not allow setting configs. Config changes will flow through AlterConfig. CreateTopic will still allow setting configs as it is nice to be able to specify configs while creating the topic. - TopicMetadataResponse shoudn't return config for the topic. DescribeConfig is the way to go. - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" as proposed in KIP-21. Aditya From: Jun Rao [j...@confluent.io] Sent: Thursday, May 21, 2015 10:50 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration What about TopicMetadataResponse in KIP-4? Do we remove the config part in it? Thanks, Jun On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hey Jun, > > I've added a section on error codes on the KIP-21 wiki. > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > > Thanks, > Aditya > > ____________ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 8:41 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Aditya, > > For completeness, could you list the set of error codes in the wiki? Also, > could you summarize the changes that are needed for the requests listed in > KIP-4 and update the wiki accordingly? > > Thanks, > > Jun > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Thanks Andrii. I'll make the changes. > > > > I've also updated KIP-21 to include the new config requests. Take a look > > and vote. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > Aditya > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > Sent: Tuesday, May 19, 2015 2:26 PM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hi, > > > > Sorry I wasn't able to participate. I don't have objections about > removing > > config changes from AlterTopic (as I understand both AddedConfig and > > DeletedConfig) - you are welcome to update the KIP page. > > > > Thanks, > > Andrii Biletskyi > > > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Updating the discussion with the latest comments. > > > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). > I'll > > > update KIP-21 with details on these. > > > 2. Discussed during the KIP hangout. We are in agreement. > > > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the > > KIP > > > can be implemented independently. Any concerns if we tackle it as two > > > separate work items implementation wise? > > > > > > We also discussed changing the AlterTopic command in KIP-4 to not > include > > > config changes. Instead, all config changes will pass through the newly > > > proposed AlterConfig. If no-one objects, I can make some changes to > KIP-4 > > > to reflect this. > > > > > > Aditya > > > > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > > Sent: Tuesday, May 19, 2015 10:51 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hey Aditya, > > > > > > Two comments: > > > > > > 1. Yeah we need to
Re: [VOTE] KIP-21 Dynamic Configuration
+1 to the updated wiki (non-binding)
On 5/19/15, 2:26 PM, "Andrii Biletskyi"
wrote:
>Hi,
>
>Sorry I wasn't able to participate. I don't have objections about removing
>config changes from AlterTopic (as I understand both AddedConfig and
>DeletedConfig) - you are welcome to update the KIP page.
>
>Thanks,
>Andrii Biletskyi
>
>On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar <
>aaurad...@linkedin.com.invalid> wrote:
>
>> Updating the discussion with the latest comments.
>>
>> 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig).
>>I'll
>> update KIP-21 with details on these.
>> 2. Discussed during the KIP hangout. We are in agreement.
>>
>> (1) has a dependency on KIP-4 being completed. Rest of the work in the
>>KIP
>> can be implemented independently. Any concerns if we tackle it as two
>> separate work items implementation wise?
>>
>> We also discussed changing the AlterTopic command in KIP-4 to not
>>include
>> config changes. Instead, all config changes will pass through the newly
>> proposed AlterConfig. If no-one objects, I can make some changes to
>>KIP-4
>> to reflect this.
>>
>> Aditya
>>
>>
>> From: Jay Kreps [jay.kr...@gmail.com]
>> Sent: Tuesday, May 19, 2015 10:51 AM
>> To: dev@kafka.apache.org
>> Subject: Re: [VOTE] KIP-21 Dynamic Configuration
>>
>> Hey Aditya,
>>
>> Two comments:
>>
>> 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it
>>does
>> make sense to allow setting config during topic creation. I agree with
>>your
>> summary that having alter topic and alter config may be confusing, but
>> there are also some non-config changes such as replication factor and
>> partition count that alter topic can carry out. What is the final state
>>you
>> are proposing?
>>
>> 2. This is implementation related so probably can be removed from the
>>KIP
>> entirely, but you seem to be proposing a separate config manager for
>>each
>> config override type. Should we just generalize TopicConfigManager to be
>> ConfigOverrideManager and have it handle all the override types we will
>> have? I think I may just be unclear on what you are proposing...
>>
>> -Jay
>>
>> On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar <
>> aaurad...@linkedin.com.invalid> wrote:
>>
>> > Yeah, that was just a typo. I've fixed it. Thanks for calling it out.
>> >
>> > In KIP-4, I believe we have 3 types of requests: CreateTopic,
>>AlterTopic
>> > and DeleteTopic. The topic configs are a sub-type of the Create and
>>Alter
>> > commands. I think it would be nice to simply have a AlterConfig
>>command
>> > that can alter any type of config rather than having a specific
>> > ClientConfig.
>> >
>> > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]]
>> > ConfigType => string
>> > AddedConfigEntry => ConfigKey ConfigValue
>> > ConfigKey => string
>> > ConfigValue => string
>> > DeletedConfig => string
>> >
>> > The downside of this approach is that we will have 2 separate ways of
>> > changing topic configs (AlterTopic and AlterConfig). While a general
>> > AlterConfig only makes sense if we plan to have more than two types of
>> > entity configs.. it's definitely more future proof. Thoughts?
>> >
>> > Aditya
>> >
>> >
>> > From: Todd Palino [tpal...@gmail.com]
>> > Sent: Monday, May 18, 2015 12:39 PM
>> > To: dev@kafka.apache.org
>> > Subject: Re: [VOTE] KIP-21 Dynamic Configuration
>> >
>> > Agree with Jun here on the JSON format. I think your intention was
>>likely
>> > to have actual JSON here and it was just a typo in the wiki?
>> >
>> > -Todd
>> >
>> > On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
>> >
>> > > Aditya,
>> > >
>> > > Another thing to consider. In KIP-4, we are adding a new RPC
>>request to
>> > > change and retrieve topic configs. Do we want to add a similar RPC
>> > request
>> > > to change configs per client id? If so, do we want to introduce a
>> > separate
>> > > new request or have a combined new request for both topic and
>>client id
>> > > level config changes?
>> > >
>> > > A minor point in the wiki, for the json format in ZK, we should
>>change
>> > > {X1=Y1,
>> > > X2=Y2..} to a json map, right?
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
>> > > aaurad...@linkedin.com.invalid> wrote:
>> > >
>> > > >
>> > > >
>> > >
>> >
>>
>>https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Config
>>uration
>> > > >
>> > > > Aditya
>> > > >
>> > >
>> >
>>
RE: [VOTE] KIP-21 Dynamic Configuration
I think we should remove the config part in TopicMetadataResponse. It's probably cleaner if Alter and Describe are the only way to view and modify configs but I don't feel very strongly about it. Re-summarizing the proposed changes to KIP-4: - Change AlterTopic to not allow setting configs. Config changes will flow through AlterConfig. CreateTopic will still allow setting configs as it is nice to be able to specify configs while creating the topic. - TopicMetadataResponse shoudn't return config for the topic. DescribeConfig is the way to go. - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" as proposed in KIP-21. Aditya From: Jun Rao [j...@confluent.io] Sent: Thursday, May 21, 2015 10:50 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration What about TopicMetadataResponse in KIP-4? Do we remove the config part in it? Thanks, Jun On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hey Jun, > > I've added a section on error codes on the KIP-21 wiki. > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > > Thanks, > Aditya > > ________ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 8:41 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Aditya, > > For completeness, could you list the set of error codes in the wiki? Also, > could you summarize the changes that are needed for the requests listed in > KIP-4 and update the wiki accordingly? > > Thanks, > > Jun > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Thanks Andrii. I'll make the changes. > > > > I've also updated KIP-21 to include the new config requests. Take a look > > and vote. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > Aditya > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > Sent: Tuesday, May 19, 2015 2:26 PM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hi, > > > > Sorry I wasn't able to participate. I don't have objections about > removing > > config changes from AlterTopic (as I understand both AddedConfig and > > DeletedConfig) - you are welcome to update the KIP page. > > > > Thanks, > > Andrii Biletskyi > > > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Updating the discussion with the latest comments. > > > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). > I'll > > > update KIP-21 with details on these. > > > 2. Discussed during the KIP hangout. We are in agreement. > > > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the > > KIP > > > can be implemented independently. Any concerns if we tackle it as two > > > separate work items implementation wise? > > > > > > We also discussed changing the AlterTopic command in KIP-4 to not > include > > > config changes. Instead, all config changes will pass through the newly > > > proposed AlterConfig. If no-one objects, I can make some changes to > KIP-4 > > > to reflect this. > > > > > > Aditya > > > > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > > Sent: Tuesday, May 19, 2015 10:51 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hey Aditya, > > > > > > Two comments: > > > > > > 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it > does > > > make sense to allow setting config during topic creation. I agree with > > your > > > summary that having alter topic and alter config may be confusing, but > > > there are also some non-config changes such as replication factor and > > > partition count that alter topic can carry out. What is
Re: [VOTE] KIP-21 Dynamic Configuration
What about TopicMetadataResponse in KIP-4? Do we remove the config part in it? Thanks, Jun On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hey Jun, > > I've added a section on error codes on the KIP-21 wiki. > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > > Thanks, > Aditya > > > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 8:41 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Aditya, > > For completeness, could you list the set of error codes in the wiki? Also, > could you summarize the changes that are needed for the requests listed in > KIP-4 and update the wiki accordingly? > > Thanks, > > Jun > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Thanks Andrii. I'll make the changes. > > > > I've also updated KIP-21 to include the new config requests. Take a look > > and vote. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > Aditya > > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > Sent: Tuesday, May 19, 2015 2:26 PM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hi, > > > > Sorry I wasn't able to participate. I don't have objections about > removing > > config changes from AlterTopic (as I understand both AddedConfig and > > DeletedConfig) - you are welcome to update the KIP page. > > > > Thanks, > > Andrii Biletskyi > > > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Updating the discussion with the latest comments. > > > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). > I'll > > > update KIP-21 with details on these. > > > 2. Discussed during the KIP hangout. We are in agreement. > > > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the > > KIP > > > can be implemented independently. Any concerns if we tackle it as two > > > separate work items implementation wise? > > > > > > We also discussed changing the AlterTopic command in KIP-4 to not > include > > > config changes. Instead, all config changes will pass through the newly > > > proposed AlterConfig. If no-one objects, I can make some changes to > KIP-4 > > > to reflect this. > > > > > > Aditya > > > > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > > Sent: Tuesday, May 19, 2015 10:51 AM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hey Aditya, > > > > > > Two comments: > > > > > > 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it > does > > > make sense to allow setting config during topic creation. I agree with > > your > > > summary that having alter topic and alter config may be confusing, but > > > there are also some non-config changes such as replication factor and > > > partition count that alter topic can carry out. What is the final state > > you > > > are proposing? > > > > > > 2. This is implementation related so probably can be removed from the > KIP > > > entirely, but you seem to be proposing a separate config manager for > each > > > config override type. Should we just generalize TopicConfigManager to > be > > > ConfigOverrideManager and have it handle all the override types we will > > > have? I think I may just be unclear on what you are proposing... > > > > > > -Jay > > > > > > On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > Yeah, that was just a typo. I've fixed it. Thanks for calling it out. > > > > > > > > In KIP-4, I believe we have 3 types
RE: [VOTE] KIP-21 Dynamic Configuration
Hey Jun, I've added a section on error codes on the KIP-21 wiki. Here are the proposed changes to KIP-4. I'll update the wiki shortly. - Change AlterTopic to not allow setting configs. Config changes will flow through AlterConfig. CreateTopic will still allow setting configs as it is nice to be able to specify configs while creating the topic. - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" as proposed in KIP-21. Thanks, Aditya From: Jun Rao [j...@confluent.io] Sent: Thursday, May 21, 2015 8:41 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-21 Dynamic Configuration Aditya, For completeness, could you list the set of error codes in the wiki? Also, could you summarize the changes that are needed for the requests listed in KIP-4 and update the wiki accordingly? Thanks, Jun On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Thanks Andrii. I'll make the changes. > > I've also updated KIP-21 to include the new config requests. Take a look > and vote. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > Aditya > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > Sent: Tuesday, May 19, 2015 2:26 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Hi, > > Sorry I wasn't able to participate. I don't have objections about removing > config changes from AlterTopic (as I understand both AddedConfig and > DeletedConfig) - you are welcome to update the KIP page. > > Thanks, > Andrii Biletskyi > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Updating the discussion with the latest comments. > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll > > update KIP-21 with details on these. > > 2. Discussed during the KIP hangout. We are in agreement. > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the > KIP > > can be implemented independently. Any concerns if we tackle it as two > > separate work items implementation wise? > > > > We also discussed changing the AlterTopic command in KIP-4 to not include > > config changes. Instead, all config changes will pass through the newly > > proposed AlterConfig. If no-one objects, I can make some changes to KIP-4 > > to reflect this. > > > > Aditya > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > Sent: Tuesday, May 19, 2015 10:51 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hey Aditya, > > > > Two comments: > > > > 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does > > make sense to allow setting config during topic creation. I agree with > your > > summary that having alter topic and alter config may be confusing, but > > there are also some non-config changes such as replication factor and > > partition count that alter topic can carry out. What is the final state > you > > are proposing? > > > > 2. This is implementation related so probably can be removed from the KIP > > entirely, but you seem to be proposing a separate config manager for each > > config override type. Should we just generalize TopicConfigManager to be > > ConfigOverrideManager and have it handle all the override types we will > > have? I think I may just be unclear on what you are proposing... > > > > -Jay > > > > On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Yeah, that was just a typo. I've fixed it. Thanks for calling it out. > > > > > > In KIP-4, I believe we have 3 types of requests: CreateTopic, > AlterTopic > > > and DeleteTopic. The topic configs are a sub-type of the Create and > Alter > > > commands. I think it would be nice to simply have a AlterConfig command > > > that can alter any type of config rather than having a specific > > > ClientConfig. > > > > > > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]] > > > ConfigType => string > > > AddedConfigEntry => ConfigKey ConfigValue > > > ConfigKey => string > > > ConfigValue => string > > > DeletedConfig => string > > > > > > The downside of this approach is that we will have 2 separate ways of > >
Re: [VOTE] KIP-21 Dynamic Configuration
Aditya, For completeness, could you list the set of error codes in the wiki? Also, could you summarize the changes that are needed for the requests listed in KIP-4 and update the wiki accordingly? Thanks, Jun On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Thanks Andrii. I'll make the changes. > > I've also updated KIP-21 to include the new config requests. Take a look > and vote. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > Aditya > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > Sent: Tuesday, May 19, 2015 2:26 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Hi, > > Sorry I wasn't able to participate. I don't have objections about removing > config changes from AlterTopic (as I understand both AddedConfig and > DeletedConfig) - you are welcome to update the KIP page. > > Thanks, > Andrii Biletskyi > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Updating the discussion with the latest comments. > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll > > update KIP-21 with details on these. > > 2. Discussed during the KIP hangout. We are in agreement. > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the > KIP > > can be implemented independently. Any concerns if we tackle it as two > > separate work items implementation wise? > > > > We also discussed changing the AlterTopic command in KIP-4 to not include > > config changes. Instead, all config changes will pass through the newly > > proposed AlterConfig. If no-one objects, I can make some changes to KIP-4 > > to reflect this. > > > > Aditya > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > Sent: Tuesday, May 19, 2015 10:51 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hey Aditya, > > > > Two comments: > > > > 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does > > make sense to allow setting config during topic creation. I agree with > your > > summary that having alter topic and alter config may be confusing, but > > there are also some non-config changes such as replication factor and > > partition count that alter topic can carry out. What is the final state > you > > are proposing? > > > > 2. This is implementation related so probably can be removed from the KIP > > entirely, but you seem to be proposing a separate config manager for each > > config override type. Should we just generalize TopicConfigManager to be > > ConfigOverrideManager and have it handle all the override types we will > > have? I think I may just be unclear on what you are proposing... > > > > -Jay > > > > On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Yeah, that was just a typo. I've fixed it. Thanks for calling it out. > > > > > > In KIP-4, I believe we have 3 types of requests: CreateTopic, > AlterTopic > > > and DeleteTopic. The topic configs are a sub-type of the Create and > Alter > > > commands. I think it would be nice to simply have a AlterConfig command > > > that can alter any type of config rather than having a specific > > > ClientConfig. > > > > > > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]] > > > ConfigType => string > > > AddedConfigEntry => ConfigKey ConfigValue > > > ConfigKey => string > > > ConfigValue => string > > > DeletedConfig => string > > > > > > The downside of this approach is that we will have 2 separate ways of > > > changing topic configs (AlterTopic and AlterConfig). While a general > > > AlterConfig only makes sense if we plan to have more than two types of > > > entity configs.. it's definitely more future proof. Thoughts? > > > > > > Aditya > > > > > > > > > From: Todd Palino [tpal...@gmail.com] > > > Sent: Monday, May 18, 2015 12:39 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Agree with Jun here on the JSON format. I think your intention was > likely > > &g
Re: [VOTE] KIP-21 Dynamic Configuration
+1 On Wed, May 20, 2015 at 05:33:31AM +, Aditya Auradkar wrote: > Thanks Andrii. I'll make the changes. > > I've also updated KIP-21 to include the new config requests. Take a look and > vote. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > Aditya > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > Sent: Tuesday, May 19, 2015 2:26 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > Hi, > > Sorry I wasn't able to participate. I don't have objections about removing > config changes from AlterTopic (as I understand both AddedConfig and > DeletedConfig) - you are welcome to update the KIP page. > > Thanks, > Andrii Biletskyi > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Updating the discussion with the latest comments. > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll > > update KIP-21 with details on these. > > 2. Discussed during the KIP hangout. We are in agreement. > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in the KIP > > can be implemented independently. Any concerns if we tackle it as two > > separate work items implementation wise? > > > > We also discussed changing the AlterTopic command in KIP-4 to not include > > config changes. Instead, all config changes will pass through the newly > > proposed AlterConfig. If no-one objects, I can make some changes to KIP-4 > > to reflect this. > > > > Aditya > > > > > > From: Jay Kreps [jay.kr...@gmail.com] > > Sent: Tuesday, May 19, 2015 10:51 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Hey Aditya, > > > > Two comments: > > > > 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does > > make sense to allow setting config during topic creation. I agree with your > > summary that having alter topic and alter config may be confusing, but > > there are also some non-config changes such as replication factor and > > partition count that alter topic can carry out. What is the final state you > > are proposing? > > > > 2. This is implementation related so probably can be removed from the KIP > > entirely, but you seem to be proposing a separate config manager for each > > config override type. Should we just generalize TopicConfigManager to be > > ConfigOverrideManager and have it handle all the override types we will > > have? I think I may just be unclear on what you are proposing... > > > > -Jay > > > > On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Yeah, that was just a typo. I've fixed it. Thanks for calling it out. > > > > > > In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic > > > and DeleteTopic. The topic configs are a sub-type of the Create and Alter > > > commands. I think it would be nice to simply have a AlterConfig command > > > that can alter any type of config rather than having a specific > > > ClientConfig. > > > > > > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]] > > > ConfigType => string > > > AddedConfigEntry => ConfigKey ConfigValue > > > ConfigKey => string > > > ConfigValue => string > > > DeletedConfig => string > > > > > > The downside of this approach is that we will have 2 separate ways of > > > changing topic configs (AlterTopic and AlterConfig). While a general > > > AlterConfig only makes sense if we plan to have more than two types of > > > entity configs.. it's definitely more future proof. Thoughts? > > > > > > Aditya > > > > > > > > > From: Todd Palino [tpal...@gmail.com] > > > Sent: Monday, May 18, 2015 12:39 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Agree with Jun here on the JSON format. I think your intention was likely > > > to have actual JSON here and it was just a typo in the wiki? > > > > > > -Todd > > > > > > On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote: > > > > > > > Aditya, > > > > > > > > Another thing
RE: [VOTE] KIP-21 Dynamic Configuration
Thanks Andrii. I'll make the changes.
I've also updated KIP-21 to include the new config requests. Take a look and
vote.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
Aditya
From: Andrii Biletskyi [andrii.bilets...@stealth.ly]
Sent: Tuesday, May 19, 2015 2:26 PM
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-21 Dynamic Configuration
Hi,
Sorry I wasn't able to participate. I don't have objections about removing
config changes from AlterTopic (as I understand both AddedConfig and
DeletedConfig) - you are welcome to update the KIP page.
Thanks,
Andrii Biletskyi
On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar <
aaurad...@linkedin.com.invalid> wrote:
> Updating the discussion with the latest comments.
>
> 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll
> update KIP-21 with details on these.
> 2. Discussed during the KIP hangout. We are in agreement.
>
> (1) has a dependency on KIP-4 being completed. Rest of the work in the KIP
> can be implemented independently. Any concerns if we tackle it as two
> separate work items implementation wise?
>
> We also discussed changing the AlterTopic command in KIP-4 to not include
> config changes. Instead, all config changes will pass through the newly
> proposed AlterConfig. If no-one objects, I can make some changes to KIP-4
> to reflect this.
>
> Aditya
>
>
> From: Jay Kreps [jay.kr...@gmail.com]
> Sent: Tuesday, May 19, 2015 10:51 AM
> To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-21 Dynamic Configuration
>
> Hey Aditya,
>
> Two comments:
>
> 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does
> make sense to allow setting config during topic creation. I agree with your
> summary that having alter topic and alter config may be confusing, but
> there are also some non-config changes such as replication factor and
> partition count that alter topic can carry out. What is the final state you
> are proposing?
>
> 2. This is implementation related so probably can be removed from the KIP
> entirely, but you seem to be proposing a separate config manager for each
> config override type. Should we just generalize TopicConfigManager to be
> ConfigOverrideManager and have it handle all the override types we will
> have? I think I may just be unclear on what you are proposing...
>
> -Jay
>
> On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar <
> aaurad...@linkedin.com.invalid> wrote:
>
> > Yeah, that was just a typo. I've fixed it. Thanks for calling it out.
> >
> > In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic
> > and DeleteTopic. The topic configs are a sub-type of the Create and Alter
> > commands. I think it would be nice to simply have a AlterConfig command
> > that can alter any type of config rather than having a specific
> > ClientConfig.
> >
> > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]]
> > ConfigType => string
> > AddedConfigEntry => ConfigKey ConfigValue
> > ConfigKey => string
> > ConfigValue => string
> > DeletedConfig => string
> >
> > The downside of this approach is that we will have 2 separate ways of
> > changing topic configs (AlterTopic and AlterConfig). While a general
> > AlterConfig only makes sense if we plan to have more than two types of
> > entity configs.. it's definitely more future proof. Thoughts?
> >
> > Aditya
> >
> >
> > From: Todd Palino [tpal...@gmail.com]
> > Sent: Monday, May 18, 2015 12:39 PM
> > To: dev@kafka.apache.org
> > Subject: Re: [VOTE] KIP-21 Dynamic Configuration
> >
> > Agree with Jun here on the JSON format. I think your intention was likely
> > to have actual JSON here and it was just a typo in the wiki?
> >
> > -Todd
> >
> > On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
> >
> > > Aditya,
> > >
> > > Another thing to consider. In KIP-4, we are adding a new RPC request to
> > > change and retrieve topic configs. Do we want to add a similar RPC
> > request
> > > to change configs per client id? If so, do we want to introduce a
> > separate
> > > new request or have a combined new request for both topic and client id
> > > level config changes?
> > >
> > > A minor point in the wiki, for the json format in ZK, we should change
> > > {X1=Y1,
> > > X2=Y2..} to a json map, right?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
> > > aaurad...@linkedin.com.invalid> wrote:
> > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
> > > >
> > > > Aditya
> > > >
> > >
> >
>
Re: [VOTE] KIP-21 Dynamic Configuration
Hi,
Sorry I wasn't able to participate. I don't have objections about removing
config changes from AlterTopic (as I understand both AddedConfig and
DeletedConfig) - you are welcome to update the KIP page.
Thanks,
Andrii Biletskyi
On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar <
aaurad...@linkedin.com.invalid> wrote:
> Updating the discussion with the latest comments.
>
> 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll
> update KIP-21 with details on these.
> 2. Discussed during the KIP hangout. We are in agreement.
>
> (1) has a dependency on KIP-4 being completed. Rest of the work in the KIP
> can be implemented independently. Any concerns if we tackle it as two
> separate work items implementation wise?
>
> We also discussed changing the AlterTopic command in KIP-4 to not include
> config changes. Instead, all config changes will pass through the newly
> proposed AlterConfig. If no-one objects, I can make some changes to KIP-4
> to reflect this.
>
> Aditya
>
>
> From: Jay Kreps [jay.kr...@gmail.com]
> Sent: Tuesday, May 19, 2015 10:51 AM
> To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-21 Dynamic Configuration
>
> Hey Aditya,
>
> Two comments:
>
> 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does
> make sense to allow setting config during topic creation. I agree with your
> summary that having alter topic and alter config may be confusing, but
> there are also some non-config changes such as replication factor and
> partition count that alter topic can carry out. What is the final state you
> are proposing?
>
> 2. This is implementation related so probably can be removed from the KIP
> entirely, but you seem to be proposing a separate config manager for each
> config override type. Should we just generalize TopicConfigManager to be
> ConfigOverrideManager and have it handle all the override types we will
> have? I think I may just be unclear on what you are proposing...
>
> -Jay
>
> On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar <
> aaurad...@linkedin.com.invalid> wrote:
>
> > Yeah, that was just a typo. I've fixed it. Thanks for calling it out.
> >
> > In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic
> > and DeleteTopic. The topic configs are a sub-type of the Create and Alter
> > commands. I think it would be nice to simply have a AlterConfig command
> > that can alter any type of config rather than having a specific
> > ClientConfig.
> >
> > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]]
> > ConfigType => string
> > AddedConfigEntry => ConfigKey ConfigValue
> > ConfigKey => string
> > ConfigValue => string
> > DeletedConfig => string
> >
> > The downside of this approach is that we will have 2 separate ways of
> > changing topic configs (AlterTopic and AlterConfig). While a general
> > AlterConfig only makes sense if we plan to have more than two types of
> > entity configs.. it's definitely more future proof. Thoughts?
> >
> > Aditya
> >
> >
> > From: Todd Palino [tpal...@gmail.com]
> > Sent: Monday, May 18, 2015 12:39 PM
> > To: dev@kafka.apache.org
> > Subject: Re: [VOTE] KIP-21 Dynamic Configuration
> >
> > Agree with Jun here on the JSON format. I think your intention was likely
> > to have actual JSON here and it was just a typo in the wiki?
> >
> > -Todd
> >
> > On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
> >
> > > Aditya,
> > >
> > > Another thing to consider. In KIP-4, we are adding a new RPC request to
> > > change and retrieve topic configs. Do we want to add a similar RPC
> > request
> > > to change configs per client id? If so, do we want to introduce a
> > separate
> > > new request or have a combined new request for both topic and client id
> > > level config changes?
> > >
> > > A minor point in the wiki, for the json format in ZK, we should change
> > > {X1=Y1,
> > > X2=Y2..} to a json map, right?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
> > > aaurad...@linkedin.com.invalid> wrote:
> > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
> > > >
> > > > Aditya
> > > >
> > >
> >
>
RE: [VOTE] KIP-21 Dynamic Configuration
Updating the discussion with the latest comments.
1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll
update KIP-21 with details on these.
2. Discussed during the KIP hangout. We are in agreement.
(1) has a dependency on KIP-4 being completed. Rest of the work in the KIP can
be implemented independently. Any concerns if we tackle it as two separate work
items implementation wise?
We also discussed changing the AlterTopic command in KIP-4 to not include
config changes. Instead, all config changes will pass through the newly
proposed AlterConfig. If no-one objects, I can make some changes to KIP-4 to
reflect this.
Aditya
From: Jay Kreps [jay.kr...@gmail.com]
Sent: Tuesday, May 19, 2015 10:51 AM
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-21 Dynamic Configuration
Hey Aditya,
Two comments:
1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does
make sense to allow setting config during topic creation. I agree with your
summary that having alter topic and alter config may be confusing, but
there are also some non-config changes such as replication factor and
partition count that alter topic can carry out. What is the final state you
are proposing?
2. This is implementation related so probably can be removed from the KIP
entirely, but you seem to be proposing a separate config manager for each
config override type. Should we just generalize TopicConfigManager to be
ConfigOverrideManager and have it handle all the override types we will
have? I think I may just be unclear on what you are proposing...
-Jay
On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar <
aaurad...@linkedin.com.invalid> wrote:
> Yeah, that was just a typo. I've fixed it. Thanks for calling it out.
>
> In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic
> and DeleteTopic. The topic configs are a sub-type of the Create and Alter
> commands. I think it would be nice to simply have a AlterConfig command
> that can alter any type of config rather than having a specific
> ClientConfig.
>
> AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]]
> ConfigType => string
> AddedConfigEntry => ConfigKey ConfigValue
> ConfigKey => string
> ConfigValue => string
> DeletedConfig => string
>
> The downside of this approach is that we will have 2 separate ways of
> changing topic configs (AlterTopic and AlterConfig). While a general
> AlterConfig only makes sense if we plan to have more than two types of
> entity configs.. it's definitely more future proof. Thoughts?
>
> Aditya
>
>
> From: Todd Palino [tpal...@gmail.com]
> Sent: Monday, May 18, 2015 12:39 PM
> To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-21 Dynamic Configuration
>
> Agree with Jun here on the JSON format. I think your intention was likely
> to have actual JSON here and it was just a typo in the wiki?
>
> -Todd
>
> On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
>
> > Aditya,
> >
> > Another thing to consider. In KIP-4, we are adding a new RPC request to
> > change and retrieve topic configs. Do we want to add a similar RPC
> request
> > to change configs per client id? If so, do we want to introduce a
> separate
> > new request or have a combined new request for both topic and client id
> > level config changes?
> >
> > A minor point in the wiki, for the json format in ZK, we should change
> > {X1=Y1,
> > X2=Y2..} to a json map, right?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
> > aaurad...@linkedin.com.invalid> wrote:
> >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
> > >
> > > Aditya
> > >
> >
>
Re: [VOTE] KIP-21 Dynamic Configuration
Hey Aditya,
Two comments:
1. Yeah we need to reconcile this with the APIs in KIP-4. I think it does
make sense to allow setting config during topic creation. I agree with your
summary that having alter topic and alter config may be confusing, but
there are also some non-config changes such as replication factor and
partition count that alter topic can carry out. What is the final state you
are proposing?
2. This is implementation related so probably can be removed from the KIP
entirely, but you seem to be proposing a separate config manager for each
config override type. Should we just generalize TopicConfigManager to be
ConfigOverrideManager and have it handle all the override types we will
have? I think I may just be unclear on what you are proposing...
-Jay
On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar <
aaurad...@linkedin.com.invalid> wrote:
> Yeah, that was just a typo. I've fixed it. Thanks for calling it out.
>
> In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic
> and DeleteTopic. The topic configs are a sub-type of the Create and Alter
> commands. I think it would be nice to simply have a AlterConfig command
> that can alter any type of config rather than having a specific
> ClientConfig.
>
> AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]]
> ConfigType => string
> AddedConfigEntry => ConfigKey ConfigValue
> ConfigKey => string
> ConfigValue => string
> DeletedConfig => string
>
> The downside of this approach is that we will have 2 separate ways of
> changing topic configs (AlterTopic and AlterConfig). While a general
> AlterConfig only makes sense if we plan to have more than two types of
> entity configs.. it's definitely more future proof. Thoughts?
>
> Aditya
>
>
> From: Todd Palino [tpal...@gmail.com]
> Sent: Monday, May 18, 2015 12:39 PM
> To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-21 Dynamic Configuration
>
> Agree with Jun here on the JSON format. I think your intention was likely
> to have actual JSON here and it was just a typo in the wiki?
>
> -Todd
>
> On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
>
> > Aditya,
> >
> > Another thing to consider. In KIP-4, we are adding a new RPC request to
> > change and retrieve topic configs. Do we want to add a similar RPC
> request
> > to change configs per client id? If so, do we want to introduce a
> separate
> > new request or have a combined new request for both topic and client id
> > level config changes?
> >
> > A minor point in the wiki, for the json format in ZK, we should change
> > {X1=Y1,
> > X2=Y2..} to a json map, right?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
> > aaurad...@linkedin.com.invalid> wrote:
> >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
> > >
> > > Aditya
> > >
> >
>
RE: [VOTE] KIP-21 Dynamic Configuration
Yeah, that was just a typo. I've fixed it. Thanks for calling it out.
In KIP-4, I believe we have 3 types of requests: CreateTopic, AlterTopic and
DeleteTopic. The topic configs are a sub-type of the Create and Alter commands.
I think it would be nice to simply have a AlterConfig command that can alter
any type of config rather than having a specific ClientConfig.
AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]]
ConfigType => string
AddedConfigEntry => ConfigKey ConfigValue
ConfigKey => string
ConfigValue => string
DeletedConfig => string
The downside of this approach is that we will have 2 separate ways of changing
topic configs (AlterTopic and AlterConfig). While a general AlterConfig only
makes sense if we plan to have more than two types of entity configs.. it's
definitely more future proof. Thoughts?
Aditya
From: Todd Palino [tpal...@gmail.com]
Sent: Monday, May 18, 2015 12:39 PM
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-21 Dynamic Configuration
Agree with Jun here on the JSON format. I think your intention was likely
to have actual JSON here and it was just a typo in the wiki?
-Todd
On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
> Aditya,
>
> Another thing to consider. In KIP-4, we are adding a new RPC request to
> change and retrieve topic configs. Do we want to add a similar RPC request
> to change configs per client id? If so, do we want to introduce a separate
> new request or have a combined new request for both topic and client id
> level config changes?
>
> A minor point in the wiki, for the json format in ZK, we should change
> {X1=Y1,
> X2=Y2..} to a json map, right?
>
> Thanks,
>
> Jun
>
>
> On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
> aaurad...@linkedin.com.invalid> wrote:
>
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
> >
> > Aditya
> >
>
Re: [VOTE] KIP-21 Dynamic Configuration
Agree with Jun here on the JSON format. I think your intention was likely
to have actual JSON here and it was just a typo in the wiki?
-Todd
On Mon, May 18, 2015 at 12:07 PM, Jun Rao wrote:
> Aditya,
>
> Another thing to consider. In KIP-4, we are adding a new RPC request to
> change and retrieve topic configs. Do we want to add a similar RPC request
> to change configs per client id? If so, do we want to introduce a separate
> new request or have a combined new request for both topic and client id
> level config changes?
>
> A minor point in the wiki, for the json format in ZK, we should change
> {X1=Y1,
> X2=Y2..} to a json map, right?
>
> Thanks,
>
> Jun
>
>
> On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
> aaurad...@linkedin.com.invalid> wrote:
>
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
> >
> > Aditya
> >
>
Re: [VOTE] KIP-21 Dynamic Configuration
Aditya,
Another thing to consider. In KIP-4, we are adding a new RPC request to
change and retrieve topic configs. Do we want to add a similar RPC request
to change configs per client id? If so, do we want to introduce a separate
new request or have a combined new request for both topic and client id
level config changes?
A minor point in the wiki, for the json format in ZK, we should change {X1=Y1,
X2=Y2..} to a json map, right?
Thanks,
Jun
On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar <
aaurad...@linkedin.com.invalid> wrote:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration
>
> Aditya
>
