I still have a couple of questions: 1. Are we introducing a new Java API for the config change protocol and if so where will that appear? Is that going to be part of the java api in the admin api kip? Let's document that. 2. The proposed JSON format uses camel case for field names, is that what we've used for other JSON in zookeeper? 3. This changes the format of the notifications, right? How will we grandfather in the old format? Clusters will have existing change notifications in the old format so I think the new code will need to be able to read those?
-Jay On Thu, May 28, 2015 at 11:41 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > bump > > ________________________________________ > From: Aditya Auradkar > Sent: Tuesday, May 26, 2015 1:16 PM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > Hey everyone, > > Completed the changes to KIP-4. After today's hangout, there doesn't > appear to be anything remaining to discuss on this KIP. > Please vote so we can formally close this. > > Thanks, > Aditya > > ________________________________________ > From: Aditya Auradkar > Sent: Thursday, May 21, 2015 11:26 AM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-21 Dynamic Configuration > > I think we should remove the config part in TopicMetadataResponse. It's > probably cleaner if Alter and Describe are the only way to view and modify > configs but I don't feel very strongly about it. > > Re-summarizing the proposed changes to KIP-4: > - Change AlterTopic to not allow setting configs. Config changes will flow > through AlterConfig. CreateTopic will still allow setting configs as it is > nice to be able to specify configs while creating the topic. > - TopicMetadataResponse shoudn't return config for the topic. > DescribeConfig is the way to go. > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > as proposed in KIP-21. > > Aditya > > ________________________________________ > From: Jun Rao [j...@confluent.io] > Sent: Thursday, May 21, 2015 10:50 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > What about TopicMetadataResponse in KIP-4? Do we remove the config part in > it? > > Thanks, > > Jun > > On Thu, May 21, 2015 at 10:25 AM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > Hey Jun, > > > > I've added a section on error codes on the KIP-21 wiki. > > > > Here are the proposed changes to KIP-4. I'll update the wiki shortly. > > - Change AlterTopic to not allow setting configs. Config changes will > flow > > through AlterConfig. CreateTopic will still allow setting configs as it > is > > nice to be able to specify configs while creating the topic. > > - Change "InvalidTopicConfiguration" error code to "InvalidEntityConfig" > > as proposed in KIP-21. > > > > > > Thanks, > > Aditya > > > > ________________________________________ > > From: Jun Rao [j...@confluent.io] > > Sent: Thursday, May 21, 2015 8:41 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > Aditya, > > > > For completeness, could you list the set of error codes in the wiki? > Also, > > could you summarize the changes that are needed for the requests listed > in > > KIP-4 and update the wiki accordingly? > > > > Thanks, > > > > Jun > > > > On Tue, May 19, 2015 at 10:33 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Thanks Andrii. I'll make the changes. > > > > > > I've also updated KIP-21 to include the new config requests. Take a > look > > > and vote. > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > > > Aditya > > > ________________________________________ > > > From: Andrii Biletskyi [andrii.bilets...@stealth.ly] > > > Sent: Tuesday, May 19, 2015 2:26 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > Hi, > > > > > > Sorry I wasn't able to participate. I don't have objections about > > removing > > > config changes from AlterTopic (as I understand both AddedConfig and > > > DeletedConfig) - you are welcome to update the KIP page. > > > > > > Thanks, > > > Andrii Biletskyi > > > > > > On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar < > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > Updating the discussion with the latest comments. > > > > > > > > 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). > > I'll > > > > update KIP-21 with details on these. > > > > 2. Discussed during the KIP hangout. We are in agreement. > > > > > > > > (1) has a dependency on KIP-4 being completed. Rest of the work in > the > > > KIP > > > > can be implemented independently. Any concerns if we tackle it as two > > > > separate work items implementation wise? > > > > > > > > We also discussed changing the AlterTopic command in KIP-4 to not > > include > > > > config changes. Instead, all config changes will pass through the > newly > > > > proposed AlterConfig. If no-one objects, I can make some changes to > > KIP-4 > > > > to reflect this. > > > > > > > > Aditya > > > > > > > > ________________________________________ > > > > From: Jay Kreps [jay.kr...@gmail.com] > > > > Sent: Tuesday, May 19, 2015 10:51 AM > > > > To: dev@kafka.apache.org > > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > > > Hey Aditya, > > > > > > > > Two comments: > > > > > > > > 1. Yeah we need to reconcile this with the APIs in KIP-4. I think it > > does > > > > make sense to allow setting config during topic creation. I agree > with > > > your > > > > summary that having alter topic and alter config may be confusing, > but > > > > there are also some non-config changes such as replication factor and > > > > partition count that alter topic can carry out. What is the final > state > > > you > > > > are proposing? > > > > > > > > 2. This is implementation related so probably can be removed from the > > KIP > > > > entirely, but you seem to be proposing a separate config manager for > > each > > > > config override type. Should we just generalize TopicConfigManager to > > be > > > > ConfigOverrideManager and have it handle all the override types we > will > > > > have? I think I may just be unclear on what you are proposing... > > > > > > > > -Jay > > > > > > > > On Mon, May 18, 2015 at 1:34 PM, Aditya Auradkar < > > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > > > Yeah, that was just a typo. I've fixed it. Thanks for calling it > out. > > > > > > > > > > In KIP-4, I believe we have 3 types of requests: CreateTopic, > > > AlterTopic > > > > > and DeleteTopic. The topic configs are a sub-type of the Create and > > > Alter > > > > > commands. I think it would be nice to simply have a AlterConfig > > command > > > > > that can alter any type of config rather than having a specific > > > > > ClientConfig. > > > > > > > > > > AlterConfig => [ConfigType [AddedConfigEntry] [DeletedConfig]] > > > > > ConfigType => string > > > > > AddedConfigEntry => ConfigKey ConfigValue > > > > > ConfigKey => string > > > > > ConfigValue => string > > > > > DeletedConfig => string > > > > > > > > > > The downside of this approach is that we will have 2 separate ways > of > > > > > changing topic configs (AlterTopic and AlterConfig). While a > general > > > > > AlterConfig only makes sense if we plan to have more than two types > > of > > > > > entity configs.. it's definitely more future proof. Thoughts? > > > > > > > > > > Aditya > > > > > > > > > > ________________________________________ > > > > > From: Todd Palino [tpal...@gmail.com] > > > > > Sent: Monday, May 18, 2015 12:39 PM > > > > > To: dev@kafka.apache.org > > > > > Subject: Re: [VOTE] KIP-21 Dynamic Configuration > > > > > > > > > > Agree with Jun here on the JSON format. I think your intention was > > > likely > > > > > to have actual JSON here and it was just a typo in the wiki? > > > > > > > > > > -Todd > > > > > > > > > > On Mon, May 18, 2015 at 12:07 PM, Jun Rao <j...@confluent.io> > wrote: > > > > > > > > > > > Aditya, > > > > > > > > > > > > Another thing to consider. In KIP-4, we are adding a new RPC > > request > > > to > > > > > > change and retrieve topic configs. Do we want to add a similar > RPC > > > > > request > > > > > > to change configs per client id? If so, do we want to introduce a > > > > > separate > > > > > > new request or have a combined new request for both topic and > > client > > > id > > > > > > level config changes? > > > > > > > > > > > > A minor point in the wiki, for the json format in ZK, we should > > > change > > > > > > {X1=Y1, > > > > > > X2=Y2..} to a json map, right? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Mon, May 18, 2015 at 9:48 AM, Aditya Auradkar < > > > > > > aaurad...@linkedin.com.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration > > > > > > > > > > > > > > Aditya > > > > > > > > > > > > > > > > > > > > > > > > > > > >
