Hi Tom, No, I am not proposing this as a way to configure replication quotas. When you describe broker configs using AdminClient, you will see all the configs persisted in /configs/brokers/<brokerId> in ZooKeeper and this includes leader.replication.throttled.rate, follower.replication.throttled.rate etc. But the broker configs that can be altered using AdminClient as a result of this KIP are those explicitly stated in the KIP (does not include any of the quota configs).
Regards, Rajini On Wed, Nov 22, 2017 at 7:54 PM, Tom Bentley <t.j.bent...@gmail.com> wrote: > Hi Rajini, > > Just to clarify, are you proposing this as a way to configure interbroker > throttling/quotas? I don't think you are, just wanted to check (since > KIP-179 proposes a different mechanism for setting them which supports > their automatic removal). > > Cheers, > > Tom > > On 22 November 2017 at 18:28, Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > > > I have made an update to the KIP to optionally return all the config > > synonyms in *DescribeConfigsResponse*. The synonyms are returned in the > > order of precedence. AlterConfigsResponse will not be modified by the > KIP. > > Since many configs already have various overrides (e.g. topic configs > with > > broker overrides, security properties with listener name overrides) and > we > > will be adding more levels with dynamic configs, it will be useful to > > obtain the full list in order of precedence. > > > > On Tue, Nov 21, 2017 at 11:24 AM, Rajini Sivaram < > rajinisiva...@gmail.com> > > wrote: > > > > > Hi Ted, > > > > > > You can quote the config name, but it is not necessary for deleting a > > > config since the name doesn't contain any special characters that > > requires > > > quoting. > > > > > > On Mon, Nov 20, 2017 at 9:20 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > >> Thanks for the quick response. > > >> > > >> It seems the config following --delete-config should be quoted. > > >> > > >> Cheers > > >> > > >> On Mon, Nov 20, 2017 at 12:02 PM, Rajini Sivaram < > > rajinisiva...@gmail.com > > >> > > > >> wrote: > > >> > > >> > Ted, > > >> > > > >> > Have added an example for --delete-config. > > >> > > > >> > On Mon, Nov 20, 2017 at 7:42 PM, Ted Yu <yuzhih...@gmail.com> > wrote: > > >> > > > >> > > bq. There is a --delete-config option > > >> > > > > >> > > Consider adding a sample with the above option to the KIP. > > >> > > > > >> > > Thanks > > >> > > > > >> > > On Mon, Nov 20, 2017 at 11:36 AM, Rajini Sivaram < > > >> > rajinisiva...@gmail.com> > > >> > > wrote: > > >> > > > > >> > > > Hi Ted, > > >> > > > > > >> > > > Thank you for reviewing the KIP. > > >> > > > > > >> > > > *Would decreasing network/IO threads be supported ?* > > >> > > > Yes, As described in the KIP, some connections will be closed if > > >> > network > > >> > > > thread count is reduced (and reconnections will be processed on > > >> > remaining > > >> > > > threads) > > >> > > > > > >> > > > *What if some keys in configs are not in the Set returned > > >> > > > by reconfigurableConfigs()? Would exception be thrown ?* > > >> > > > No, *reconfigurableConfigs() *will be used to decide which > classes > > >> are > > >> > > > notified when a configuration update is made*. > > >> > **reconfigure(Map<String, > > >> > > ?> > > >> > > > configs)* will be invoked with all of the configured configs of > > the > > >> > > broker, > > >> > > > similar to *configure(Map<String, ?> configs). *For example, > > when > > >> > > > *SslChannelBuilder* is made reconfigurable, it could just > create a > > >> new > > >> > > > SslFactory with the latest configs, using the same code as > > >> > *configure()*. > > >> > > > We avoid reconfiguring *SslChannelBuilder *unnecessarily*, *for > > >> example > > >> > > if > > >> > > > a topic config has changed, since topic configs are not listed > in > > >> the > > >> > > > *SslChannelBuilder#**reconfigurableConfigs().* > > >> > > > > > >> > > > *The sample commands for bin/kafka-configs include > '--add-config'. > > >> > Would > > >> > > > there be '--remove-config' ?* > > >> > > > bin/kafka-configs.sh is an existing tool whose parameters will > not > > >> be > > >> > > > modified by this KIP. There is a --delete-config option. > > >> > > > > > >> > > > *ssl.keystore.password appears a few lines above. Would there be > > any > > >> > > > issue with mixture of connections (with old and new password) ?* > > >> > > > No, passwords (and the actual keystore) are only used during > > >> > > > authentication. Any channel created using the old SslFactory > will > > >> not > > >> > be > > >> > > > impacted. > > >> > > > > > >> > > > Regards, > > >> > > > > > >> > > > Rajini > > >> > > > > > >> > > > > > >> > > > On Mon, Nov 20, 2017 at 4:39 PM, Ted Yu <yuzhih...@gmail.com> > > >> wrote: > > >> > > > > > >> > > > > bq. (e.g. increase network/IO threads) > > >> > > > > > > >> > > > > Would decreasing network/IO threads be supported ? > > >> > > > > > > >> > > > > bq. void reconfigure(Map<String, ?> configs); > > >> > > > > > > >> > > > > What if some keys in configs are not in the Set returned by > > >> > > > > reconfigurableConfigs() > > >> > > > > ? Would exception be thrown ? > > >> > > > > If so, please specify which exception would be thrown. > > >> > > > > > > >> > > > > The sample commands for bin/kafka-configs include > > '--add-config'. > > >> > > > > Would there be '--remove-config' ? > > >> > > > > > > >> > > > > bq. Existing connections will not be affected, new connections > > >> will > > >> > use > > >> > > > the > > >> > > > > new keystore. > > >> > > > > > > >> > > > > ssl.keystore.password appears a few lines above. Would there > be > > >> any > > >> > > issue > > >> > > > > with mixture of connections (with old and new password) ? > > >> > > > > > > >> > > > > > > >> > > > > Cheers > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > On Mon, Nov 20, 2017 at 5:57 AM, Rajini Sivaram < > > >> > > rajinisiva...@gmail.com > > >> > > > > > > >> > > > > wrote: > > >> > > > > > > >> > > > > > Hi all, > > >> > > > > > > > >> > > > > > I have submitted KIP-226 to enable dynamic reconfiguration > of > > >> > brokers > > >> > > > > > without restart: > > >> > > > > > > > >> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > >> > > > > > 226+-+Dynamic+Broker+Configuration > > >> > > > > > > > >> > > > > > The KIP proposes to extend the current dynamic replication > > quota > > >> > > > > > configuration for brokers to support dynamic reconfiguration > > of > > >> a > > >> > > > limited > > >> > > > > > set of configuration options that are typically updated > during > > >> the > > >> > > > > lifetime > > >> > > > > > of a broker. > > >> > > > > > > > >> > > > > > Feedback and suggestions are welcome. > > >> > > > > > > > >> > > > > > Thank you... > > >> > > > > > > > >> > > > > > Regards, > > >> > > > > > > > >> > > > > > Rajini > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
