Mickael Maison created KAFKA-20973:
--------------------------------------
Summary: Unable to dynamically update non string configs with
config providers
Key: KAFKA-20973
URL: https://issues.apache.org/jira/browse/KAFKA-20973
Project: Kafka
Issue Type: Bug
Reporter: Mickael Maison
When using config.providers in a dynamic broker config update (via
IncrementalAlterConfigs), provider references like ${file:/path:key} work for
STRING and PASSWORD typed configs but silently fail for other types
(INT, LONG, BOOLEAN, LIST with validators, etc.).
The root cause is in DynamicBrokerConfig.fromPersistentProps (line 327), which
calls removeInvalidConfigs and validateConfigTypes on the raw persisted values
before resolveVariableConfigs is called. For non-STRING
types, the literal placeholder string ${file:/path:key} fails type parsing
(e.g., "Not a number of type INT" for num.io.threads), and the config entry is
silently removed from the props.
There is a unit test,
DynamicBrokerReconfigurationTest.testUpdatesUsingConfigProvider which is
expected to demonstrate this work but the test is flawed because it only tests
custom and per listener configurations which are handle differently. If we try
to update a config like num.io.threads, the test fails with:
{noformat}
ERROR Dynamic broker config contains invalid values in: Set(num.io.threads),
these configs will be ignored (kafka.server.DynamicBrokerConfig:63)
org.apache.kafka.common.config.ConfigException: Invalid value
${file:num.io.threads:threads} for configuration num.io.threads: Not a number
of type INT
at
org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:789)
at
org.apache.kafka.server.config.DynamicConfig$Broker.validate(DynamicConfig.java:72)
at
org.apache.kafka.server.config.DynamicBrokerConfig.validateConfigTypes(DynamicBrokerConfig.java:151)
at
org.apache.kafka.server.config.DynamicBrokerConfig.validateConfigTypes(DynamicBrokerConfig.java:145)
at
kafka.server.DynamicBrokerConfig.removeInvalidConfigs(DynamicBrokerConfig.scala:371)
at
kafka.server.DynamicBrokerConfig.fromPersistentProps(DynamicBrokerConfig.scala:327)
at
kafka.server.DynamicBrokerConfig.$anonfun$updateBrokerConfig$1(DynamicBrokerConfig.scala:279)
at org.apache.kafka.server.util.LockUtils.inLock(LockUtils.java:96)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
