[
https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507868#comment-17507868
]
Chris Egerton commented on KAFKA-7509:
--------------------------------------
Thanks [~mjsax]--I think that's an acceptable approach, but it would come at a
few costs:
# Breaking backwards compatibility for a lot of Kafka components
(producer/consumer interceptors, Connect REST extensions, etc.)
# Complicating the configuration behavior for applications with this type of
"nested" configuration surface
# Makes it impossible to have sub-configured components that are aware of
their parent's configuration (for example, you could no longer have a Connect
REST extension that's aware of the entire Connect worker's config without
duplicating those same properties into the sub-config for that REST extension)
I personally hesitate to inflict a breaking change on users just for the sake
of cleaning up log messages, but the second point actually provides a bit of an
opportunity to clean up some of the uglier parts of Connect (and possibly other
projects as well).
Right now, if you want to configure a Connect worker (or connector) with
credentials to use for its connectors' Kafka clients, you have to specify the
same credentials in up to three different places: one for each type of Kafka
client. (This isn't a hypothetical scenario; it's required if you want to run a
sink connector that uses the DLQ feature.)
The ability to define a single "forward." (or possibly just "clients.") prefix
for properties to be passed to all nested configurations (or all nested
configurations used exclusively for Kafka clients) would help with situations
like this by eliminating the need to duplicate properties within a single
configuration.
Ultimately, this still seems a little too aggressive of a change for the
problem that it's trying to solve. If we were redesigning these APIs from the
ground up, it would certainly be beneficial, but considering how much has been
built on top of these APIs already and how much work it'd take for users to
adjust to the proposed changes, it doesn't seem like a friendly tradeoff. Plus,
for some situations (like the example with REST extensions), it's unclear how
we'd want to proceed.
> Kafka Connect logs unnecessary warnings about unused configurations
> -------------------------------------------------------------------
>
> Key: KAFKA-7509
> URL: https://issues.apache.org/jira/browse/KAFKA-7509
> Project: Kafka
> Issue Type: Improvement
> Components: clients, KafkaConnect
> Affects Versions: 0.10.2.0
> Reporter: Randall Hauch
> Priority: Major
>
> When running Connect, the logs contain quite a few warnings about "The
> configuration '{}' was supplied but isn't a known config." This occurs when
> Connect creates producers, consumers, and admin clients, because the
> AbstractConfig is logging unused configuration properties upon construction.
> It's complicated by the fact that the Producer, Consumer, and AdminClient all
> create their own AbstractConfig instances within the constructor, so we can't
> even call its {{ignore(String key)}} method.
> See also KAFKA-6793 for a similar issue with Streams.
> There are no arguments in the Producer, Consumer, or AdminClient constructors
> to control whether the configs log these warnings, so a simpler workaround
> is to only pass those configuration properties to the Producer, Consumer, and
> AdminClient that the ProducerConfig, ConsumerConfig, and AdminClientConfig
> configdefs know about.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
