[
https://issues.apache.org/jira/browse/KAFKA-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671976#comment-16671976
]
Randall Hauch edited comment on KAFKA-6890 at 11/1/18 6:32 PM:
---------------------------------------------------------------
First of all, we're going to need a KIP for this, since it will change the
configurations exposed to users. We need to make sure that this doesn't break
backward compatibility, etc.
Second, I like this change and the ability to override some of the
producer/consumer behavior. I also think that generally it's better to give
users more control, but I'm actually very concerned about allowing connector
configurations to override {{bootstrap servers}}. Yes, I can totally see
several valid use cases and benefits, but this also gives users that don't know
what they're doing to really mess things up. Some people will undoubtably think
they *have* to define these properties in every connector, and then when they
change their bootstrap URL in the worker they might forget it in their
connector configs. Or, worse yet, having the internal topics in one cluster and
the connector records in a different cluster will very much conflict with any
hopes of adding EOS to source connectors.
was (Author: rhauch):
First of all, we're going to need a KIP for this, since it will change the
configurations exposed to users. We need to make sure that this doesn't break
backward compatibility, etc.
Second, I like this change and the ability to override some of the
producer/consumer behavior. I also think that generally it's better to give
users more control, but I'm actually very concerned about allowing connector
configurations to override {{bootstrap servers}}. Yes, I can totally see the
benefits, but this also gives users that don't know what they're doing to
really mess things up. Some people will undoubtably think they *have* to define
these properties in every connector, and then when they change their bootstrap
URL in the worker they might forget it in their connector configs. Or, worse
yet, having the internal topics in one cluster and the connector records in a
different cluster will very much conflict with any hopes of adding EOS to
source connectors.
> Add connector level configurability for producer/consumer client configs
> ------------------------------------------------------------------------
>
> Key: KAFKA-6890
> URL: https://issues.apache.org/jira/browse/KAFKA-6890
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect
> Reporter: Allen Tang
> Priority: Minor
>
> Right now, each source connector and sink connector inherit their client
> configurations from the worker properties. Within the worker properties, all
> configurations that have a prefix of "producer." or "consumer." are applied
> to all source connectors and sink connectors respectively.
> We should also provide connector-level overrides whereby connector properties
> that are prefixed with "producer." and "consumer." are used to feed into the
> producer and consumer clients embedded within source and sink connectors
> respectively. The prefixes will be removed via a String#substring() call, and
> the remainder of the connector property key will be used as the client
> configuration key. The value is fed directly to the client as the
> configuration value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
