[ https://issues.apache.org/jira/browse/KAFKA-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316820#comment-16316820 ]
Randall Hauch commented on KAFKA-6387: -------------------------------------- There are a number of cases where this behavior might actually change how the producers and consumers used for connectors are configured. Consider a worker configuration includes the following configuration properties: {code} retries=2 max.partition.fetch.bytes=262144 {code} This configuration file does not define "{{consumer.max.partition.fetch.bytes}}" or "{{producer.max.partition.fetch.bytes}}", so currently these would default to 1048576. However, after this proposed change if implemented the "{{consumer.max.partition.fetch.bytes}}" or "{{producer.max.partition.fetch.bytes}}" values would be the inherited value of 262144, not the default 1048576. In short, implementing this change would break backward compatibility. We could implement a configuration switch that controls whether the configurations are inherited, but this adds complexity to the already-complex configuration mechanism. > Worker's producer and consumer configs should inherit from worker configs > ------------------------------------------------------------------------- > > Key: KAFKA-6387 > URL: https://issues.apache.org/jira/browse/KAFKA-6387 > Project: Kafka > Issue Type: Improvement > Components: KafkaConnect > Affects Versions: 1.0.0 > Reporter: Randall Hauch > Labels: needs-kip > > Currently, the worker configuration file defines the connection properties > for the three separate types of connections being made to the Kafka cluster: > # the worker group membership, > # producers for source connectors, > # the consumers for sink connectors. > The configs are namespaced because to properly support things like > interceptors where the configs for 2 and 3 would conflict (same config name, > different value). > However, it would be beneficial when such control is not required for the > producers and consumers to inherit the top-level configurations yet be able > to override them with the {{producer.}} and {{consumer.}} namespaced > configurations. This way the producer- and consumer-specific configurations > need only be specified if/when they need to override the top-level > configurations. This may be necessary, for example, to have different ACLs > than the connector tasks compared to the producers and consumers. > This will require a minimal KIP to explain the new behavior. -- This message was sent by Atlassian JIRA (v6.4.14#64029)