Mickael Maison created KAFKA-14746:
--------------------------------------
Summary: Throwing in Connector.taskConfigs generates a lot of logs
Key: KAFKA-14746
URL: https://issues.apache.org/jira/browse/KAFKA-14746
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Reporter: Mickael Maison
If a Connector throws in its taskConfigs() method, the runtime ends up retrying
using DistributedHerder.RECONFIGURE_CONNECTOR_TASKS_BACKOFF_MS which is a fixed
value (250ms). For each retry, the runtime prints the connector configuration
and the enriched configuration so this can quickly generate a lot of logs.
There is some value in throwing in taskConfigs() as it allows to fail fast in
case the connector is given bad credentials for example.
The way Connectors are expected to work today is to instead always create tasks
and let each task fail in case the configuration is wrong. We should document
that and make it clear in the javadoc that throwing in taskConfigs is not
recommended.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
