Chris Egerton created KAFKA-13327:
-------------------------------------

             Summary: Preflight validations of connectors leads to 500 responses
                 Key: KAFKA-13327
                 URL: https://issues.apache.org/jira/browse/KAFKA-13327
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
            Reporter: Chris Egerton
            Assignee: Chris Egerton


The Connect framework performs some preflight validations for all connectors 
that are created in addition to allowing connectors to define their own custom 
validation logic by providing a {{ConfigDef}} object in 
[Connector::config|https://kafka.apache.org/30/javadoc/org/apache/kafka/connect/connector/Connector.html#config()]
 and performing multi-property validation in 
[Connector::validate|https://kafka.apache.org/30/javadoc/org/apache/kafka/connect/connector/Connector.html#validate(java.util.Map)].

When performed correctly, this validation information is surfaced to the user 
in the form of a 
[ConfigInfos|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigInfos.java]
 object containing a list of [config 
objects|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigInfo.java#L42-L45]
 whose 
[values|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigInfo.java#L42-L45]
 contain one or more [error 
messages|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigValueInfo.java#L61-L64].
 This can be used as the response for a REST request to PUT 
/connector-plugins/\{connectorType}/config/validate and allows programmatic UIs 
to render error messages for every invalid property to the user.

However, some validations performed by the Connect framework do not follow this 
pattern and instead result in a 500 response being returned to the user. For 
example, logic specific to sink connectors (see 
[AbstractHerder|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java#L436]
 and 
[SinkConnectorConfig|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SinkConnectorConfig.java#L88-L125])
 simply throws an exception instead of documenting the error with the offending 
property and returning it in a standard response.

 

We should correct this logic wherever possible so that configurations that are 
not fatally invalid (i.e., may have invalid properties but can still be 
translated into a meaningful {{ConfigInfos}} response object) do not cause a 
500 response to be returned to the user.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to