David Dufour created KAFKA-13253:
------------------------------------
Summary: Kafka Connect losing task (re)configuration when
connector name has special characters
Key: KAFKA-13253
URL: https://issues.apache.org/jira/browse/KAFKA-13253
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Affects Versions: 2.7.1
Reporter: David Dufour
When not leader, DistributedHerder.reconfigureConnector() forwards the task
configuration to the leader as follow:
{quote}String reconfigUrl = RestServer.urlJoin(leaderUrl, "/connectors/" +
connName + "/tasks");
log.trace("Forwarding task configurations for connector {} to leader",
connName);
RestClient.httpRequest(reconfigUrl, "POST", null, rawTaskProps, null, config,
sessionKey, requestSignatureAlgorithm);
{quote}
The problem is that if the connector name contains some special characters,
such as '<', '>'... they need to be 'URLEncoded' appropriately, otherwise an
uncatched exception is raised in RestClient and the forward is lost.
Here is the kind of exception we can catch by adding the necessary code in
RestClient:
{quote}java.lang.IllegalArgumentException: Illegal character in path at index
51:
http://10.224.0.15:8083/connectors/mirror1-cluster->mirror2-cluster.MirrorHeartbeatConnector/tasks
{quote}
An additional catch() should be added in RestClient.httpRequest(), here:
{quote}} catch (IOException | InterruptedException | TimeoutException |
ExecutionException e) {
log.error("IO error forwarding REST request: ", e);
throw new ConnectRestException(Response.Status.INTERNAL_SERVER_ERROR, "IO Error
trying to forward REST request: " + e.getMessage(), } finally {
{quote}
to catch all other exceptions because without, this kind of problem is
completly silent.
To reproduce:
* start 2 kafka clusters
* start a kafka connect (distributed) with at least 2 nodes
* start an HeartbeatConnector with name "cluster1->cluster2"
If the node which generated the task is not the leader (not systematic), it
will forward the creation to the leader and it will be lost. As a result, the
connector will stay in RUNNING state but without any task.
Problem not easy to reproduce, it is important to start with empty connect
topics to reproduce more easily
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
