[ 
https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gwen Shapira updated KAFKA-3213:
--------------------------------
    Description: 
Looking at logs of attempt to reconfigure connector while leader is restarting, 
I see:

{code}

[2016-01-29 20:31:01,799] ERROR IO error forwarding REST request:  
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector 
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, 
retrying after backoff: 
(org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,803] DEBUG Sending POST with input 
[{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}]
 to http://worker2:8083/connectors/test-mysql-jdbc/tasks 
(org.apache.kafka.connect.runtime.rest.RestServer)
[2016-01-29 20:31:01,803] ERROR IO error forwarding REST request:  
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector 
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
trying to forward REST request: Connection refused
{code}

Note that it looks like we are retrying every 1ms, while I'd expect a retry 
every 250ms.


  was:
Looking at logs of attempt to reconfigure connector while leader is restarting, 
I see:

[code]

[2016-01-29 20:31:01,799] ERROR IO error forwarding REST request:  
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector 
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, 
retrying after backoff: 
(org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,803] DEBUG Sending POST with input 
[{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}]
 to http://worker2:8083/connectors/test-mysql-jdbc/tasks 
(org.apache.kafka.connect.runtime.rest.RestServer)
[2016-01-29 20:31:01,803] ERROR IO error forwarding REST request:  
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector 
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
trying to forward REST request: Connection refused
[code]
Note that it looks like we are retrying every 1ms, while I'd expect a retry 
every 250ms.



> [CONNECT] It looks like we are not backing off properly when reconfiguring 
> tasks
> --------------------------------------------------------------------------------
>
>                 Key: KAFKA-3213
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3213
>             Project: Kafka
>          Issue Type: Bug
>          Components: copycat
>            Reporter: Gwen Shapira
>            Assignee: Ewen Cheslack-Postava
>
> Looking at logs of attempt to reconfigure connector while leader is 
> restarting, I see:
> {code}
> [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request:  
> (org.apache.kafka.connect.runtime.rest.RestServer)
> java.net.ConnectException: Connection refused
> [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector 
> tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
> trying to forward REST request: Connection refused
> [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, 
> retrying after backoff: 
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
> trying to forward REST request: Connection refused
> [2016-01-29 20:31:01,803] DEBUG Sending POST with input 
> [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}]
>  to http://worker2:8083/connectors/test-mysql-jdbc/tasks 
> (org.apache.kafka.connect.runtime.rest.RestServer)
> [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request:  
> (org.apache.kafka.connect.runtime.rest.RestServer)
> java.net.ConnectException: Connection refused
> [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector 
> tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error 
> trying to forward REST request: Connection refused
> {code}
> Note that it looks like we are retrying every 1ms, while I'd expect a retry 
> every 250ms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to