OK, thanks Ewen.
On Sun, Jan 15, 2017 at 4:42 PM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > This is currently expected. Internally the Connect cluster uses the same > rebalancing process as consumer groups which means it has similar > limitations -- all tasks must stop just as you would need to stop consuming > from all partitions and commit offsets during a consumer group rebalance. > > There's an issue filed about this: https://issues.apache.org/ > jira/browse/KAFKA-3351 and it's something we're aware eventually becomes a > scalability limitations. There are some ideas about how to avoid this, but > nothing concrete on the roadmap yet. > > -Ewen > > On Fri, Jan 13, 2017 at 10:32 AM, Mathieu Fenniak < > mathieu.fenn...@replicon.com> wrote: > > > Hey kafka-users, > > > > Is it normal for a Kafka Connect source connector that > > calls requestTaskReconfiguration to cause all the connectors on the > > kafka-connect distributed system to be stopped and started? > > > > One of my three connectors (2x source, 1x sink) runs a background thread > > that will occasionally invoke `context.requestTaskReconfiguration()` > when > > it detects new work that it wants to distribute to its tasks. When this > > occurs, I observe all three of the connectors stop and start. One of the > > other connectors doesn't have the smoothest stop/start cycle, so I'm > hoping > > that this might be avoidable? > > > > I'm running on Kafka Connect 0.10.1.0. > > > > Mathieu > > >
