[
https://issues.apache.org/jira/browse/KAFKA-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959741#comment-14959741
]
ASF GitHub Bot commented on KAFKA-2371:
---------------------------------------
GitHub user ewencp opened a pull request:
https://github.com/apache/kafka/pull/321
KAFKA-2371: Add distributed support for Copycat.
This adds coordination between DistributedHerders using the generalized
consumer
support, allowing automatic balancing of connectors and tasks across
workers. A
few pieces that require interaction between workers (resolving config
inconsistencies, forwarding of configuration changes to the leader worker)
are
incomplete because they require REST API support to implement properly.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ewencp/kafka kafka-2371-distributed-herder
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/321.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #321
----
commit 05300605f5a50b92e89be40919634487822538e4
Author: Jason Gustafson <[email protected]>
Date: 2015-08-13T20:33:05Z
KAFKA-2464; client-side assignment for new consumer
commit 6d48c10527200127876df7a2ed749d432d824a14
Author: Jason Gustafson <[email protected]>
Date: 2015-10-08T23:12:34Z
Revert changes to NetworkClient; squash later
commit 7f457b30269b41a227d2022a94635d2d428ca176
Author: Jason Gustafson <[email protected]>
Date: 2015-10-09T17:25:53Z
Disallow empty topic list to fetch all topic metadata
commit f0c1daa9af1e70a2824088443c8110edd983577d
Author: Jason Gustafson <[email protected]>
Date: 2015-10-12T20:17:48Z
split consumer protocol version into separate structure
commit 013df65b6ee4dbc184046b994f206517640dae17
Author: Jason Gustafson <[email protected]>
Date: 2015-10-12T23:26:14Z
merge listeners for pattern subscription and metadata changes
commit 3caadea325d83bce0175c7692c70fa1dc903fe1e
Author: Jason Gustafson <[email protected]>
Date: 2015-10-13T00:48:02Z
rename client coordinator classes
commit c943ab628d9578a22f0e16d4b9b77cdc12444457
Author: Jason Gustafson <[email protected]>
Date: 2015-10-13T21:43:03Z
clean up JoinGroup/SyncGroup handling
commit 82f220ce77e6c8240e7d94e8a9590100b96fa0e6
Author: Jason Gustafson <[email protected]>
Date: 2015-10-13T21:43:35Z
miscellaneous review fixes
commit 30ffd7e40a6cf484966c3f159cf1623312cb7f71
Author: Jason Gustafson <[email protected]>
Date: 2015-10-15T18:13:12Z
rework join group to support protocol-specific metadata
commit a37aeb3aecf374943ad62282ce8096094e998b41
Author: Ewen Cheslack-Postava <[email protected]>
Date: 2015-10-15T20:45:00Z
Merge remote-tracking branch 'origin/trunk' into merged2
commit 430b0bbc255389e5b2a11126d50ec5e0ae85faaa
Author: Ewen Cheslack-Postava <[email protected]>
Date: 2015-10-13T17:31:16Z
KAFKA-2371: Add distributed support for Copycat.
This adds coordination between DistributedHerders using the generalized
consumer
support, allowing automatic balancing of connectors and tasks across
workers. A
few pieces that require interaction between workers (resolving config
inconsistencies, forwarding of configuration changes to the leader worker)
are
incomplete because they require REST API support to implement properly.
commit 800984eaa835adfb2cc0f9e49b7523ae0632ae99
Author: Ewen Cheslack-Postava <[email protected]>
Date: 2015-10-15T21:40:28Z
FIXME REMOVE AFTER REBASE.
----
> Add distributed coordinator implementation for Copycat
> ------------------------------------------------------
>
> Key: KAFKA-2371
> URL: https://issues.apache.org/jira/browse/KAFKA-2371
> Project: Kafka
> Issue Type: Sub-task
> Components: copycat
> Reporter: Ewen Cheslack-Postava
> Assignee: Ewen Cheslack-Postava
> Fix For: 0.9.0.0
>
>
> Copycat needs a Coordinator implementation that handles multiple Workers that
> automatically manage the distribution of connectors and tasks across them. To
> start, this implementation should handle any connectors that have been
> registered via either a CLI or REST interface for starting/stopping
> connectors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
