Hi Gunnar, I agree with you that the goal should be to implement offset management capabilities in the rest interface. I believe Randall's intention with this KIP was to lay the groundwork for accessing and changing offsets in Connect and then build on that functionality once the changes necessary for that groundwork are implemented.
That being said, I think the major changes for this to become possible will be in the OffsetBackingStore and implementing classes, since as far as I can tell these currently don't support retrieving all offsets for a connector, but require a partition key to be passed in, which we do not necessarily have available in all scenarios. Adding in the rest endpoints and necessary functionality once these basic changes are in place should be a fairly simple exercise and we could consider adding them to the KIP to avoid unnecessary overhead in creating a dedicated KIP for just the rest functionality. Kind regards, Sönke On Fri, Nov 17, 2017 at 9:32 PM, Gunnar Morling <gun...@hibernate.org> wrote: > Hi, > > I was reading KIP-199 [1] for adding a tool for Kafka Connect offset > management. This would be a very useful functionality for users of the > Debezium CDC connectors, too. > > What I was wondering, instead of having a separate tool for this, has it > been considered to expose offset management via the REST API of Connect? > There could be a resource /connectors/<connector-name>/offsets for read > and > write access. In line with the current KIP, write access would only be > allowed if the connector is stopped. Exposing this in the REST API might > allow for a consistent experience with the other connector management > functionalities. > > I'm not sure whether there are any guidelines on when to have some > functionality as a separate tool vs. in the API, but I thought I'd bring up > the idea and see what others think. > > Thanks, > > --Gunnar > > [1] > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 199%3A+Add+Kafka+Connect+offset+tool > -- Sönke Liebau Partner Tel. +49 179 7940878 OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
