I’m currently wondering if it’s possible to use the internal
`__consumer_offsets` topic to manage offsets outside the consumer group APIs.
I’m using the low-level API to manage the consumers but I’d still like to store
offsets in Kafka.
If it’s not possible to publish and fetch offsets from
clusters, and keep under a certain
> limit of partitions in each cluster. That is of course additional
> operational overhead and complexity.
>
> I'm not sure I 100% understand your mechanism for tracking pending offsets,
> but it seems like that might be your best option.
>
>
Hey all,
I’m currently in the process of designing a system around Kafka and I’m
wondering the recommended way to manage topics. Each event stream we have needs
to be isolated from each other. A failure from one should not affect another
event stream from processing (by failure, we mean a
