Mirror Maker (2) might be a good solution.. It's "mostly" Kafka Connect, so
it should be possible to do filtering etc.
https://developers.redhat.com/articles/2023/11/13/demystifying-kafka-mirrormaker-2-use-cases-and-architecture#mirrormaker_2_internal_topics
/Anders
On Tue, Nov 21, 2023 at 11:55 A
his handler seems to be doing seeks on the consumer, and I'm thinking this
might be what's somehow causing this...
I'll try to dig deeper into that error handler -- but if anyone have
experience from Spring-Kafka and error handlers, feel free to jump in :)
Cheers /Anders
>
> On M
Hi!
We're running Kafka 3.3 (at Aiven) and are seeing warning logs about offset
commits from both consumers and producers.
This happens for two of our consumers (out of hundreds). I've tried
Googling for the error (of course) but I haven't found anything obvious.
[kafka-01-78]2023-06-05T06:44:12.
Hi -
depending on the rules for how to filter/drop incoming messages (and
depending on the mechanics of the library you use to consume the messages),
it might be possible to filter out messages based on message headers,
maybe? That way you would not need to deserialize the message key/value
before
Quick informal update:
Plugging in an SMT works like a charm 👍. Thanks for pointing me in the
right direction.
Took some time to understand that the MirrorSinkConnector mentioned in the
KIP doesn't actually exist in Kafka. When running MM2 as a "pure"
MirrorSourceConnector I need to point the conne
On Tue, Sep 14, 2021 at 3:57 PM Ryanne Dolan wrote:
> Hey Anders, take a look at Connect's serdes and SMTs. MirrorMaker can be
> configured to use them.
>
> Ryanne
>
> On Tue, Sep 14, 2021, 3:13 AM Anders Engström wrote:
>
> > Hi!
> > I'm trying to replic
Hi!
I'm trying to replicate a few topics using Mirror Maker 2 (2.8).
Both the source and the target cluster use Schema Registry (Karapace, on
Aiven)
and the Confluent Avro serialization format when publishing messages.
This means that messages replicated from source->target are not readable
with
