Hello, the heartbeat connector is a sink connector, so it normally would
write to the target cluster. I can think of two ways to achieve what you
want:
1) set up a second connect cluster that sinks to the source cluster, and
run just the heartbeat connector there.
2) override the heartbeat
Makes sense to me, +1.
On Tue, Jan 16, 2024 at 5:04 PM Kondrát Bertalan wrote:
> Hey Team,
>
> I would like to start a discussion thread about the *KIP-1016 Make MM2
> heartbeats topic name configurable
> <
>
-1, non-binding, for reasons previously stated.
Ryanne
On Fri, Aug 4, 2023, 3:46 AM Andrew Schofield <
andrew_schofield_j...@outlook.com> wrote:
> Hi,
> After almost 2 1/2 years in the making, I would like to call a vote for
> KIP-714 (
>
hudeqi, I'd call the configuration property something that describes what
it does rather than it's intended use-case.
Ryanne
On Tue, Aug 8, 2023, 4:46 AM hudeqi <16120...@bjtu.edu.cn> wrote:
> Hi, all. I want to submit a kip, and hope get some review and good
> suggestions. the kip is here:
"it becomes impossible to ensure that MM2 is
able to sync offsets from consumer groups that are behind the last-synced
offset emitted by MirrorSourceTask."
That's sorta true. The idea is that offset syncs are exceptionally rare.
They basically occur when the tasks start or restart and that should
+1 (non-binding)
On Sat, Oct 22, 2022, 2:38 AM Urbán Dániel wrote:
> Hi everyone,
>
> I would like to start a vote on KIP-710 which aims to support running a
> dedicated MM2 cluster in distributed mode:
>
>
>
w my POC handles this, but we'd need to get this right for
it to make sense.
Ryanne
> Thanks,
> Mickael
>
> On Fri, Jun 4, 2021 at 9:23 PM Ryanne Dolan wrote:
> >
> > Hey Tom, thanks for taking a look.
> >
> > > It's a bit weird that there's a separate st
votes.. I
> think this could be a useful feature.
>
> Thanks!
> Sagar.
>
> On Sat, May 8, 2021 at 1:16 PM Dongjin Lee wrote:
>
> > Thank you for the KIP. I need this feature.
> >
> > +1 (non-binding)
> >
> > Thanks,
> > Dongjin
> >
&g
broker case. How could we know the broker is new
> added? I guess it's by checking the broker load via some metrics
> dynamically, right?
>
>
> Thank you.
> Luke
>
> On Fri, Mar 18, 2022 at 10:30 AM Ryanne Dolan
> wrote:
>
> > Thanks Mickael, this makes sense to me
Thanks Mickael, this makes sense to me! I've been wanting something like
this in order to decommission a broker without new partitions getting
accidentally assigned to it.
Ryanne
On Thu, Mar 17, 2022, 5:56 AM Mickael Maison
wrote:
> Hi,
>
> I'd like to start a new discussion on KIP-660. I
it internally.
>
> Thanks,
>
> Antón
>
> On Thu, Jul 29, 2021 at 7:02 PM Ryanne Dolan
> wrote:
>
> > Jamie, this would depend on KIP-712 (or similar) aka "shallow mirroring".
> > This is a work in progress, but I'm optimistic it'll happen at some
> p
Ranga, yes that will work. You are correct that mm2's compatibility matrix
is essentially the same as the client's. I've tested 2.4 mm2 against 0.10.2
brokers and it works fine. Anything older than that may still generally
work, but you may see errors for unsupported apis.
Ryanne
On Mon, Feb 7,
+1 (non-binding)
Ryanne
On Mon, Dec 13, 2021, 4:18 AM Mickael Maison
wrote:
> Hi all,
>
> I'd like to start a vote on KIP-769 which proposes adding new
> endpoints to the Connect REST API to list all connectors plugins and
> retrieve their configurations.
>
>
I think we should be very careful about introducing new runtime
dependencies into the clients. Historically this has been rare and
essentially necessary (e.g. compression libs).
Ryanne
On Mon, Dec 13, 2021, 1:06 PM Kirk True wrote:
> Hi Jun,
>
> On Thu, Dec 9, 2021, at 2:28 PM, Jun Rao wrote:
and make it easier for the use cases where customers wish to
> provide a different method to handle resources.
>
> Omnia
>
> On Tue, Oct 26, 2021 at 4:10 PM Ryanne Dolan
> wrote:
>
> > I like the idea of failing-fast whenever a custom impl is provided, but I
> > su
Neat! Makes sense to me.
Ryanne
On Tue, Oct 26, 2021, 11:02 AM Mickael Maison
wrote:
> Hi all,
>
> I wrote a KIP to allow setting the number of network threads per listener:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-788%3A+Allow+configuring+num.network.threads+per+listener
>
>
not sure if adding these concerns on the users is acceptable or not.
> One solution to address these concerns could be adding some checks to make
> sure the methods MM2 uses from the Admin interface exists to fail faster.
> What do you think
>
> Omnia
>
>
> On Mon, Oct 25, 2
Thanks Omnia, neat idea. I wonder if we could use the existing Admin
interface instead of defining a new one?
Ryanne
On Mon, Oct 25, 2021, 12:54 PM Omnia Ibrahim
wrote:
> Hey everyone,
> Please take a look at KIP-787
>
>
-1
Ryanne
On Mon, Oct 18, 2021, 4:30 AM Magnus Edenhill wrote:
> Hi all,
>
> I'd like to start a vote on KIP-714.
> https://cwiki.apache.org/confluence/x/2xRRCg
>
> Discussion thread:
> https://www.mail-archive.com/dev@kafka.apache.org/msg119000.html
>
> Thanks,
> Magnus
>
Thanks Jordan, this is a major blindspot today.
Ryanne
On Wed, Sep 1, 2021, 6:03 PM Jordan Bull wrote:
> Hi all,
>
> I would like to start the discussion for KIP-767 involving adding latency
> metrics to Connect. The KIP can be found at
>
>
er.
>
> What do you think?
>
> On Sat, Jul 17, 2021 at 3:16 PM Ryanne Dolan
> wrote:
> >
> > Whoops, looks like I got the KIP number wrong in the original email
> subject
> > line. Please use this corrected thread.
> >
> > Ryanne
> >
> &
I think it'd be worth adding a GET version, fwiw. Could be the same handler
with just a different spelling maybe.
On Fri, Aug 20, 2021, 7:44 AM Mickael Maison
wrote:
> Hi Chris,
>
> You're right, you can achieve the same functionality using the
> existing validate endpoint.
> In my mind it was
Jamie, this would depend on KIP-712 (or similar) aka "shallow mirroring".
This is a work in progress, but I'm optimistic it'll happen at some point.
ftr, "IdentityReplicationPolicy" has landed for the upcoming release, tho
"identity" in that context just means that topics aren't renamed.
Ryanne
Whoops, looks like I got the KIP number wrong in the original email subject
line. Please use this corrected thread.
Ryanne
On Fri, Jul 16, 2021, 3:45 PM Ryanne Dolan wrote:
> Hey y'all, please review the following small proposal:
>
>
> https://cwiki.apache.org/confluence/display/K
Hey y'all, please review the following small proposal:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-762%3A+Delete+Committed+Connect+Records
The idea is to enable Source->Sink pipelines to clean up after themselves
by automatically truncating intermediate topics.
Ryanne
Badai, the KIP makes sense to me, but could you maybe add some evidence to
support your proposed values? I'm sure they come from experience but I
don't think they are obvious and so may appear arbitrary.
Ryanne
On Tue, Jul 6, 2021, 4:23 PM Badai Aqrandista
wrote:
> Hi all
>
> I have just
Ryanne Dolan created KAFKA-13038:
Summary: document IdentityReplicationPolicy
Key: KAFKA-13038
URL: https://issues.apache.org/jira/browse/KAFKA-13038
Project: Kafka
Issue Type: Bug
Satish, we encounter this frequently and consider it a major bug. Your
solution makes sense to me.
Ryanne
On Tue, Jun 22, 2021, 7:29 PM Satish Duggana
wrote:
> Hi,
> Bumping up the discussion thread on KIP-501 about avoiding out-of-sync or
> offline partitions when follower fetch requests are
e internal topics of MM2. If we added config
> >>> like connect to each internal topic then these customers will end up
> adding
> >>> 4 configs just to handle the same naming rule, 1 to include customised
> >>> replication policy for naming replicated topics +
s), it still requires reconfiguring and restarting the
> application, which is disruptive. Correlating client metrics with server
> metrics is also often hard. These issues are all mitigated by centralizing
> metrics collection on the broker.
>
> best,
> Colin
>
>
interpret them. In
> what format? Are all relevant metadata fields
>provided?
>
> The KIP aims to solve all these obstacles by giving the Kafka operator the
> tools to collect this information.
>
> Regards,
> Magnus
>
>
> Den tis 15 juni 2021 kl 02:37 skrev
Magnus, I think such a substantial change requires more motivation than is
currently provided. As I read it, the motivation boils down to this: you
want your clients to phone-home unless they opt-out. As stated in the KIP,
"there are plenty of existing solutions [...] to send metrics [...] to a
Matthew, I wonder what the expected behavior would be when topic-creation
is disabled and MM is asked to replicate a topic that doesn't exist on the
target cluster? Maybe the task should fail at that point, or maybe it
should replicate whatever it can?
I think the current behavior is reasonable,
+1 (non-binding), thanks!
Ryanne
On Fri, Jun 11, 2021, 3:25 AM Tom Bentley wrote:
> Hi Mickael,
>
> Thanks for the KIP, +1 (binding).
>
> Cheers,
>
> Tom
>
> On Tue, Apr 13, 2021 at 4:56 PM Mickael Maison
> wrote:
>
> > Hi,
> >
> > It has been a few weeks since I opened this vote and I have
+1 (non-binding)
Ryanne
On Mon, Jun 7, 2021, 3:03 PM Konstantine Karantasis
wrote:
> Thanks Randall.
>
> +1 (binding)
>
> Konstantine
>
> On Mon, Jun 7, 2021 at 12:47 PM Randall Hauch wrote:
>
> > Hello all,
> >
> > I would like to start a vote on KIP-745:
> >
> >
>
+1 (non-binding)
Ryanne
On Mon, Jun 7, 2021, 9:26 AM Satish Duggana
wrote:
> +1 (non-binding)
>
> On Mon, 7 Jun 2021 at 19:30, Dongjin Lee wrote:
>
> > +1 (non-binding)
> >
> > As a note: I think the exact removal schedule may be changed.
> >
> > Best,
> > Dongjin
> >
> > On Mon, Jun 7, 2021,
+1 (non-binding), thanks!
Ryanne
On Sat, Jun 5, 2021, 4:37 PM Dongjin Lee wrote:
> Hi all,
>
> I'd like to open a voting thread for KIP-390: Support Compression Level
> (rebooted):
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level
>
> Best,
> Dongjin
Gotcha, thanks.
On Fri, Jun 4, 2021, 8:20 PM Ismael Juma wrote:
> Documentation, yes. Including the downloads page.
>
> Ismael
>
> On Fri, Jun 4, 2021, 4:44 PM Ryanne Dolan wrote:
>
> > Thanks Ismael, this will be great for 4.0, but I don't understand what
> > ex
Thanks Ismael, this will be great for 4.0, but I don't understand what
exactly will change in 3.0? Just documentation?
I guess I don't know what it means to deprecate a build target.
Thanks!
Ryanne
On Fri, Jun 4, 2021, 9:06 AM Ismael Juma wrote:
> Hi all,
>
> Please take a look at the KIP
is could be as simple as using
> hard.rate.limiters rather than just rate.limiters, so that
> soft.rate.limiters could be added later, though maybe there are use cases
> where a single limiter needs to supply both soft and hard limits.
>
> Thanks again,
>
> Tom
>
> On Fri
+1 (non-binding)
On Thu, Jun 3, 2021, 10:23 AM Chris Egerton
wrote:
> Hi all,
>
> I'd like to call for a vote on KIP-618, which adds support for exactly-once
> delivery guarantees for source connectors in the Kafka Connect framework.
>
> I suspect there might be a little more discussion to be
[
https://issues.apache.org/jira/browse/KAFKA-7815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryanne Dolan resolved KAFKA-7815.
-
Fix Version/s: 2.4.0
Resolution: Fixed
Equivalent functionality was included as part
Thanks Randall, this would be great!
Ryanne
On Wed, Jun 2, 2021 at 11:40 AM Randall Hauch wrote:
> Hello all,
>
> Many users struggle with the connector restart REST API only restarting the
> Connector instance rather than everything they associated with a "named"
> connector. I've created a
Indeed that is what I am saying. The most ideal situation is
> that
> > > > > there is
> > > > > > > a default internal threadpool that Kafka uses, however users of
> > > Kafka
> > > > > can
> > > > > > > configure there own threadpool. Havin
>> fixed. It's returning a Future but since it's internally blocking and
> >> using
> >> the caller's thread from an API perspective it gives the incorrect
> >> impression that it's asynchronous (when it's not).
> >>
> >> On Wed, May 26,
; > > 1. If we are going this road it makes sense to do this "properly"
> > (i.e.
> > > > > > using queues as Ryan suggested). The reason I am saying this is
> > that
> > > > it
> > > > > > seems that the original goal of the KIP
commits by EOD today so that
> > you
> > > can cherry pick the test commit in your PR.
> > >
> > > Thanks a lot for the work!
> > >
> > > On Fri, May 21, 2021 at 12:53 AM Ryanne Dolan
> > wrote:
> > >
> > > > Matthew, I stole your i
sing in the new code path), we're doing something wrong.
>
> best,
> Colin
>
>
> On Thu, Apr 1, 2021, at 15:48, Ryanne Dolan wrote:
> > Colin, the only feature gap I'm aware of is that users must provide their
> > own ReplicationPolicy in order to repl
h your
> > implementation as an initial step? Though this is my first time doing a
> KIP
> > so I am not sure how long it typically takes to get one approved.
> >
> > Regards
> >
> > On Wed, May 19, 2021 at 3:58 AM Ryanne Dolan
> > wrote:
> >
> >> He
Hey Matthew, as you call out in the KIP there are few impls floating
around, including my WIP PR here:
https://github.com/apache/kafka/pull/10652
The tests are currently passing except for a couple asserts related to
failback (commented out). It appears your PR doesn't address failback, so I
failed to enqueue) or whether we
> succeeded in sending or not".
>
> But you're right, it should be on the table, thank you for suggesting it!
>
> Best,
> Moses
>
> On Tue, May 18, 2021 at 12:23 PM Ryanne Dolan
> wrote:
>
> > Moses, in the case of a full queue, could
+1 (non-binding)
Thanks!
Ryanne
On Tue, May 18, 2021, 6:38 AM Chris Egerton
wrote:
> Hi all,
>
> I'd like to call for a vote on KIP-738:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-738%3A+Removal+of+Connect%27s+internal+converter+properties
>
> The discussion thread (which was
> Akka. One of the questions I would have is how you would handle back
> > pressure and avoid memory exhaustion when the producer's buffer is
> > full and tasks would start to accumulate in the out-of-band queue or
> > thread pool introduced with this KIP.
> >
> > Thanks
hat you think. The voting thread is open.
Ryanne
On Fri, Apr 9, 2021, 1:41 PM Ryanne Dolan wrote:
> Hey y'all, I'd like to draw you attention to a new KIP:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-731%3A+Record+Rate+Limiting+for+Kafka+Connect
>
> Lemme know w
t;
> Yes, I think some exceptions will move to being async instead of sync.
> They'll still be surfaced in the Future, so I'm not so confident that it
> would be that big a shock to users though.
>
> Best,
> Moses
>
> On Thu, May 13, 2021 at 7:44 PM Ryanne Dolan
> wrote:
of the semantics of this
> method. It doesn't seem like serialization depends on knowing the cluster,
> I think it's incidental that it comes after the first "blocking" segment in
> the method.
>
> Best,
> Moses
>
> On Thu, May 13, 2021 at 2:38 PM Ryanne Dol
Hey Moses, I like the direction here. My thinking is that a single
additional work queue, s.t. send() can enqueue and return, seems like the
lightest touch. However, I don't think we can trivially process that queue
in an internal thread pool without subtly changing behavior for some users.
For
+1 (non-binding) Thanks!
Ryanne
On Wed, May 5, 2021, 4:04 PM Randall Hauch wrote:
> I'd like to start a vote on KIP-722:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-722%3A+Enable+connector+client+overrides+by+default
>
> +1 (binding) from myself.
>
> Thanks, and best regards!
>
>
[
https://issues.apache.org/jira/browse/KAFKA-12726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryanne Dolan resolved KAFKA-12726.
--
Resolution: Fixed
Closing as duplicate.
> misbehaving Task.stop() can prevent other Ta
+1 (non-binding), thanks!
On Thu, Jan 21, 2021, 4:31 AM Omnia Ibrahim wrote:
> Hi
> Can I get a vote on this, please?
>
> Best
> Omnia
>
> On Tue, Dec 15, 2020 at 12:16 PM Omnia Ibrahim
> wrote:
>
>> If anyone interested in reading the discussions you can find it here
>>
eckpoints.internal,
>> "mm2-offset-syncs..internal", heartbeat) without any
>> customisation using `replication.policy.separator` and use the separator in
>> the DefaultReplicationPolicy
>>
>> On Wed, Apr 28, 2021 at 1:31 AM Ryanne Dolan
>> wr
Hey y'all, I'd like to start the vote on KIP-731, which enables operators
to limit Connector throughput.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-731%3A+Record+Rate+Limiting+for+Kafka+Connect
Thanks for your votes!
Ryanne
Ryanne Dolan created KAFKA-12726:
Summary: misbehaving Task.stop() can prevent other Tasks from
stopping
Key: KAFKA-12726
URL: https://issues.apache.org/jira/browse/KAFKA-12726
Project: Kafka
t;>> if (upstream == null) {
>>> return topic;
>>> } else {
>>> return originalTopic(upstream);
>>> }
>>> }
>>>
>>>
>>> */** Internal topics are never replicated.
Bump!
On Fri, Apr 9, 2021, 1:41 PM Ryanne Dolan wrote:
> Hey y'all, I'd like to draw you attention to a new KIP:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-731%3A+Record+Rate+Limiting+for+Kafka+Connect
>
> Lemme know what you think. Thanks!
>
> Ryanne
>
Ryanne Dolan created KAFKA-12645:
Summary: KIP-731: Record Rate Limiting for Kafka Connect
Key: KAFKA-12645
URL: https://issues.apache.org/jira/browse/KAFKA-12645
Project: Kafka
Issue Type
Hey y'all, I'd like to draw you attention to a new KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-731%3A+Record+Rate+Limiting+for+Kafka+Connect
Lemme know what you think. Thanks!
Ryanne
es to reliability and functionality for the majority of use
> cases. We intend to focus on MirrorMaker 2 for future development and hence
> we propose deprecating MirrorMaker 2 for future removal."
>
> Is this accurate? How does it sound?
>
> Ismael
>
> On Thu, Apr 1, 2021 at 9:10
r 2 for future development and
> hence
> > we propose deprecating MirrorMaker 2 for future removal."
> >
> > Is this accurate? How does it sound?
> >
> > Ismael
> >
> > On Thu, Apr 1, 2021 at 9:10 AM Ryanne Dolan
> wrote:
> >
> > > Ah, d
gt; > > > 2. For the concern on transactional message support as in KIP-98,
> > > since
> > > > > MM1
> > > > > > and MM2 currently don't support transactional messages, KIP-712
> > will
> > > > not
> > > > >
ange.
>
> On Thu, Apr 1, 2021, 8:46 AM Ryanne Dolan wrote:
>
> > Hey y'all, looks like we've got the requisite votes for this to pass, and
> > the various concerns wrt KIP-712 are now being discussed on that thread.
> So
> > I'm going to go ahead and close the vote here
ll the MM1 use cases would
> be
> > > > Ismael's explanation about why MM1 is no longer useful, I think. OTOH
> > the
> > > > KIP says it is still useful. So personally I'm confused about which
> the
> > > > situation is.
> > > >
> > > &g
+1, thanks!
On Mon, Mar 29, 2021, 10:54 AM Ismael Juma wrote:
> Thanks for the KIP, +1 (binding).
>
> Ismael
>
> On Mon, Mar 29, 2021 at 8:35 AM Tom Bentley wrote:
>
> > Hi,
> >
> > I'd like to start a vote on KIP-707, which proposes to add
> > KafkaFuture.toCompletionStage(), deprecate
e
> network throughput of the mirror maker producer when the original batch
> size from source broker is too small."
>
> This is unrelated to shallow iteration.
>
> Ismael
>
> On Sun, Mar 28, 2021, 10:15 PM Ryanne Dolan wrote:
>
> > Ismael, I don't think KIP-98 i
Ismael, I don't think KIP-98 is related. Shallow iteration was removed in
KAFKA-732, which predates KIP-98 by a few years.
Ryanne
On Sun, Mar 28, 2021, 11:25 PM Ismael Juma wrote:
> Thanks for the KIP. I have a few high level comments:
>
> 1. Like Tom, I'm not convinced about the proposal to
e
> > KIP says it is still useful. So personally I'm confused about which the
> > situation is.
> >
> > Are we deprecating something which for some users MM2 cannot replace? If
> > so, I think the KIP should explain clearly why we're intentionally doing
> > tha
th these KIPs would be a confusing thing to try to communicate to
> > users.
> >
> > Tom
> >
> > On Fri, Mar 26, 2021 at 5:12 PM Ryanne Dolan
> wrote:
> >
> > > Tom, to clarify, MM2 can definitely replace MM1 in all use cases I've
> > >
> not performing any transformation or repartitioning in our case.
> > >
> > > Thanks,
> > > - Ambud
> > >
> > > On Mon, Feb 22, 2021 at 9:20 AM Vahid Hashemian <
> > vahid.hashem...@gmail.com>
> > > wrote:
> > >
> &
2 cannot replace? If
> so, I think the KIP should explain clearly why we're intentionally doing
> that.
>
> Kind regards,
>
> Tom
>
> On Fri, Mar 26, 2021 at 3:22 PM Ryanne Dolan
> wrote:
>
> > Ismael, I think it is very difficult in general to argue for deprecation
or release to officially deprecate
> this legacy code". I would hope we would explain why it's no longer useful
> instead.
>
> Thanks,
> Ismael
>
> On Sat, Mar 20, 2021 at 10:41 AM Ryanne Dolan
> wrote:
>
> > Hey y'all, I'm starting the vote on KIP-720, which prop
Omnia, have we considered just adding methods to ReplicationPolicy? I'm
reluctant to add a new class because, as Mickael points out, we'd need to
carry it around in client code.
Ryanne
On Fri, Feb 19, 2021 at 8:31 AM Mickael Maison
wrote:
> Hi Omnia,
>
> Thanks for the clarifications.
>
> -
+1 from me. Super looking forward to this.
But N.B. it looks like KIP-720 will pass, which deprecates MM1. I don't
think there is any reason both KIPs can't pass, but it looks like any
classes introduced in KIP-712 would get immediately deprecated.
Ryanne
On Thu, Mar 25, 2021, 12:34 PM Henry
Hey y'all, I'm starting the vote on KIP-720, which proposes to deprecate
the original MirrorMaker in the upcoming 3.0 major release.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-720%3A+Deprecate+MirrorMaker+v1
Thanks!
Ryanne
My two cents: keep the same method and class names and just use a different
package. Strongly dislike coming up with slightly different names for
everything.
Ryanne
On Tue, Feb 2, 2021, 4:41 AM Tom Bentley wrote:
> I've previously discounted the possibility of an "Admin2" client, but
> seeing
Looks good to me, I'll vote +1.
Ryanne
On Thu, Mar 18, 2021, 10:36 AM Mickael Maison
wrote:
> Hi,
>
> I've not seen any feedback on this KIP.
> As it's relatively straight forward, I'll open a vote in the next few
> days if I don't see anything.
>
> Thanks
>
> On Fri, Feb 26, 2021 at 6:20 PM
gt;
> Kind regards
> Georg Friedrich
>
> -Original Message-
> From: Ryanne Dolan
> Sent: Wednesday, March 17, 2021 4:57 AM
> To: dev
> Subject: Re: MirrorMaker 2.0 - Offset Sync - Questions/Improvements
>
> Georg, sorry for the delay, but hopefully I can still
out the documentation? Currently, there doesn't seem to be much
> documentation around Mirror Maker 2. Is there a plan to address that?
>
> On Wed, 17 Mar 2021 at 10:56, Manikumar wrote:
>
> > +1. Thanks for the KIP.
> >
> >
> > On Sun, Mar 14, 2021 at 12:24 PM Ryann
Georg, sorry for the delay, but hopefully I can still help.
1) I think the detail you're missing is that the offset syncs are very
sparse. Normally, you only get a new sync when the Connector first starts
running. You are right that it is possible for a consumer to lag behind the
most recent
Thanks Randall, makes sense to me.
Ryanne
On Tue, Mar 16, 2021, 4:31 PM Randall Hauch wrote:
> Hello all,
>
> I'd like to propose KIP-722 to change the default value of the existing
> `connector.client.config.override.policy` Connect worker configuration, so
> that by default connectors can
Hey y'all, I'd like to start the discussion on KIP-720, which proposes to
deprecate the original MirrorMaker in the upcoming 3.0 major release.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-720%3A+Deprecate+MirrorMaker+v1
Thanks!
Ryanne
Ryanne Dolan created KAFKA-12436:
Summary: deprecate MirrorMaker v1
Key: KAFKA-12436
URL: https://issues.apache.org/jira/browse/KAFKA-12436
Project: Kafka
Issue Type: Improvement
t I've omitted the KIP algorithm implementation details based on received
> feedback. But I acknowledge that this information can be put in the KIP for
> better clarity. I took the liberty of updating the KIP with the example
> mentioned above [1].
> I hope this answeres your question.
>
&g
I guess I don't understand how multiple tags work together to achieve rack
awareness. I realize I could go look at how Elasticseach works, but ideally
this would be more plain in the KIP.
In particular I'm not sure how the tag approach is different than appending
multiple tags together, e.g. how
i, Feb 12, 2021 at 12:20 PM Ryanne Dolan
> wrote:
>
>> Hey Henry, great KIP. The performance improvements are impressive!
>> However, often cpu, ram, gc are not the metrics most important to a
>> replication pipeline -- often the network is mostly saturated anyway. Do
>&g
Hey Henry, great KIP. The performance improvements are impressive! However,
often cpu, ram, gc are not the metrics most important to a replication
pipeline -- often the network is mostly saturated anyway. Do you know how
this change affects latency or thruput? I suspect less GC pressure means
Thanks Omnia, this looks great. I like the approach of introducing another
policy class.
Ryanne
On Tue, Dec 1, 2020, 9:36 AM Omnia Ibrahim wrote:
> Hi everyone
> I want to start discussion of the KIP 690, the proposal is here
>
>
Thanks Mickael, the KIP makes sense to me, esp for cases where an external
system (like cruise control or an operator) knows more about the target
cluster state than the broker does.
Ryanne
On Thu, Aug 20, 2020, 10:46 AM Mickael Maison
wrote:
> Hi,
>
> I've created KIP-660 to make the replica
Awesome, this will be a huge advancement. I also want to point out that
this KIP implements MirrorSinkConnector as well, finally, which is a very
often requested missing feature in my experience.
Ryanne
On Fri, Aug 21, 2020, 9:45 AM Ning Zhang wrote:
> Hello, I wrote a KIP about MirrorMaker2
Xavier, I'm dismayed to see some of these instances are my fault. Fully
support your plan.
John, I had the same thought -- "list" is extraneous here. In the case of
"topics.whitelist" we already have precedent to just use "topics".
Ryanne
On Sat, Jun 20, 2020, 12:43 PM John Roesler wrote:
>
Nitin, there is a recently accepted KIP to add the formatters here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters.
Not
sure if there is a JIRA or PR for the KIP yet.
MirrorCheckpointConnector MBeans won't get created until there are
1 - 100 of 282 matches
Mail list logo
