Re:[ANNOUNCE] New Kafka PMC Member: Greg Harris

2024-04-15 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Congrats! Well deserved From: dev@kafka.apache.org At: 04/13/24 14:42:22 UTC-4:00To: dev@kafka.apache.org Subject: [ANNOUNCE] New Kafka PMC Member: Greg Harris Hi all, Greg Harris has been a Kafka committer since July 2023. He has remained very active and instructive in the community since

Re:[DISCUSS] KIP-1028: Docker Official Image for Apache Kafka

2024-03-21 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi, What is the difference between this KIP and KIP-975: Docker Image for Apache Kafka? From: dev@kafka.apache.org At: 03/21/24 07:30:07 UTC-4:00To: dev@kafka.apache.org Subject: [DISCUSS] KIP-1028: Docker Official Image for Apache Kafka Hi everyone, I would like to start the discussion on

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-02 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
+1 (non-binding) Thanks Chris! From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect Hi all, Happy New Year! Wanted to give this a bump now that the holidays are over for a lot of us.

Re:[DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-20 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Thanks for the KIP Chris, adding this check makes total sense. I do have one question. The second paragraph in the Public Interfaces section states: "If the connector generated excessive tasks after being reconfigured, then any existing tasks for the connector will be allowed to continue

Re: [DISCUSS] KIP-987: Connect Static Assignments

2023-10-20 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi, I think different algorithms might work for different workload/scenarios. I have some thoughts that are somewhat tangential to this KIP: it might be a good idea to elevate the ConnectAssignor to the category of plugin, so users can provide their own implementation. The fact that there's

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-09 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Good stuff, +1 (non-binding) from me as well De: dev@kafka.apache.org A: 10/09/23 05:16:06 UTC-4:00A: dev@kafka.apache.org Subject: Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect Hi Chris, +1 (non binding) Thanks Fede On Sun, Oct 8, 2023 at 10:11 AM Yash Mayya

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-21 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Congrats! Well deserved From: dev@kafka.apache.org At: 09/21/23 17:05:01 UTC-4:00To: dev@kafka.apache.org Cc: r...@confluent.io.invalid Subject: Re: [ANNOUNCE] New committer: Yash Mayya Congratulations, Yash! On Thu 21. Sep 2023 at 21.57, Randall Hauch wrote: > Congratulations, Yash! > >

Re:[VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-30 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
This makes sense to me, +1 (non-binding) From: dev@kafka.apache.org At: 08/30/23 02:58:59 UTC-4:00To: dev@kafka.apache.org Subject: [VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint Hi all, This is the vote thread for KIP-970 which proposes deprecating (in

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-08-16 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
+1 (non-binding) Thanks for your KIP! From: dev@kafka.apache.org At: 08/16/23 04:48:13 UTC-4:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well. Thanks Sagar and Chris for your votes. I will add the details Chris has asked for

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

2023-08-10 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
: dev@kafka.apache.org At: 08/08/23 08:33:21 UTC-4:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect Hi, +1 (binding) Thanks for the KIP! On Mon, Aug 7, 2023 at 3:15 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) wrote: > > Hello, > > I still ne

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

2023-08-07 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
field < > > > andrew_schofield_j...@outlook.com> wrote: > > > > > > > Thanks for the KIP. As you say, not that controversial. > > > > > > > > +1 (non-binding) > > > > > > > > Thanks, > > > > Andrew &g

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

2023-08-01 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
t; > > Thanks for the KIP. As you say, not that controversial. > > > > > > > > +1 (non-binding) > > > > > > > > Thanks, > > > > Andrew > > > > > > > > > On 25 Jul 2023, at 18:22, Hector Geraldino

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

2023-07-28 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
t; > > andrew_schofield_j...@outlook.com> wrote: > > > > > > > Thanks for the KIP. As you say, not that controversial. > > > > > > > > +1 (non-binding) > > > > > > > > Thanks, > > > > Andrew > > >

Re: Apache Kafka 3.6.0 release

2023-07-26 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi Hector, > > Thanks for the update on KIP-959. > > > > ~Satish. > > > > On Wed, 26 Jul 2023 at 18:38, Hector Geraldino (BLOOMBERG/ 919 3RD A) > > wrote: > > > > > > Hi Satish, > > > > > > I added KIP-959 [1] to the list. The KIP has rece

Re: Apache Kafka 3.6.0 release

2023-07-26 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Thanks for the update on KIP-959. > > ~Satish. > > On Wed, 26 Jul 2023 at 18:38, Hector Geraldino (BLOOMBERG/ 919 3RD A) > wrote: > > > > Hi Satish, > > > > I added KIP-959 [1] to the list. The KIP has received enough votes to > pass, but I'm waiting the 72 hou

Re: Apache Kafka 3.6.0 release

2023-07-26 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi Satish, I added KIP-959 [1] to the list. The KIP has received enough votes to pass, but I'm waiting the 72 hours before announcing the results. There's also a (small) PR with the implementation for this KIP that hopefully will get reviewed/merged soon. Best, [1]

[VOTE] KIP-959 Add BooleanConverter to Kafka Connect

2023-07-25 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi everyone, The changes proposed by KIP-959 (Add BooleanConverter to Kafka Connect) have a limited scope and shouldn't be controversial. I'm opening a voting thread with the hope that it can be included in the next upcoming 3.6 release. Here are some links: KIP:

Re: [DISCUSS] KIP-959 Add BooleanConverter to Kafka Connect

2023-07-25 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
he BooleanSerializer and BooleanDeserializer introduced in KIP-907. Cheers, Chris On Tue, Jul 25, 2023 at 9:52 AM Hector Geraldino (BLOOMBERG/ 919 3RD A) < hgerald...@bloomberg.net> wrote: > Hi everyone, > > I'd like to start a discussion of KIP-959, which aims to add a > Boole

[DISCUSS] KIP-959 Add BooleanConverter to Kafka Connect

2023-07-25 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi everyone, I'd like to start a discussion of KIP-959, which aims to add a BooleanConverter to Kafka Connect: https://cwiki.apache.org/confluence/display/KAFKA/KIP-959%3A+Add+BooleanConverter+to+Kafka+Connect This KIP is a counterpart of KIP-907: Add Boolean Serde to public interface

Re: [ANNOUNCE] New committer: Greg Harris

2023-07-10 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Congrats Greg! Well deserved From: dev@kafka.apache.org At: 07/10/23 12:18:48 UTC-4:00To: dev@kafka.apache.org Subject: Re: [ANNOUNCE] New committer: Greg Harris Congratulations! On Mon, Jul 10, 2023 at 9:17 AM Randall Hauch wrote: > > Congratulations, Greg. > > On Mon, Jul 10, 2023 at 11:13 

Re: [VOTE] KIP-864: Add End-To-End Latency Metrics to Connectors

2023-05-11 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
This will help us greatly. +1 (non-binding) From: dev@kafka.apache.org At: 05/10/23 17:32:35 UTC-4:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-864: Add End-To-End Latency Metrics to Connectors Hi everyone, Bumping this vote thread. 2 +1 binding and 1 +1 non-binding so far. Cheers,

Re:[VOTE] KIP-911: Add source tag to MirrorSourceConnector metrics

2023-03-17 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
+1 (non-binding) From: dev@kafka.apache.org At: 03/15/23 07:08:33 UTC-4:00To: dev@kafka.apache.org Subject: [VOTE] KIP-911: Add source tag to MirrorSourceConnector metrics Hi, I'd like to start the vote on KIP-911 to add the source cluster alias as a tag on the MirrorSourceConnector metrics:

Re:[ANNOUNCE] New Kafka PMC Member: Chris Egerton

2023-03-10 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Congratulations Chris! Well deserved From: dev@kafka.apache.org At: 03/09/23 13:12:15 UTC-5:00To: dev@kafka.apache.org Subject: [ANNOUNCE] New Kafka PMC Member: Chris Egerton Hi, Everyone, Chris Egerton has been a Kafka committer since July 2022. He has been very instrumental to the community

[DISCUSS] KIP-901: Add connectorDeleted flag when stopping tasks

2023-01-24 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi everyone, I've submitted KIP-901, which adds an overloaded Task#stop(boolean connectorDeleted) method to the public Kafka Connect APIs: https://cwiki.apache.org/confluence/display/KAFKA/KIP-901%3A+Add+connectorDeleted+flag+when+stopping+tasks This KIP can be seen as a companion (or

Re: [DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-30 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
jected alternatives" section with the reasons why > I pivoted from introducing new callback methods to retrofit the existing > one. > > Please let me know what your thoughts are. > > Cheers, > Hector > > From: Hector Geraldino (BLOOMBERG/ 919 3RD A) At: 11/16/22 17:38:5

Re: [DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-17 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
e existing one. Please let me know what your thoughts are. Cheers, Hector From: Hector Geraldino (BLOOMBERG/ 919 3RD A) At: 11/16/22 17:38:59 UTC-5:00To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-883: Add delete callback method to Connector API Hi Mickael, I agree that the new STO

Re: [DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-16 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
; > compatibility? > > > > In terms of backward-compatibility for the framework vs connector versions, > > I think the default-noop method is very reasonable. > > However, what happens when someone upgrades from a version of a connector > > without a destroy() implementatio

Re: [DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-16 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
about that approach? Thanks! Sagar. On Sat, Nov 12, 2022 at 1:53 AM Hector Geraldino (BLOOMBERG/ 919 3RD A) < hgerald...@bloomberg.net> wrote: > Thanks Greg for taking your time to review not just the KIP but also the > PR. > > 1. You made very valid points regarding th

Re: [DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-11 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
ctor Geraldino (BLOOMBERG/ 919 3RD A) < hgerald...@bloomberg.net> wrote: > Hi everyone, > > I've submitted KIP-883, which introduces a callback to the public > Connector API called when deleting a connector: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-883%3A

[DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-03 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi everyone, I've submitted KIP-883, which introduces a callback to the public Connector API called when deleting a connector: https://cwiki.apache.org/confluence/display/KAFKA/KIP-883%3A+Add+delete+callback+method+to+Connector+API It adds a new `deleted()` method (open to better naming

Re:[DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-07 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi Mathieu. I took a look at your KIP and have a couple questions. If the goal is to do the partition assignments at a topic level, wouldn't having single-partition topics solve this problem? You also mentioned that your goal is to minimize the potential of a poison pill message breaking all

Connector API callbacks for create/delete events

2022-10-05 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi, We've some custom connectors that require provisioning external resources (think of creating queues, S3 buckets, or activating accounts) when the connector instance is created, but also need to cleanup these resources (delete, deactivate) when the connector instance is deleted. The

Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol

2022-09-12 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
, the schema registry wants to elect a single writer for a partition and the owner of the partition can be considered as the leader. I haven't really tried this out but that seems doable. What do you think? Best, David On Fri, Sep 9, 2022 at 8:45 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) wrote: > &

Re:[VOTE] KIP-848: The Next Generation of the Consumer Rebalance Protocol

2022-09-09 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
+1 (non-binding) Really looking forward for the discussion on how other group types (especially connect) will support this new protocol. From: dev@kafka.apache.org At: 09/09/22 04:32:46 UTC-4:00To: dev@kafka.apache.org Subject: [VOTE] KIP-848: The Next Generation of the Consumer Rebalance

Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol

2022-09-09 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
So it seems there's a consensus on having dedicated APIs for Connect, which means having data model (group, member, assignment) and APIs (heartbeat request/response, assignment prepare and install) tailored specifically to connect. I wonder if adding support for other coordinator group types

Re: Consumer Lag-Apache_kafka_JMX metrics

2022-08-16 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
As far as I know, such metric does not exist. Strictly speaking, consumer lag can be defined as the difference between the last produced offset (high watermark) and the last committed offset by the group, but such metric has very little value without considering the time dimension. It'd be

Re:[DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol

2022-07-14 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Kudos David, Guozhang, and Jason for putting together such a great proposal. I don't want to hijack the discussion, just wanted to mention that it would be great if the final design is made extensible enough, so other use cases (like Kafka Connect, Schema Registry, etc.) can be added later on.

Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator

2022-02-03 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
/22 05:01:41 UTC-5:00To: Hector Geraldino (BLOOMBERG/ 919 3RD A ) , dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator Hi Hector, Thanks for the KIP. I finally had a bit of time to read it. I understand that a few services have been leveraging Kafka's

Re: [DISCUSS] KIP-814: Static membership protocol should let the leader skip assignment

2022-01-24 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi David, Is the idea here to skip calling performAssignment(...) on the AbstractCoordinator.onJoinLeader(...) method, or adding a new boolean parameter to the performAssignment(...) method? The reason I ask is because I raised KIP-795 a few weeks back, which aims to add a public API for

Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator

2021-12-13 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
findings. Thanks again! Hector From: dev@kafka.apache.org At: 11/10/21 06:43:59 UTC-5:00To: Hector Geraldino (BLOOMBERG/ 919 3RD A ) , dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator Hi Hector, Thanks for the KIP. At a high level, I think

Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator

2021-11-29 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
. Thanks again! Hector From: dev@kafka.apache.org At: 11/10/21 06:43:59 UTC-5:00To: Hector Geraldino (BLOOMBERG/ 919 3RD A ) , dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator Hi Hector, Thanks for the KIP. At a high level, I think the question

Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator

2021-11-11 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
, see what it entails, and update the KIP with my findings. Thanks again! Hector From: dev@kafka.apache.org At: 11/10/21 06:43:59 UTC-5:00To: Hector Geraldino (BLOOMBERG/ 919 3RD A ) , dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator Hi Hector

[DISCUSS] KIP-795: Add public APIs for AbstractCoordinator

2021-11-08 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi Kafka devs, I would like to start the discussion of KIP-795: Add public APIs for AbstractCoordinator https://cwiki.apache.org/confluence/display/KAFKA/KIP-795%3A+Add+public+APIs+for+AbstractCoordinator Looking forward for some feedback from the community. Regards, Hector

DISMISS - Re:[DISCUSS] KIP-784: Add public APIs for AbstractCoordinator

2021-11-08 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Please dismiss this message, as the KIP number is wrong. I'll send a new message with the correct KIP shortly. Apologies From: dev@kafka.apache.org At: 11/08/21 08:45:22 UTC-5:00To: dev@kafka.apache.org Subject: [DISCUSS] KIP-784: Add public APIs for AbstractCoordinator Hi Kafka devs, I

[DISCUSS] KIP-784: Add public APIs for AbstractCoordinator

2021-11-08 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hi Kafka devs, I would like to start the discussion of KIP-784: Add public APIs for AbstractCoordinator https://cwiki.apache.org/confluence/display/KAFKA/KIP-784%3A+Add+public+APIs+for+AbstractCoordinator Looking forward for some feedback from the community. Regards, Hector

Wiki Permissions

2021-11-02 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Hello, I'd like to be added to the contributors list, so I can submit a KIP. My Jira ID is: hgeraldino Wiki ID: hgeraldino Thanks, Hector