Re: [DISCUSS] KIP-845: HasField predicate for kafka connect

Hi Kumud, Thanks for the KIP. I'm a little bit skeptical about the necessity for this predicate but I think we may be able to satisfy your requirements with a slightly different approach. The motivation section deals largely with skipping the invocation of SMTs that expect a certain field to be

Re: [VOTE] KIP-827: Expose logdirs total and usable space via Kafka API

Thanks for the KIP Mickael, I'm +1 (binding) -John On Tue, May 31, 2022, at 18:48, Jun Rao wrote: > Hi, Mickael, > > Thanks for the KIP. +1 > > Jun > > On Wed, May 25, 2022 at 7:54 AM Tom Bentley wrote: > >> Hi Mickael, >> >> Thanks for the KIP! +1 (binding). >> >> Kind regards, >> >> Tom >>

Re: [DISCUSS] KIP-787 - MM2 Interface to manage Kafka resources

Hi Omnia, Thank you for your patience with this KIP! I have a few quick thoughts: 1. Is the introduction of the new "ForwardingAdmin" class necessary, or can the same behavior can be achieved by subclassing the existing KafkaAdminClient class? 2. Would it be just as accurate to name the new

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #964

See

Re: [VOTE] KIP-827: Expose logdirs total and usable space via Kafka API

Hi, Mickael, Thanks for the KIP. +1 Jun On Wed, May 25, 2022 at 7:54 AM Tom Bentley wrote: > Hi Mickael, > > Thanks for the KIP! +1 (binding). > > Kind regards, > > Tom > > On Thu, 19 May 2022 at 11:28, Federico Valeri > wrote: > > > Thanks Mickael. > > > > +1 (non binding) > > > > On Wed,

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hi, Mickael, Thanks for the explanation. The KIP looks to me now. Jun On Tue, May 31, 2022 at 6:44 AM Mickael Maison wrote: > Hi Jun, > > Igor answered to your question. > Users should rely on their host metrics to monitor disk usage. But > with tooling and automation it's sometimes not ideal

Re: [DISCUSS] KIP-844: Transactional State Stores

Hey, Thank you for the wealth of great suggestions and questions! I am going to address the feedback in batches and update the proposal async, as it is probably going to be easier for everyone. I will also write a separate message after making updates to the KIP. @John, > Did you consider

Re: [DISCUSS] KIP-714: Client metrics and observability

Hi, Magnus, Thanks for the reply. 51. I think it's fine to have a list of recommended metrics for every client to implement. I am just not sure that standardizing on the metric names across all clients is practical. The list of common metrics in the KIP have completely different names from the

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

Hey Ismael, just checking do you mean the `metric` method instead? On Tue, May 31, 2022 at 1:45 PM Ismael Juma wrote: > Should we deprecate the `sensor` method? One other thing to take into > account is that these methods are meant to be used like a dsl for > configuring sensors and metrics. So

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

Should we deprecate the `sensor` method? One other thing to take into account is that these methods are meant to be used like a dsl for configuring sensors and metrics. So brevity is a plus (but clarity is critical still). Ismael On Tue, May 31, 2022 at 11:09 AM John Roesler wrote: >

Re: [VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

> We should add a comment in AlterPartitionResponse about the new error > code INELIGIBLE_REPLICA. This is very important for error codes so we > can track which ones are returned in which RPC version. I also wonder Here I'm referring to AlterPartitionResponse.json cheers, Colin > > > On

Re: [VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Hi David, Thanks again for working on this. It's an important one, for sure. It looks like the proposal has changed a bit so I'll re-review. With regard to the topic ID field in AlterPartitionRequest, I think this is a good idea. It's also good that it replaces the name instead of

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

Hi Sophie, Just a quick question: for filling the gap of sub-topology visibilities, would task-level produced metrics be sufficient? On Mon, May 30, 2022 at 10:59 AM Bill Bejeck wrote: > Thanks for the KIP Sophie. > > I'm in favor of this change as well. I don't have any comments in > addition

Re: [VOTE] KIP-846: Processor-level Streams metrics for records/bytes Producedd

+1 non binding On Tue, May 31, 2022 at 12:19 PM John Roesler wrote: > +1 (binding) > > Thanks, > John > > On Mon, May 30, 2022, at 13:00, Bill Bejeck wrote: > > +1 (binding) > > > > -Bill > > > > On Mon, May 30, 2022 at 4:49 AM Sagar wrote: > > > >> +1 (non-binding). > >> > >> Thanks! > >>

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

Generally, I agree with Ismael that having a new, weird name will make it hard to keep them straight. Then again, we need to make them different to prevent confusion about their semantics. To be clear, I'll be a +1 regardless of how we break this dilemma. One suggestion: We currently have

Re: [VOTE] KIP-846: Processor-level Streams metrics for records/bytes Producedd

+1 (binding) Thanks, John On Mon, May 30, 2022, at 13:00, Bill Bejeck wrote: > +1 (binding) > > -Bill > > On Mon, May 30, 2022 at 4:49 AM Sagar wrote: > >> +1 (non-binding). >> >> Thanks! >> Sagar. >> >> On Mon, May 30, 2022 at 1:11 PM Bruno Cadonna wrote: >> >> > +1 (binding) >> > >> >

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

Oh yeah there's another metric function which is get-only. I think we should go ahead with getOrCreateMetric. Thanks! Sagar. On Tue, May 31, 2022 at 10:02 PM Guozhang Wang wrote: > I'd prefer the getOrCreateMetric function name, since for the existing " > sensor(String name)" function that

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

I'd prefer the getOrCreateMetric function name, since for the existing " sensor(String name)" function that only takes a single `String` parameter, its semantics is already "get or create". Whereas the existing "metric(MetricName)" function's semantics is "get" only. So in my mind, the

[VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Hi folks, I'd like to start a vote for KIP-841: https://cwiki.apache.org/confluence/x/phmhD. Thanks, David

Re: [Vote] KIP-787 - MM2 Interface to manage Kafka resources

Hi Omnia, I think the approach you settled on is the best option, this will allow integrating MirrorMaker in more environments. +1 binding Thanks for the KIP (and your persistence!) Mickael On Mon, May 30, 2022 at 12:09 PM Omnia Ibrahim wrote: > > Hi, > Can I get a vote on this, please? >

Re: [DISCUSS] KIP-660: Pluggable ReplicaPlacer

Hi Vikas, You make some very good points and most importantly I agree that being able to prevent putting new partitions on a broker should be part of Kafka itself and not require a plugin. This feature would addresses 2 out of the 3 scenarios mentioned in the motivation section. The last one

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hi Jun, Igor answered to your question. Users should rely on their host metrics to monitor disk usage. But with tooling and automation it's sometimes not ideal to retrieve values from metrics. So exposing disk usage via the Kafka API will simplify coordinating disk operations. I've updated the

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hi Raman, Unfortunately the replica size only includes the log files and it does not include indexes or other metadata files. Obviously any extra non Kafka files are also not included either. For these reasons, I decided to have a separate field with the actual usable space reported by the

[jira] [Created] (KAFKA-13951) Fix flaky test KRaftClusterTest.testCreateClusterAndCreateListDeleteTopic

Divij Vaidya created KAFKA-13951: Summary: Fix flaky test KRaftClusterTest.testCreateClusterAndCreateListDeleteTopic Key: KAFKA-13951 URL: https://issues.apache.org/jira/browse/KAFKA-13951 Project:

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Hi all, Thanks for your feedback. I just updated the KIP as follows: * I propose to add a `TopicId` field at the same time while we are bumping the AlterPartition API version. * I propose to add the `InShuttingDown` field to the registration records to track if a broker is in controlled

Re: [DISCUSS] KIP-714: Client metrics and observability

Den fre 20 maj 2022 kl 01:23 skrev Jun Rao : > Hi, Magus, > > Thanks for the reply. > > 50. Sounds good. > > 51. I miss-understood the proposal in the KIP then. The proposal is to > define a set of common metric names that every client should implement. The > problem is that every client already

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Hi Cong, Kafka does not expose disk utilization metrics. This is something you need to provide in your environment. You definitively should have a mechanism for exposing metrics from your Kafka broker hosts and you should absolutely monitor disk usage and have appropriate alerts. Thanks, Mickael

[VOTE] KIP-837 Allow MultiCasting a Result Record.

Hi All, I would like to start a voting thread on https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211883356. I am just starting this as the discussion thread has been open for 10+ days. In case there are some comments, we can always discuss them over there. Thanks! Sagar.

[jira] [Created] (KAFKA-13950) Resource leak at multiple places in the code

Divij Vaidya created KAFKA-13950: Summary: Resource leak at multiple places in the code Key: KAFKA-13950 URL: https://issues.apache.org/jira/browse/KAFKA-13950 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-13949) Connect /connectors endpoint should support querying the active topics and the task configs

Viktor Somogyi-Vass created KAFKA-13949: --- Summary: Connect /connectors endpoint should support querying the active topics and the task configs Key: KAFKA-13949 URL:

Re: [DISCUSS] KIP-843 Adding metricOrElseCreate method to Metrics.

Hi Sagar, Yes, I think I misunderstood your KIP. I have no other comments, then. Thank you. Luke On Tue, May 31, 2022 at 12:46 PM Sagar wrote: > Hey Luke, > > Actually the addMetric still throws IllegalArgumentException. The new > method that is being introduced as part of the KIP doesn't