Re: [DISCUSS] KIP-844: Transactional State Stores

2022-05-24 Thread Guozhang Wang
Hello Alex, Thanks for writing the proposal! Glad to see it coming. I think this is the kind of a KIP that since too many devils would be buried in the details and it's better to start working on a POC, either in parallel, or before we resume our discussion, rather than blocking any

Re: [VOTE] KIP-836: Addition of Information in DescribeQuorumResponse about Voter Lag

2022-05-24 Thread José Armando García Sancio
+1 (binding). On Mon, May 23, 2022 at 11:52 AM Niket Goel wrote: > > Thanks everyone for the feedback on this. KIP-836 was approved with > 3 binding votes from Luke, David Arthur and David Jacot. > - Niket > > > On May 20, 2022, at 11:44 PM, David Jacot wrote: > > > > +1 (binding) > > > >

Re: [DISCUSS] KIP-836: Addition of Information in DescribeQuorumResponse about Voter Lag

2022-05-24 Thread José Armando García Sancio
On Fri, May 20, 2022 at 4:04 PM Niket Goel wrote: > > So Ideally the LastCatchUpTimeMs is supposed to be a proxy for lag. We could > report this field as lag, but I am personally not in favor of doing that as > the LastCaughtUpTimeMs is just an approximation of lag, but not actual lag. > > I

[jira] [Created] (KAFKA-13935) Factor out IBP from Partition and Log code

2022-05-24 Thread David Arthur (Jira)
David Arthur created KAFKA-13935: Summary: Factor out IBP from Partition and Log code Key: KAFKA-13935 URL: https://issues.apache.org/jira/browse/KAFKA-13935 Project: Kafka Issue Type:

[ANNOUNCE] Call for Speakers is open for Current 2022: The Next Generation of Kafka Summit

2022-05-24 Thread Robin Moffatt
Hi everyone, We’re very excited to announce our Call for Speakers for Current 2022: The Next Generation of Kafka Summit! With the permission of the ASF, Current will include Kafka Summit as part of the event. We’re looking for talks about all aspects of event-driven design, streaming

Re: [VOTE] KIP-714: Client Metrics and Observability

2022-05-24 Thread Jason Gustafson
+1 Thanks Magnus! On Tue, May 17, 2022 at 5:43 AM Magnus Edenhill wrote: > Hey all, > > It's that time of year again where we re-restart this vote thread after > some additional > discussions on the disco thread and minor adjustments to the > KIP. > > We're currently at +5 (non-binding) and -1

[jira] [Created] (KAFKA-13934) Consider consolidating TimeWindow / SessionWindow / SlidingWindow

2022-05-24 Thread Guozhang Wang (Jira)
Guozhang Wang created KAFKA-13934: - Summary: Consider consolidating TimeWindow / SessionWindow / SlidingWindow Key: KAFKA-13934 URL: https://issues.apache.org/jira/browse/KAFKA-13934 Project: Kafka

Re: [DISCUSS] KIP-844: Transactional State Stores

2022-05-24 Thread Sagar
Hi Alexander, Thanks for the KIP! This seems like a great proposal. I have the same opinion as John on the Configuration part though. I think the 2 level config and its behaviour based on the setting/unsetting of the flag seems confusing to me as well. Since the KIP seems specifically centred

Re: [DISCUSS] KIP-837 Allow MultiCasting a Result Record.

2022-05-24 Thread Sagar
Hi All, Bumping this discussion thread again to see if there are any comments/concerns on this. Thanks! Sagar. On Wed, May 18, 2022 at 11:44 PM Sagar wrote: > Hi All, > > I would like to open a discussion thread on > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211883356 >

Re: [DISCUSS] KIP-844: Transactional State Stores

2022-05-24 Thread John Roesler
Thanks for the KIP, Alex! I'm really happy to see your proposal. This improvement fills a long-standing gap. I have a few questions: 1. Configuration The KIP only mentions RocksDB, but of course, Streams also ships with an InMemory store, and users also plug in their own custom state stores.

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread Bruno Cadonna
Hi Sagar, +1 (binding) Thanks, Bruno On 24.05.22 18:22, Sagar wrote: Hi John, No worries. addMetric throws an exception which needs to be handled. This new method removes that need similar to what you have done for the sensor method. Thanks! Sagar. On Tue, May 24, 2022 at 8:19 PM Guozhang

Re: [DISCUSS] KIP-843 Adding metricOrElseCreate method to Metrics.

2022-05-24 Thread Bruno Cadonna
Hi Sagar, Thank you for the updates! 2. Great if we just need one overload! Best, Bruno On 24.05.22 18:30, Sagar wrote: Hi Bruno, Thanks for your comments. 1) I updated the header in the public interfaces section. 2) Well, the overloads were there initially but then it was suggested to

Re: [DISCUSS] KIP-843 Adding metricOrElseCreate method to Metrics.

2022-05-24 Thread Sagar
Hi Bruno, Thanks for your comments. 1) I updated the header in the public interfaces section. 2) Well, the overloads were there initially but then it was suggested to remove them during PR review: https://github.com/apache/kafka/pull/12121#discussion_r875156714 3) I updated the KIP with the

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread Sagar
Hi John, No worries. addMetric throws an exception which needs to be handled. This new method removes that need similar to what you have done for the sensor method. Thanks! Sagar. On Tue, May 24, 2022 at 8:19 PM Guozhang Wang wrote: > +1. Thanks Sagar. > > Guozhang > > On Tue, May 24, 2022 at

Re: [DISCUSS] KIP-843 Adding metricOrElseCreate method to Metrics.

2022-05-24 Thread Bruno Cadonna
Sagar, Thanks for the KIP! 1. Could you please add the class to which you want to add the method in section "Public Interfaces"? Ideally, in the header of the code part that contains "prefixScan" currently. 2. Do you also plan to add overloads for metricOrElseCreate() similar as for

Re: [VOTE] KIP-833: Mark KRaft as Production Ready

2022-05-24 Thread Bill Bejeck
Hi Colin, Thanks for the KIP, I'm +1 (binding). -Bill On Tue, May 24, 2022 at 11:28 AM José Armando García Sancio wrote: > Hi Colin, > > Thank you for the KIP. +1 (binding). > > -- > -José >

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

2022-05-24 Thread José Armando García Sancio
Hi David, Thanks for the KIP. In the "Compatibility, Deprecation, and Migration Plan", you have: > The change is backward compatible. I assume that we don't need to increase the metadata.version/IBP for AlterPartition because AlterPartitionManager uses ApiVersions for that channel. Should we

Re: [VOTE] KIP-833: Mark KRaft as Production Ready

2022-05-24 Thread José Armando García Sancio
Hi Colin, Thank you for the KIP. +1 (binding). -- -José

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread Guozhang Wang
+1. Thanks Sagar. Guozhang On Tue, May 24, 2022 at 7:31 AM John Roesler wrote: > Hi again Sagar, > > My apologies; I was thinking of the `sensor` method: > org.apache.kafka.common.metrics.Metrics#sensor(java.lang.String, > org.apache.kafka.common.metrics.MetricConfig, long, >

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread John Roesler
Hi again Sagar, My apologies; I was thinking of the `sensor` method: org.apache.kafka.common.metrics.Metrics#sensor(java.lang.String, org.apache.kafka.common.metrics.MetricConfig, long, org.apache.kafka.common.metrics.Sensor.RecordingLevel, org.apache.kafka.common.metrics.Sensor...) I'm in

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread John Roesler
Hi Sagar, Thanks for the KIP! I’m not at my computer right now, but I think I confronted a similar problem a while back for the Streams metrics. I think that I already made the “addMetric” method to be idempotent, so if it’s already registered, the call just returns the old one instead of

Re: [VOTE] KIP-833: Mark KRaft as Production Ready

2022-05-24 Thread John Roesler
+1 (binding) from me. Thanks, Colin! -John On Tue, May 24, 2022, at 01:58, David Jacot wrote: > +1. Thanks Colin! > > On Tue, May 24, 2022 at 4:50 AM Luke Chen wrote: >> >> +1 from me. >> Kraft is coming!!! >> >> Luke >> >> On Tue, May 24, 2022 at 7:26 AM Israel Ekpo wrote: >> >> > +1

[VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

2022-05-24 Thread Sagar
Hi All, I would like to open a voting thread for the following KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-843%3A+Adding+metricOrElseCreate+method+to+Metrics Thanks! Sagar.

Re: [DISCUSS] KIP-843 Adding metricOrElseCreate method to Metrics.

2022-05-24 Thread Sagar
Thanks Guozhang. I made the edit. Let me open a VOTE thread. Thanks! Sagar. On Mon, May 23, 2022 at 10:19 PM Guozhang Wang wrote: > Thanks Sagar, > > The KIP looks good to me overall, I just have a minor comment on the > motivation of it: "The Metrics registry is often used by concurrent >

Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

2022-05-24 Thread Jorge Esteban Quilcate Otoya
Thanks, Chris! I have updated the KIP with the rejected alternatives updated. Also, I have drafted the support for arrays and deep scans as part of the proposed notation to make it more complete, even though these can be implemented in multiple PRs. Looking forward to your feedback. Cheers,

[DISCUSS] KIP-844: Transactional State Stores

2022-05-24 Thread Alexander Sorokoumov
Hi all, I've written a KIP for making Kafka Streams state stores transactional and would like to start a discussion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-844%3A+Transactional+State+Stores Best, Alex

[jira] [Created] (KAFKA-13933) Stuck SSL tests in case of authentication failure.

2022-05-24 Thread Federico Valeri (Jira)
Federico Valeri created KAFKA-13933: --- Summary: Stuck SSL tests in case of authentication failure. Key: KAFKA-13933 URL: https://issues.apache.org/jira/browse/KAFKA-13933 Project: Kafka

[jira] [Created] (KAFKA-13932) Replication data loss in some cases

2022-05-24 Thread Bale (Jira)
Bale created KAFKA-13932: Summary: Replication data loss in some cases Key: KAFKA-13932 URL: https://issues.apache.org/jira/browse/KAFKA-13932 Project: Kafka Issue Type: Bug Components:

Re: [VOTE] KIP-833: Mark KRaft as Production Ready

2022-05-24 Thread David Jacot
+1. Thanks Colin! On Tue, May 24, 2022 at 4:50 AM Luke Chen wrote: > > +1 from me. > Kraft is coming!!! > > Luke > > On Tue, May 24, 2022 at 7:26 AM Israel Ekpo wrote: > > > +1 (non-binding) from me. > > > > I am very happy to finally see this. > > > > On Mon, May 23, 2022 at 6:08 PM Jason