[jira] [Resolved] (KAFKA-9102) Increase default zk session timeout and max lag

[ https://issues.apache.org/jira/browse/KAFKA-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9102. Resolution: Fixed > Increase default zk session timeout and max lag >

Jenkins build is back to normal : kafka-trunk-jdk11 #914

See

Jenkins build is back to normal : kafka-trunk-jdk8 #3999

See

[DISCUSS] KIP-544: Make metrics exposed via JMX configurable

Hi All, I wrote a short KIP to make the set of metrics exposed via JMX configurable. https://cwiki.apache.org/confluence/display/KAFKA/KIP-544%3A+Make+metrics+exposed+via+JMX+configurable Let me know what you think. Thanks, Xavier

[jira] [Created] (KAFKA-9106) metrics exposed via JMX shoud be configurable

Xavier Léauté created KAFKA-9106: Summary: metrics exposed via JMX shoud be configurable Key: KAFKA-9106 URL: https://issues.apache.org/jira/browse/KAFKA-9106 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

+1 to Sophie's suggestion. Having both lag in terms of time and offsets is good and makes for a more complete API. Since there are two soft votes for separate active/standby API methods, I also change my position on that. Fine with 2 separate methods. Once we remove the lag information from these

[jira] [Resolved] (KAFKA-9105) Truncate producer state when incrementing log start offset

[ https://issues.apache.org/jira/browse/KAFKA-9105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9105. Resolution: Fixed > Truncate producer state when incrementing log start offset >

[jira] [Resolved] (KAFKA-8968) Refactor Task-level Metrics

[ https://issues.apache.org/jira/browse/KAFKA-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-8968. Resolution: Fixed > Refactor Task-level Metrics > --- > >

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

I am jumping in a little late here. Overall I agree with the proposal to push decision making on what/how to query in the query layer. For point 5 from above, I'm slightly in favor of having a new method, "standbyMetadataForKey()" or something similar. Because even if we return all tasks in one

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Guozhang, 1. I am familiar with the cogroup of spark, it is very similar to their join operator but instead it makes the values iterable. I think that the use cases are different enough that it makes sense to specify the aggregator when we do. I like the idea of "absorb" and I think it could

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Walker, thanks for the KIP! I made a pass on the writeup and have some comments below: Meta: 1. Syntax-wise, I'm wondering if we have compared our current proposal with Spark's co-group syntax (I know they are targeting for different use cases, but wondering if their syntax is closer to the

Build failed in Jenkins: kafka-2.4-jdk8 #40

See Changes: [jason] KAFKA-9038; Allow creating partitions while a reassignment is in [jason] KAFKA-8992; Redefine RemoveMembersFromGroup interface on AdminClient --

Re: [VOTE] KIP-514 Add a bounded flush() API to Kafka Producer

+1 (binding) -Harsha On Fri, Oct 25 2019 at 11:01 AM, wrote: > > +1 > > On Thu, Oct 24, 2019 at 9:33 PM radai wrote: > > Hello, > > > > > I'd like to initiate a vote on KIP-514. > > > > > links: > the kip - > >

Re: [VOTE] KIP-514 Add a bounded flush() API to Kafka Producer

+1 On Thu, Oct 24, 2019 at 9:33 PM radai wrote: > Hello, > > I'd like to initiate a vote on KIP-514. > > links: > the kip - > https://cwiki.apache.org/confluence/display/KAFKA/KIP-514%3A+Add+a+bounded+flush%28%29+API+to+Kafka+Producer > the PR - https://github.com/apache/kafka/pull/7569 > >

[jira] [Created] (KAFKA-9105) Truncate producer state when incrementing log start offset

Bob Barrett created KAFKA-9105: -- Summary: Truncate producer state when incrementing log start offset Key: KAFKA-9105 URL: https://issues.apache.org/jira/browse/KAFKA-9105 Project: Kafka Issue

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Matthias, I am not familiar with KIP-307. After looking into it a bit I think I understand better what Named is for and it answers some of my concerns. I think that we should consider how many more methods it would make, there would be 12 aggregate functions including timeWindowed and

Re: [DISCUSS] Apache Kafka 2.4.0 release

Hi all, Quick update on the 2.4 release. We still have one blocker to close. I will create the first RC after closing the blocker. https://issues.apache.org/jira/browse/KAFKA-8972 Thank you! On Fri, Oct 18, 2019 at 12:51 AM Matthias J. Sax wrote: > Just FYI: > > There was also

Build failed in Jenkins: kafka-trunk-jdk8 #3998

See Changes: [github] KAFKA-9089; Reassignment should be resilient to unexpected errors [github] KAFKA-8729: Add upgrade docs for KIP-467 on augmented produce response [wangguoz] KAFKA-8972: Need to flush

Subject: [VOTE] 2.2.2 RC2

Hello all, we identified around three dozen bug fixes, including an update of a third party dependency, and wanted to release a patch release for the Apache Kafka 2.2.0 release. This is the *second* candidate for release of Apache Kafka 2.2.2. (RC1 did not include a fix for

Build failed in Jenkins: kafka-trunk-jdk11 #913

See Changes: [jason] KAFKA-8992; Redefine RemoveMembersFromGroup interface on AdminClient -- [...truncated 2.71 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Re: [VOTE] KIP-541: Create a fetch.max.bytes configuration for the broker

+1 binding, this will be a nice improvement. Thanks, Colin! -David On Fri, Oct 25, 2019 at 4:33 AM Tom Bentley wrote: > +1 nb. Thanks! > > On Fri, Oct 25, 2019 at 7:43 AM Ismael Juma wrote: > > > +1 (binding) > > > > On Thu, Oct 24, 2019, 4:56 PM Colin McCabe wrote: > > > > > Hi all, > > > >

Build failed in Jenkins: kafka-2.4-jdk8 #39

See Changes: [wangguoz] KAFKA-8729: Add upgrade docs for KIP-467 on augmented produce response [jason] KAFKA-9089; Reassignment should be resilient to unexpected errors [wangguoz] KAFKA-8972: Need to flush state

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Hi All, Would like to bump this in the hope of one binding vote (or any additional feedback). Thanks, Viktor On Wed, Sep 18, 2019 at 5:25 PM Viktor Somogyi-Vass wrote: > Hi All, > > Harsha, Ryanne: thanks for the vote! > > I'd like to bump this again as today is the KIP freeze date and there

Build failed in Jenkins: kafka-trunk-jdk11 #912

See Changes: [github] KAFKA-9089; Reassignment should be resilient to unexpected errors [github] KAFKA-8729: Add upgrade docs for KIP-467 on augmented produce response [wangguoz] KAFKA-8972: Need to flush

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

I think now we are aligned on almost all the design parts. Summarising below what has been discussed above and we have a general consensus on. - Rather than broadcasting lag across all nodes at rebalancing/with the heartbeat, we will just return a list of all available standby’s in the

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

I think now we are aligned on almost all the design parts. Summarising below what has been discussed above and we have a general consensus on. 1. Rather than broadcasting lag across all nodes at rebalancing/with the heartbeat, we will just return a list of all available standby’s in the system

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

>Do you want to discount supporting compacted topics this early in the KIP design or do you want to leave open the option of supporting them eventually? We are not targeting compact topics in this KIP. This KIP's goal is to address non-compact topics with tiered storage work well. Generally,

Re: [VOTE] KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'

Hi Bill, Yes we will deprecate 'UsePreviousTimeOnInvalidTimeStamp' and then add 'UsePartitionOnInvalidTimeStamp'. With Best Regards, Rabi Kumar K C On Thu, Oct 24, 2019 at 5:32 PM Guozhang Wang wrote: > +1 (binding). > > > Guozhang > > On Thu, Oct 24, 2019 at 8:22 AM Bill Bejeck wrote: > > >

Re: [VOTE] KIP-541: Create a fetch.max.bytes configuration for the broker

+1 nb. Thanks! On Fri, Oct 25, 2019 at 7:43 AM Ismael Juma wrote: > +1 (binding) > > On Thu, Oct 24, 2019, 4:56 PM Colin McCabe wrote: > > > Hi all, > > > > I'd like to start the vote on KIP-541: Create a fetch.max.bytes > > configuration for the broker. > > > > KIP:

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

Hello all, To best shape up KIP-455 before 2.4 releases, we had some very minor last-minute improvements which I'd like to share for the record. 1) MINOR: Check against empty replicas in AlterPartitionReassignments ( https://github.com/apache/kafka/commit/78e7c90e90efa18b2a5b298e49154834d8d5bf67

[jira] [Resolved] (KAFKA-8992) Don't expose Errors in `RemoveMemberFromGroupResult`

[ https://issues.apache.org/jira/browse/KAFKA-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8992. Resolution: Fixed > Don't expose Errors in `RemoveMemberFromGroupResult` >

[jira] [Created] (KAFKA-9104) Allow AdminClient to manage users resources

Jakub created KAFKA-9104: Summary: Allow AdminClient to manage users resources Key: KAFKA-9104 URL: https://issues.apache.org/jira/browse/KAFKA-9104 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-9038) Allow creating partitions while partition reassignment is in progress

[ https://issues.apache.org/jira/browse/KAFKA-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9038. Fix Version/s: 2.4.0 Resolution: Fixed > Allow creating partitions while partition

Build failed in Jenkins: kafka-2.4-jdk8 #38

See Changes: [cmccabe] MINOR: Re-implement NewPartitionReassignment#of() (#7592) -- [...truncated 5.10 MB...] kafka.server.KafkaApisTest >

Build failed in Jenkins: kafka-trunk-jdk11 #911

See Changes: [cmccabe] MINOR: Re-implement NewPartitionReassignment#of() (#7592) -- [...truncated 5.49 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Walker, I am not sure if I can follow your argument. What do you exactly mean by > I also >> think that in this case it would be better to separate the 2 option out >> into separate overloads. Maybe you can give an example what method signature you have in mind? >> We could take a named

Re: [VOTE] KIP-541: Create a fetch.max.bytes configuration for the broker

+1 (binding) On Thu, Oct 24, 2019, 4:56 PM Colin McCabe wrote: > Hi all, > > I'd like to start the vote on KIP-541: Create a fetch.max.bytes > configuration for the broker. > > KIP: https://cwiki.apache.org/confluence/x/4g73Bw > > Discussion thread: >

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Just to close the loop @Vinoth: > 1. IIUC John intends to add (or we can do this in this KIP) lag information > to AssignmentInfo, which gets sent to every participant. As explained by John, currently KIP-441 plans to only report the information to the leader. But I guess, with the new proposal

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Just to chime in on the "report lag vs timestamp difference" issue, I would actually advocate for both. As mentioned already, time difference is probably a lot easier and/or more useful to reason about in terms of "freshness" of the state. But in the case when all queried stores are far behind,

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Sending the older replies again as they were unreadable due to bad formatting. >> There was a follow-on idea I POCed to continuously share lag information in >> the heartbeat protocol +1 that would be great, I will update the KIP assuming this work will finish soon >> I think that adding a