[jira] [Created] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

Christo Lolov created KAFKA-16790: - Summary: Calls to RemoteLogManager are made before it is configured Key: KAFKA-16790 URL: https://issues.apache.org/jira/browse/KAFKA-16790 Project: Kafka

[VOTE] KIP-950: Tiered Storage Disablement

Heya! I would like to start a vote on KIP-950: Tiered Storage Disablement in order to catch the last Kafka release targeting Zookeeper - https://cwiki.apache.org/confluence/display/KAFKA/KIP-950%3A++Tiered+Storage+Disablement Best, Christo

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

te retries is a good idea since if a broker is down > forever, this request will never complete. > You mentioned the existing topic deletion is using the similar pattern, how > does it handle this issue? > > Thanks. > Luke > > On Thu, May 9, 2024 at 9:21 PM Christo Lolov > wrote:

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

listening for > changes in > >>>> ZK. > >>>> > >>>> 3. In the KRaft handler steps, you said: > >>>> "The controller also updates the Topic metadata to increment the > >>>> tiered_epoch and update the tiered_stateto

Re: [VOTE] KIP-1018: Introduce max remote fetch timeout config

Heya Kamal, Thanks for the KIP and the answers in the discussion! +1 from me :) Best, Christo On Thu, 9 May 2024 at 11:11, Federico Valeri wrote: > +1 non binding > > Thanks > > On Thu, May 9, 2024 at 12:05 PM Luke Chen wrote: > > > > Hi Kamal, > > > > Thanks for the KIP! > > +1 from me. >

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

Heya! Is it difficult to instead add the metric at kafka.network:type=RequestMetrics,name=TieredStorageMs (or some other name=*)? Alternatively, if it is difficult to add it there, is it possible to add 2 metrics, one at the RequestMetrics level (even if it is total-time-ms - (all other times))

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

Heya Kamal, I quite like the proposal and would support it! However, today I don't think we have a metric which shows the latency of fetch requests which are served from remote, am I wrong? I looked at both

Re: [VOTE] KIP-1023: Follower fetch from tiered offset

Heya Abhijeet, Thanks a lot for pushing this forward, especially with the explanation of EARLIEST_PENDING_UPLOAD_OFFSET_TIMESTAMP! +1 from me :) Best, Christo On Fri, 26 Apr 2024 at 12:50, Luke Chen wrote: > Hi Abhijeet, > > Thanks for the KIP. > +1 from me. > > Thanks. > Luke > > On Fri, Apr

Re: [ANNOUNCE] New committer: Igor Soarez

Congratulations Igor :) ! On Thu, 25 Apr 2024 at 17:07, Igor Soarez wrote: > Thanks everyone, I'm very honoured to join! > > -- > Igor >

Re: [VOTE] KIP-1037: Allow WriteTxnMarkers API with Alter Cluster Permission

Heya Nikhil, Thanks for the proposal, as mentioned before it makes sense to me! +1 (binding) Best, Christo On Sat, 20 Apr 2024 at 00:25, Justine Olshan wrote: > Hey Nikhil, > > I meant to comment on the discussion thread, but my draft took so long, you > opened the vote. > > Regardless, I

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

ould have to use already existing methods Let me know if there are any new comments or I have missed something! Best, Christo On Mon, 15 Apr 2024 at 12:40, Christo Lolov wrote: > Heya Doguscan, > > I believe that the state of the world after this KIP will be the following: > > For Z

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

ableRemoteTopic >> within the StopReplicas API which will then disappear in KRaft world as it >> is already scheduled for deprecation. What do you think? >> >> Sounds good to me. >> >> Thanks. >> Luke >> >> On Tue, Apr 9, 2024 at 6:46 PM Christo Lolov

Re: [DISCUSS] KIP-1037: Allow WriteTxnMarkers API with Alter Cluster Permission

Heya Nikhil, Thank you for raising this KIP! Your proposal makes sense to me. In essence you are saying that the permission required by WriteTxnMarkers should be the same as for CreateAcls and DeleteAcls, which is reasonable. If we trust an administrator to assign the correct permissions then we

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

Congratulations, Greg :) On Mon, 15 Apr 2024 at 07:34, Zhisheng Zhang <31791909...@gmail.com> wrote: > Congratulations Greg! > > > Manikumar 于2024年4月15日周一 13:49写道： > > > Congratulations, Greg. > > > > On Mon, Apr 15, 2024 at 11:18 AM Bruno Cadonna > wrote: > > > > > > Congratulations, Greg! >

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

ng unexpected upload/fetch tasks, right? > I agree we should make sure in ZK mode, all brokers are completed the > stopPartitions before moving to "Disabled" state because ZK node watcher is > working in a separate thread. But not sure about KRaft mode. > > Thanks. > Luke >

[jira] [Created] (KAFKA-16480) ListOffsets change should have an associated API/IBP version update

Christo Lolov created KAFKA-16480: - Summary: ListOffsets change should have an associated API/IBP version update Key: KAFKA-16480 URL: https://issues.apache.org/jira/browse/KAFKA-16480 Project: Kafka

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

; > > Thanks for reviewing the KIP. > > > > The follower needs the earliest-pending-upload-offset (and the > > corresponding leader epoch) from the leader. > > This is the first offset the follower will have locally. > > > > Regards, > > Abhijeet. > &g

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

troller -> Broker RPCs. In fact, KIP-631 deprecated these > > RPCs. > > > > > > > > I will come up with some recommendations on how we could improve that > > > one but until then, @Luke please feel free to review the KIP. > > > > > > > > @Satish, if we want this to

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

Heya! First of all, thank you very much for the proposal, you have explained the problem you want solved very well - I think a faster bootstrap of an empty replica is definitely an improvement! For my understanding, which concrete offset do you want the leader to give back to a follower -

Re: [ANNOUNCE] New committer: Christo Lolov

Thank you everyone! It wouldn't have been possible without quite a lot of reviews and extremely helpful inputs from you and the rest of the community! I am looking forward to working more closely with you going forward :) On Tue, 26 Mar 2024 at 14:31, Kirk True wrote: > Congratulations

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

ana > wrote: > > > Hi Christo, > > Thanks for volunteering to contribute to the KIP discussion. I suggest > > considering this KIP for both ZK and KRaft as it will be helpful for > > this feature to be available in 3.8.0 running with ZK clusters. > > > > Thanks, &g

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Hello! I volunteer to get this KIP moving forward and implemented in Apache Kafka 3.8. I have caught up with Mehari offline and we have agreed that given Apache Kafka 4.0 being around the corner we would like to propose this feature only for KRaft clusters. Any and all reviews and comments are

[jira] [Created] (KAFKA-16154) Make broker changes to return an offset for LATEST_TIERED_TIMESTAMP

Christo Lolov created KAFKA-16154: - Summary: Make broker changes to return an offset for LATEST_TIERED_TIMESTAMP Key: KAFKA-16154 URL: https://issues.apache.org/jira/browse/KAFKA-16154 Project: Kafka

Re: [VOTE] KIP-1005: Expose EarliestLocalOffset and TieredOffset

11 Jan 2024 at 17:52, Divij Vaidya > > > > wrote: > > > > > > > > > > +1 (binding) > > > > > > > > > > Divij Vaidya > > > > > > > > > > > > > > > > > > > > On Tue, Dec 26, 2023

[jira] [Resolved] (KAFKA-15734) KRaft support in BaseConsumerTest

[ https://issues.apache.org/jira/browse/KAFKA-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15734. --- Resolution: Fixed > KRaft support in BaseConsumerT

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

e timestamp greater than recently > committed record timestamp is given.]" > > WDYT? > > Thanks. > Luke > > On Mon, Jan 15, 2024 at 6:46 PM Christo Lolov > wrote: > > > Heya Luke, > > > > Thank for the question! I have expanded in the KIP - in my opin

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

gt; Thank you for making the change Christo. It looks good to me. > > > > -- > > Divij Vaidya > > > > > > > > On Thu, Jan 11, 2024 at 11:19 AM Christo Lolov > > wrote: > > > > > Thank you Divij! > > > > > > I hav

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

; > -- > Divij Vaidya > > > > On Tue, Nov 21, 2023 at 2:19 PM Christo Lolov > wrote: > > > Heya! > > > > Thanks a lot for this. I have updated the KIP to include exposing the > > tiered-offset as well. Let me know whether the Public Interfaces section >

[VOTE] KIP-1005: Expose EarliestLocalOffset and TieredOffset

Heya all! KIP-1005 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-1005%3A+Expose+EarliestLocalOffset+and+TieredOffset) has been open for around a month with no further comments - I would like to start a voting round on it! Best, Christo

Re: [VOTE] KIP-1007: Introduce Remote Storage Not Ready Exception

Heya Kamal, The proposed change makes sense to me as it will be a more explicit behaviour than what Kafka does today - I am happy with it! +1 (non-binding) from me Best, Christo On Tue, 12 Dec 2023 at 09:01, Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi, > > I would like

[jira] [Created] (KAFKA-16002) Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments

Christo Lolov created KAFKA-16002: - Summary: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments Key: KAFKA-16002 URL: https://issues.apache.org/jira/browse/KAFKA-16002

Re: [VOTE] KIP-963: Additional metrics in Tiered Storage

t 05:21, Kamal Chandraprakash > > wrote: > > > > > > +1 (non-binding). Thanks for the KIP! > > > > > > On Tue, Nov 21, 2023, 03:04 Divij Vaidya > > wrote: > > > > > > > + 1 (binding) > > > > > > >

[jira] [Created] (KAFKA-15883) Implement RemoteCopyLagBytes

Christo Lolov created KAFKA-15883: - Summary: Implement RemoteCopyLagBytes Key: KAFKA-15883 URL: https://issues.apache.org/jira/browse/KAFKA-15883 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

> highest-copied-remote-offset via > > GetOffsetShell tool? This will be useful during the debugging session. > > > > > > On Mon, Nov 20, 2023 at 5:38 PM Christo Lolov > > wrote: > > > > > Hello all! > > > > > > I would like to start

[DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

Hello all! I would like to start a discussion for https://cwiki.apache.org/confluence/display/KAFKA/KIP-1005%3A+Add+EarliestLocalOffset+to+GetOffsetShell . A new offset called local log start offset was introduced as part of KIP-405: Kafka Tiered Storage. KIP-1005 aims to expose this offset by

[VOTE] KIP-963: Additional metrics in Tiered Storage

Hello all! Now that the discussion for KIP-963 has winded down, I would like to open it for a vote targeting 3.7.0 as the release. You can find the current version of the KIP at https://cwiki.apache.org/confluence/display/KAFKA/KIP-963%3A+Additional+metrics+in+Tiered+Storage Best, Christo

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

fine grained remote fetch flow sequence > metrics separately later? > > Thanks, > Satish. > > On Tue, 14 Nov 2023 at 22:07, Christo Lolov > wrote: > > > > Heya everyone, > > > > Apologies for the delay in my response and thank you very much for all > your

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

> > > can't > > > > > be relied as a stable unit of measure. So, if we can keep bytes- > and > > > > > segment-based lag, LGTM. > > > > > 3.4. Agree, these metrics should be on the broker side. Though if > > > plugin > >

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

internals) to have some metrics attached -- maybe if you > could shared scenarios where this metrics would be useful would be helpful. > > 7. On the metrics naming: not sure the `Total*` prefix is really needed or > adds meaning. When I found it useful is when there are related metric that &g

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

of metrics and their granularity! Best, Christo On Fri, 13 Oct 2023 at 10:14, Christo Lolov wrote: > Heya Gantigmaa, > > Apologies for the (very) late reply! > > Now that 3.6 has been released and reviewers have a bit more time I will > be picking up this KIP again. I am more

[jira] [Created] (KAFKA-15660) File-based Tiered Storage should delete folders upon topic deletion

Christo Lolov created KAFKA-15660: - Summary: File-based Tiered Storage should delete folders upon topic deletion Key: KAFKA-15660 URL: https://issues.apache.org/jira/browse/KAFKA-15660 Project: Kafka

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

ose metrics > were not mentioned in the original KIP-405 or KIP-930. Do you think these > would make sense to be added to this KIP and get included in the > discussion? > > Regards, > Gantigmaa > > On Wed, Aug 9, 2023 at 1:53 PM Christo Lolov > wrote: > > > H

[jira] [Resolved] (KAFKA-15385) Replace EasyMock with Mockito for AbstractStreamTest

[ https://issues.apache.org/jira/browse/KAFKA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15385. --- Resolution: Duplicate Closing this in favour of https://issues.apache.org/jira/browse/KAFKA

[jira] [Resolved] (KAFKA-15382) Replace EasyMock with Mockito for KStreamTransformValuesTest

[ https://issues.apache.org/jira/browse/KAFKA-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15382. --- Resolution: Duplicate Closing this ticket in favour of https://issues.apache.org/jira

[jira] [Resolved] (KAFKA-15383) Replace EasyMock with Mockito for KTableImplTest

[ https://issues.apache.org/jira/browse/KAFKA-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15383. --- Resolution: Duplicate Closing this ticket in favour of https://issues.apache.org/jira

[jira] [Resolved] (KAFKA-15384) Replace EasyMock with Mockito for KTableTransformValuesTest

[ https://issues.apache.org/jira/browse/KAFKA-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15384. --- Resolution: Fixed Closing this ticket in favour of https://issues.apache.org/jira/browse

Re: [VOTE] 3.6.0 RC0

Heya, I have compiled and ran the test target successfully for the 3.6.0-rc0 branch on: Java 11, Scala 2.13 - ARM Java 17, Scala 2.13 - ARM Java 20, Scala 2.18 - ARM Java 11, Scala 2.13 - Intel x86 Java 17, Scala 2.13 - Intel x86 Java 20, Scala 2.13 - Intel x86 I will update the Zookeeper KIP

[jira] [Resolved] (KAFKA-15399) Enable OffloadAndConsumeFromLeader test

[ https://issues.apache.org/jira/browse/KAFKA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15399. --- Resolution: Fixed > Enable OffloadAndConsumeFromLeader t

[jira] [Created] (KAFKA-15427) Integration tests in TS test harness detect resource leaks

Christo Lolov created KAFKA-15427: - Summary: Integration tests in TS test harness detect resource leaks Key: KAFKA-15427 URL: https://issues.apache.org/jira/browse/KAFKA-15427 Project: Kafka

[jira] [Resolved] (KAFKA-15298) Disable DeleteRecords on Tiered Storage topics

[ https://issues.apache.org/jira/browse/KAFKA-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15298. --- Resolution: Won't Fix > Disable DeleteRecords on Tiered Storage top

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

t; the record size. > > Do you think having the tier-lag by number of segments (or) size of > segments in bytes will be useful > to the operator? > > Thanks, > Kamal > > On Tue, Aug 8, 2023 at 8:56 PM Christo Lolov > wrote: > > > Hello all! > > > > I wo

[DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

Hello all! I would like to start a discussion for KIP-963: Upload and delete lag metrics in Tiered Storage (https://cwiki.apache.org/confluence/x/sZGzDw). The purpose of this KIP is to introduce a couple of metrics to track lag with respect to remote storage from the point of view of Kafka.

[DISCUSS] Cluster-wide disablement of Tiered Storage

Hello all! I wanted to gather more opinions for https://issues.apache.org/jira/browse/KAFKA-15267 In summary, the problem which I would like to solve is disabling TS (and freeing the resources used by RemoteLog*Manager) because I have decided I no longer want to use it without having to

[jira] [Created] (KAFKA-15298) Disable DeleteRecords on Tiered Storage topics

Christo Lolov created KAFKA-15298: - Summary: Disable DeleteRecords on Tiered Storage topics Key: KAFKA-15298 URL: https://issues.apache.org/jira/browse/KAFKA-15298 Project: Kafka Issue Type

[jira] [Created] (KAFKA-15267) Cluster-wide disablement of Tiered Storage

Christo Lolov created KAFKA-15267: - Summary: Cluster-wide disablement of Tiered Storage Key: KAFKA-15267 URL: https://issues.apache.org/jira/browse/KAFKA-15267 Project: Kafka Issue Type: Sub

Re: Apache Kafka 3.6.0 release

Hello! A couple of days ago I opened a new KIP for discussion - KIP-952 [1]. I believe it might be a blocker for the release of 3.6.0, but I wanted to bring it up here for a decision on its urgency with the current set of people who are looking at Tiered Storage (Satish, Luke, Ivan, Divij) given

[DISCUSS] KIP-952: Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage

Hello! A customer upgrading from Kafka < 2.8 to the future release 3.6 and wanting to enable tiered storage would have to take responsibility for ensuring that all segments lacking a producer snapshot file have expired and are deleted before enabling the feature. In our experience customers are

[jira] [Created] (KAFKA-15195) Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage

Christo Lolov created KAFKA-15195: - Summary: Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage Key: KAFKA-15195 URL: https://issues.apache.org/jira/browse

Re: Apache Kafka 3.6.0 release

Hello! I will add KIP-902 to the release plan. I would appreciate a few more reviews on the pull request (https://github.com/apache/kafka/pull/13260) for that KIP as the longer we have it in trunk with tests running against it the more confidence we will have before the release. Best, Christo

Re: [VOTE] KIP-937: Improve Message Timestamp Validation

+1 (non-binding) from me as well! This is the type of problem which is difficult to become aware of so the more guardrails we put into place the better. On Wed, 21 Jun 2023 at 23:30, Beyene, Mehari wrote: > Thank you, Justin. That makes sense. > I have updated the KIP to remove the concept of

Re: [DISCUSS] KIP-928: Making Kafka resilient to log directories becoming full

, Christo Lolov wrote: > Hey Colin, > > Thanks for the review! > > I am also skeptical that much space can be reclaimed via compaction as > detailed in the limitations section of the KIP. > > In my head there are two ways to get out of the saturated state - > configur

Re: [DISCUSS] KIP-928: Making Kafka resilient to log directories becoming full

the broker that had the disk full, it will delete the > topic directory on startup as part of its snapshot load process. > > So I'm not sure if we really need this. Maybe we should re-evaluate once > we have JBOD + KRaft. > > best, > Colin > > > On Mon, May 22, 2023, at

Re: [DISCUSS] KIP-928: Making Kafka resilient to log directories becoming full

Heya Igor, Thank you for reading through the KIP and providing feedback! 11. Good question. I will check whether a change is needed in the processing of the metadata records and come back. My hunch says no as long as the Kafka broker is still alive to process the metadata records. This being

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

Heya! 5th of June 16:30 - 17:00 UTC works for me. Best, Christo On Thu, 25 May 2023 at 15:14, Igor Soarez wrote: > Hi Divij, Christo, > > Thank you for pointing that out. > > Let's aim instead for Monday 5th of June, at the same time – 16:30-17:00 > UTC. > > Please let me know if this doesn't

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

Heya Igor! I don't have any concerns or suggestions for improvements at this stage - the overall approach makes sense to me! I would be quite interested in attending a call, but as Divij has pointed out the 29th of May is a public holiday, so I won't be able to make that date. If there is

Re: [VOTE] 3.5.0 RC0

Hey Mickael! I am giving a +1 (non-binding) for this candidate release. * Built from the binary tar.gz source with Java 17 and Scala 2.13 on Intel (m5.4xlarge) and ARM (m6g.4xlarge) machines. * Ran unit and integration tests on Intel and ARM machines. * Ran the Quickstart in both Zookeeper and

Re: [DISCUSS] KIP-935: Extend AlterConfigPolicy with existing configurations

Hello! This proposal will address problems with configuration dependencies which I run into very frequently, so I am fully supporting the development of this feature! Best, Christo On Mon, 22 May 2023 at 17:18, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Hi everyone, > >

Re: [DISCUSS] KIP-934: Add DeleteTopicPolicy

Heya Jorge, Thank you for the KIP! This feature sounds great to me since I have encountered problems with this, so I am supporting it. Do you have any idea why the previous KIPs were abandoned - I went through the email conversations and pull requests, but I didn't find a good reason? Best,

RE: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

Hello Igor! I have been working on a KIP to extend the functionality of JBOD broker disk failures (https://cwiki.apache.org/confluence/display/KAFKA/KIP-928%3A+Making+Kafka+resilient+to+log+directories+becoming+full) and was wondering what is the state of this KIP - were you planning on

Re: [DISCUSS] KIP-905: Broker interceptors

Hello David, Thank you for the proposal - it is an interesting read! I have a few questions about it. 1. Can you take a stance on whether you are proposing the feature just for producers or for consumers as well? If it is just for producers can we remove references to consumers? If it is for

[DISCUSS] KIP-928: Making Kafka resilient to log directories becoming full

Hello all! I would like to start a discussion on KIP-928: Making Kafka resilient to log directories becoming full which can be found at https://cwiki.apache.org/confluence/display/KAFKA/KIP-928%3A+Making+Kafka+resilient+to+log+directories+becoming+full . In summary, I frequently run into

Re: [DISCUSS] KIP-895: Dynamically refresh partition count of __consumer_offsets

Hello all and thank you for the insightful comments! As pointed out in the third rejected alternative I think the ideal solution is what a few of you have also mentioned - if partitions need to be expanded consumer groups should not lose their offsets and be able to find them. I was aware that

Re: [VOTE] KIP-902: Upgrade Zookeeper to 3.8.1

e with you that we're going to have to move to the new > version of ZK. This fits in with the overall timeline of one more year of > Kafka releases supporting ZK. If Apache Kafka 4.0 is April 2024, we'll need > to be getting security updates for ZK during this time. > > > > O

Re: [VOTE] KIP-902: Upgrade Zookeeper to 3.8.1

ern better than I could. Best, Christo On Mon, 10 Apr 2023 at 00:53, Colin McCabe wrote: > On Wed, Mar 15, 2023, at 04:58, Christo Lolov wrote: > > Hello Colin, > > > > Thank you for taking the time to review the proposal! > > > > I have attached a compat

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hello! What would you suggest as the best way to get more eyes on KIP-902 as I would like it to be included it in 3.5.0? Best, Christo > On 16 Mar 2023, at 10:33, Mickael Maison wrote: > > Hi, > > This is a reminder that KIP freeze is less than a week away (22 Mar). > For a KIP to be

Re: [VOTE] KIP-902: Upgrade Zookeeper to 3.8.1

sion. > > > > Vote +1 (non binding) > > > > -- > > Divij Vaidya > > > > > > > > On Thu, Feb 23, 2023 at 3:32 PM Christo Lolov > > wrote: > > > >> Hello! > >> > >> I would like to start the vote for KIP-90

Re: [DISCUSS] KIP-902: Upgrade Zookeeper to 3.8.1

nd of life and > are not maintained. (source: Time Based Release Plan#WhatIsOurEOLPolicy)." > > Ismael > >> On Wed, Feb 15, 2023 at 1:47 AM Christo Lolov >> wrote: >> >> Hello! >> >> I would like to start a discussion for KIP-902: Upgrade Zooke

Re: [DISCUSS] KIP-902: Upgrade Zookeeper to 3.8.1

thanks for the great analysis for the ZK upgrading. > > One question: > Since we are going to remove ZK in v4.0, and we don't need the latest > feature in the "current release" ZK 3.8.1, why can't we choose the "stable > release" (3.7.1)? > > Thank you. > L

[VOTE] KIP-902: Upgrade Zookeeper to 3.8.1

Hello! I would like to start the vote for KIP-902, which upgrades Zookeeper to version 3.8.1. The KIP can be found at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784 The discussion thread is https://lists.apache.org/thread/5jbn2x0rtmqz5scyoygbdbj4vo0mpbw1 Thanks

[jira] [Resolved] (KAFKA-13690) Flaky test EosIntegrationTest.shouldWriteLatestOffsetsToCheckpointOnShutdown[at_least_once]

[ https://issues.apache.org/jira/browse/KAFKA-13690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-13690. --- Resolution: Fixed > Flaky t

[jira] [Resolved] (KAFKA-13966) Flaky test `QuorumControllerTest.testUnregisterBroker`

[ https://issues.apache.org/jira/browse/KAFKA-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-13966. --- Resolution: Fixed > Flaky test `QuorumControllerTest.testUnregisterBro

[DISCUSS] KIP-902: Upgrade Zookeeper to 3.8.1

Hello! I would like to start a discussion for KIP-902: Upgrade Zookeeper to 3.8.1. The Zookeeper version currently used in Kafka reached its end of life in December 2022. You can find the KIP at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784 Thanks in advance for

Re: [DISCUSS] KIP-899: Allow clients to rebootstrap

Hello! Thank you for the KIP. I would like to summarise my understanding of the problem in case I am wrong. Currently a long-running client refreshes their metadata from a set of brokers obtained when first contacting the cluster. If they have been “away” for too long those brokers might have

RE: Last sprint to finish line: Replace EasyMock/Powermock with Mockito

Hello! Below you will find the latest state of the Mockito migration. 81% or 39/48 of streams-related tests have been migrated. The last 5 pull requests which are in need of reviews are: * https://github.com/apache/kafka/pull/12449 * https://github.com/apache/kafka/pull/12524 *

Re: [DISCUSS] KIP-895: Dynamically refresh partition count of __consumer_offsets

Greetings, I am bumping the below DISCUSSion thread for KIP-895. The KIP presents a situation where consumer groups are in an undefined state until a rolling restart of a cluster is performed. While I have demonstrated the behaviour using a cluster using Zookeeper I believe the same problem can

[jira] [Resolved] (KAFKA-14199) Installed kafka in ubuntu and not able to access in browser. org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1195725856 larger than 10

[ https://issues.apache.org/jira/browse/KAFKA-14199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-14199. --- Resolution: Fixed > Installed kafka in ubuntu and not able to access in brow

[jira] [Resolved] (KAFKA-13805) Upgrade vulnerable dependencies march 2022

[ https://issues.apache.org/jira/browse/KAFKA-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-13805. --- Resolution: Fixed > Upgrade vulnerable dependencies march 2

[jira] [Resolved] (KAFKA-14324) [CVE-2018-25032] introduced by rocksdbjni:6.29.4.1

[ https://issues.apache.org/jira/browse/KAFKA-14324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-14324. --- Resolution: Fixed > [CVE-2018-25032] introduced by rocksdbjni:6.29.

RE: Last sprint to finish line: Replace EasyMock/Powermock with Mockito

Hello! This email summarises the current state of Kafka's Mockito migration. The JIRA tickets used to track the progress are https://issues.apache.org/jira/browse/KAFKA-14132 and https://issues.apache.org/jira/browse/KAFKA-14133

Re: Last sprint to finish line: Replace EasyMock/Powermock with Mockito

Hello, I have not been able to make a lot of progress on the Mockito migration myself, but Yash and Divij opened and merged a PR each. The following PRs have made it into trunk: https://github.com/apache/kafka/pull/12615 https://github.com/apache/kafka/pull/12677

RE: Last sprint to finish line: Replace EasyMock/Powermock with Mockito

Hello! This is the (roughly) bi-weekly update on the Mockito migration. Firstly, the following PRs have been merged since the last email so thank you to the writers (Yash and Divij) and reviewers (Dalibor, Mickael, Yash, Bruno and Chris): https://github.com/apache/kafka/pull/12459

[jira] [Created] (KAFKA-14133) Remaining EasyMock to Mockito tests

Christo Lolov created KAFKA-14133: - Summary: Remaining EasyMock to Mockito tests Key: KAFKA-14133 URL: https://issues.apache.org/jira/browse/KAFKA-14133 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-14132) Remaining PowerMock to Mockito tests

Christo Lolov created KAFKA-14132: - Summary: Remaining PowerMock to Mockito tests Key: KAFKA-14132 URL: https://issues.apache.org/jira/browse/KAFKA-14132 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-14109) Clean up JUnit 4 test infrastructure

Christo Lolov created KAFKA-14109: - Summary: Clean up JUnit 4 test infrastructure Key: KAFKA-14109 URL: https://issues.apache.org/jira/browse/KAFKA-14109 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-14108) Allow JUnit 4 and JUnit 5 tests to run while the migration is going on

Christo Lolov created KAFKA-14108: - Summary: Allow JUnit 4 and JUnit 5 tests to run while the migration is going on Key: KAFKA-14108 URL: https://issues.apache.org/jira/browse/KAFKA-14108 Project

[jira] [Resolved] (KAFKA-14001) Part 1

[ https://issues.apache.org/jira/browse/KAFKA-14001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-14001. --- Reviewer: Bruno Cadonna Resolution: Fixed > Part 1 > -- > >

[jira] [Resolved] (KAFKA-13821) Update Kafka Streams WordCount demo to new Processor API

[ https://issues.apache.org/jira/browse/KAFKA-13821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-13821. --- Resolution: Fixed > Update Kafka Streams WordCount demo to new Processor

[jira] [Resolved] (KAFKA-13897) Add 3.1.1 to system tests and streams upgrade tests

[ https://issues.apache.org/jira/browse/KAFKA-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-13897. --- Resolution: Won't Fix > Add 3.1.1 to system tests and streams upgrade te

[jira] [Created] (KAFKA-14004) Part 3

Christo Lolov created KAFKA-14004: - Summary: Part 3 Key: KAFKA-14004 URL: https://issues.apache.org/jira/browse/KAFKA-14004 Project: Kafka Issue Type: Sub-task Reporter: Christo

[jira] [Created] (KAFKA-14003) Part 2

Christo Lolov created KAFKA-14003: - Summary: Part 2 Key: KAFKA-14003 URL: https://issues.apache.org/jira/browse/KAFKA-14003 Project: Kafka Issue Type: Sub-task Reporter: Christo

  1   2   >