Re: [VOTE] 3.7.0 RC4

2024-03-01 Thread Stanislav Kozlovski
ed the maven central repository. > > Ismael > > On Tue, Feb 27, 2024 at 9:49 AM Stanislav Kozlovski > wrote: > > > The thinking is that it is available for use - and it is in maven > central - > > https://central.sonatype.com/artifact/org.apache.kafka/kaf

[ANNOUNCE] Apache Kafka 3.7.0

2024-02-27 Thread Stanislav Kozlovski
Provenzano, Purshotam Chauhan, Qichao Chu, Matthias J. Sax, Rajini Sivaram, Renaldo Baur Filho, Ritika Reddy, Robert Wagner, Rohan, Ron Dagostino, Roon, runom, Ruslan Krivoshein, rykovsi, Sagar Rao, Said Boudjelda, Satish Duggana, shuoer86, Stanislav Kozlovski, Taher Ghaleb, Tang Yunzi, TapDang, Taras

Re: [VOTE] 3.7.0 RC4

2024-02-27 Thread Stanislav Kozlovski
> See "Wait for about a day for the artifacts to show up in apache mirror > (releases, public group) and maven central (mvnrepository.com or maven.org > )." > in the release process wiki. > > -- > Divij Vaidya > > > > On Tue, Feb 27, 2024 at 4:43 PM Stanisla

Re: [VOTE] 3.7.0 RC4

2024-02-27 Thread Stanislav Kozlovski
/org.apache.kafka/kafka Am I free to announce the release, or should I wait more for MVNRepository to get up to date? For what it's worth, I "Released" the files 24 hours ago On Mon, Feb 26, 2024 at 10:42 AM Stanislav Kozlovski wrote: > > This vote passes with *10 +1 votes* (3 bindings

Re: [VOTE] 3.7.0 RC4

2024-02-26 Thread Stanislav Kozlovski
ASAP. On Mon, Feb 26, 2024 at 10:42 AM Stanislav Kozlovski wrote: > > This vote passes with *10 +1 votes* (3 bindings) and no 0 or -1 votes. > > +1 votes > > PMC Members (binding): > * Mickael Maison > * Justine Olshan > * Divij Vaidya > > Community (non-binding

Re: [VOTE] 3.7.0 RC4

2024-02-26 Thread Stanislav Kozlovski
: > Hi, > > Thanks for sorting out the docs issues. > +1 (binding) > > Mickael > > On Fri, Feb 23, 2024 at 11:50 AM Stanislav Kozlovski > wrote: > > > > Some quick updates: > > > > There were some inconsistencies between the documentation in

Re: [VOTE] 3.7.0 RC4

2024-02-23 Thread Stanislav Kozlovski
eps from here: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Process#ReleaseProcess-Afterthevotepasses > > > > > > > > 2. Upload artifacts to maven etc. These artifacts do not have RC > suffix &

Re: [VOTE] 3.7.0 RC4

2024-02-20 Thread Stanislav Kozlovski
in license file > > > > scala-reflect-2.12.18 is missing in license file > > > > ``` > > > > > > > > ## Long running tests for memory leak (on ARM machine with zstd) > > > > > > > > Test: Run

Re: [VOTE] 3.7.0 RC4

2024-02-18 Thread Stanislav Kozlovski
just have some flakes On Sun, Feb 18, 2024 at 10:45 AM Stanislav Kozlovski wrote: > The upgrade test passed -> > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708103771--apache--3.7--bb6990114b/2024-02-16--001./2

Re: [VOTE] 3.7.0 RC4

2024-02-18 Thread Stanislav Kozlovski
tps://jenkins.confluent.io/job/system-test-kafka-branch-builder/6062/ On Fri, Feb 16, 2024 at 6:39 PM Stanislav Kozlovski wrote: > Thanks all for the help in verifying. > > I have updated > https://gist.github.com/stanislavkozlovski/820976fc7bfb5f4dcdf9742fd96a9982 > with the system tests. > Ther

Re: [VOTE] 3.7.0 RC4

2024-02-16 Thread Stanislav Kozlovski
15 Feb 2024, at 05:07, Paolo Patierno > > wrote: > > > > > > > > +1 (non-binding). I used the staged binaries with Scala 2.13 and > mostly > > > > focused on the ZooKeeper to KRaft migration with multiple tests. > > Everything > > &

Re: [VOTE] 3.7.0 RC4

2024-02-14 Thread Stanislav Kozlovski
he > staged > > Maven artifacts to run my tests. All seems to work fine. Thanks. > > > > Jakub > > > > On Fri, Feb 9, 2024 at 4:20 PM Stanislav Kozlovski > > wrote: > > > > > Hello Kafka users, developers and client-developers, > > > &

Re: Apache Kafka 3.7.0 Release

2024-02-12 Thread Stanislav Kozlovski
> "Kafka > > > cluster will be unavailable during KRaft migration rollback". > > > The impact for this issue is that if brokers try to rollback to ZK mode > > > during KRaft migration process, there will be a period of time the > > cluster > > &g

[VOTE] 3.7.0 RC4

2024-02-09 Thread Stanislav Kozlovski
Hello Kafka users, developers and client-developers, This is the second candidate we are considering for release of Apache Kafka 3.7.0. Major changes include: - Early Access to KIP-848 - the next generation of the consumer rebalance protocol - Early Access to KIP-858: Adding JBOD support to

Re: [VOTE] 3.7.0 RC2

2024-02-05 Thread Stanislav Kozlovski
t; KRaft early access in 3.7.0 is probably the right call. Even without > the bugs we found, there's still quite a few JBOD follow up work to do > (KAFKA-16061) + system tests and documentation updates. > > Thanks, > Mickael > > On Fri, Feb 2, 2024 at 4:49 PM Stanislav Kozlovski >

Re: [VOTE] 3.7.0 RC2

2024-02-02 Thread Stanislav Kozlovski
tanislav, > >>> > >>> Thanks for bringing these JIRAs/PRs up. > >>> > >>> I'll be testing the open PRs for KAFKA-14616 and KAFKA-16162 this week > and I hope to have some feedback > >>> by Friday. I gather the latter JIRA is marked as a WIP b

Re: [VOTE] 3.7.0 RC2

2024-01-23 Thread Stanislav Kozlovski
observe any obvious impact. I was able to send and > receive > > >>>>> messages as normally. But to be honest, I have no idea what else > > >>>>> this might impact, so I did not try anything special. > > >>>>> > > >>&g

Re: [VOTE] 3.7.0 RC2

2024-01-16 Thread Stanislav Kozlovski
Hi Kirk, Given we are going to have to roll a new RC anyway, and the change is so simple - might as well get it in! On Mon, Jan 15, 2024 at 8:26 PM Kirk True wrote: > Hi Stanislav, > > On Sun, Jan 14, 2024, at 1:17 PM, Stanislav Kozlovski wrote: > > Hey Kirk and Chris, >

Re: [VOTE] 3.7.0 RC2

2024-01-15 Thread Stanislav Kozlovski
to others > > to > > > decide if this is a blocker or not. > > > > > > > Hi Jakub, > > > > Thanks for trying the RC. I think what you found is a blocker bug because > > it will generate huge amount of logspam. I guess we didn't find it in >

[jira] [Created] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16141: --- Summary: StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7 Key: KAFKA-16141

[jira] [Created] (KAFKA-16140) zookeeper_migration_test#TestMigration#test_recooncile_kraft_to_zk system test fails concistently on 3.7

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16140: --- Summary: zookeeper_migration_test#TestMigration#test_recooncile_kraft_to_zk system test fails concistently on 3.7 Key: KAFKA-16140 URL: https://issues.apache.org

[jira] [Created] (KAFKA-16139) StreamsUpgradeTest fails consistently in 3.7.0

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16139: --- Summary: StreamsUpgradeTest fails consistently in 3.7.0 Key: KAFKA-16139 URL: https://issues.apache.org/jira/browse/KAFKA-16139 Project: Kafka

[jira] [Created] (KAFKA-16138) QuotaTest system test fails consistently in 3.7

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16138: --- Summary: QuotaTest system test fails consistently in 3.7 Key: KAFKA-16138 URL: https://issues.apache.org/jira/browse/KAFKA-16138 Project: Kafka

[jira] [Created] (KAFKA-16136) CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated() is very flaky

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16136: --- Summary: CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated() is very flaky Key: KAFKA-16136 URL: https://issues.apache.org/jira/browse/KAFKA-16136

[jira] [Created] (KAFKA-16135) kafka.api.PlaintextConsumerTest.testPerPartitionLeadMetricsCleanUpWithSubscribe(String, String).quorum=kraft+kip848.groupProtocol=consumer is flaky

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16135: --- Summary: kafka.api.PlaintextConsumerTest.testPerPartitionLeadMetricsCleanUpWithSubscribe(String, String).quorum=kraft+kip848.groupProtocol=consumer is flaky Key: KAFKA-16135

[jira] [Created] (KAFKA-16134) kafka.api.PlaintextConsumerTest.testPerPartitionLagMetricsCleanUpWithSubscribe(String, String).quorum=kraft+kip848.groupProtocol=consumer is flaky

2024-01-15 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-16134: --- Summary: kafka.api.PlaintextConsumerTest.testPerPartitionLagMetricsCleanUpWithSubscribe(String, String).quorum=kraft+kip848.groupProtocol=consumer is flaky Key: KAFKA-16134

Re: [VOTE] 3.7.0 RC2

2024-01-14 Thread Stanislav Kozlovski
eported in > > > > > https://issues.apache.org/jira/browse/KAFKA-16029. I think we > should > > > > > prioritize fixing it for this release. I know it's probably benign > > but > > > > it's > > > > > really not a good look for u

[VOTE] 3.7.0 RC2

2024-01-11 Thread Stanislav Kozlovski
c226?context=explore /** Thanks, Stanislav Kozlovski kafka/test:test (alpine 3.18.5) === Total: 0 (HIGH: 0, CRITICAL: 0)

Re: Apache Kafka 3.7.0 Release

2024-01-10 Thread Stanislav Kozlovski
gt;>>> >> > clients. It is only used by the server. > >>>> >> > > >>>> >> > best, > >>>> >> > Colin > >>>> >> > > >>>> >> > > >>>> >> &

Re: Apache Kafka 3.7.0 Release

2024-01-08 Thread Stanislav Kozlovski
is not used by clients. So I > > > don't see why you would want to publish it (unless I'm misunderstanding > > > something). > > > > > > best, > > > Colin > > > > > > > > > On Fri, Jan 5, 2024, at 10:05, Stanislav Kozlovski wrote: >

Re: Apache Kafka 3.7.0 Release

2024-01-05 Thread Stanislav Kozlovski
Thanks for reporting the blockers, folks. Good job finding. I have one ask - can anybody with Gradle expertise help review this small PR? https://github.com/apache/kafka/pull/15127 (+1, -1) In particular, we are wondering whether we need to publish module metadata as part of the gradle publishing

Re: Apache Kafka 3.7.0 Release

2024-01-04 Thread Stanislav Kozlovski
; On Thu, Jan 4, 2024 at 12:49 PM Matthias J. Sax wrote: > > > We found a blocker for 3.7: > > https://issues.apache.org/jira/browse/KAFKA-16077 > > > > Already having a PR under review to fix it. > > > > > > -Matthias > > > > On 1/3

[jira] [Resolved] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2024-01-03 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-16046. - Resolution: Fixed > Stream Stream Joins fail after restorat

Re: Apache Kafka 3.7.0 Release

2024-01-03 Thread Stanislav Kozlovski
thub.com/apache/kafka/pull/15073 ( > https://issues.apache.org/jira/browse/KAFKA-16046) was identified as a > blocker regression and should be merged to trunk by EOD. > > Cheers, > Almog > > On Tue, Jan 2, 2024 at 4:20 AM Stanislav Kozlovski > wrote: > > > Hi Apoorv,

Re: Apache Kafka 3.7.0 Release

2024-01-02 Thread Stanislav Kozlovski
>> >> > early January? >> >> > The reason I'm asking is that there are still a couple of testing >> gaps in >> >> > JBOD (https://issues.apache.org/jira/browse/KAFKA-14127) which I >> think is >> >> > very important to fi

Re: Apache Kafka 3.7.0 Release

2023-12-28 Thread Stanislav Kozlovski
sting and validation. In my opinion it would strengthen the > release if > > > we could spend a week in January to really finish off JBOD and do a 2 > week > > > stabilization. > > > > > > What do you all think? > > > > > > Best, > &

[jira] [Resolved] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-12679. - Resolution: Fixed Marking this as done as per Lucas' comment

[jira] [Resolved] (KAFKA-15147) Measure pending and outstanding Remote Segment operations

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-15147. - Resolution: Fixed > Measure pending and outstanding Remote Segment operati

[jira] [Resolved] (KAFKA-15327) Client consumer should commit offsets on close

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-15327. - Resolution: Fixed Resolving since this was merged   > Client consumer sho

[jira] [Resolved] (KAFKA-15780) Wait for consistent kraft metadata when creating topics in tests

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-15780. - Resolution: Fixed Resolving since this was merged. Nice work!   > W

[jira] [Resolved] (KAFKA-15817) Avoid reconnecting to the same IP address if multiple addresses are available

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-15817. - Resolution: Fixed Resolving since this was merged. good job!   > Av

[jira] [Resolved] (KAFKA-16007) ZK migrations can be slow for large clusters

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-16007. - Resolution: Fixed Closing since it's merged   > ZK migrations can be s

Re: Kafka trunk test & build stability

2023-12-26 Thread Stanislav Kozlovski
> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81) > > >> &g

[jira] [Resolved] (KAFKA-15818) Implement max poll interval

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-15818. - Resolution: Fixed > Implement max poll inter

[jira] [Resolved] (KAFKA-16026) AsyncConsumer does not send a poll event to the background thread

2023-12-26 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-16026. - Resolution: Fixed https://github.com/apache/kafka/pull/15035

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

2023-12-26 Thread Stanislav Kozlovski
Hey all, As the release manager for 3.7.0, I am pretty interested to know if we should consider this a blocker. Do we have clarity as to whether users could practically rely on this Go script? From a shallow look, it's only used in one line in the Dockerfile. I guess the downside is that images

Re: Kafka trunk test & build stability

2023-12-19 Thread Stanislav Kozlovski
l/15008 > > Thanks for looking into this! > Greg > > On Tue, Dec 19, 2023 at 3:45 PM Николай Ижиков > wrote: > > > > Hello, Stanislav. > > > > Can you, please, make the gist public. > > Private gists not available for some GitHub users even if link are

Kafka trunk test & build stability

2023-12-19 Thread Stanislav Kozlovski
Hey everybody, I've heard various complaints that build times in trunk are taking too long, some taking as much as 8 hours (the timeout) - and this is slowing us down from being able to meet the code freeze deadline for 3.7. I took it upon myself to gather up some data in Gradle Enterprise to see

Re: New Release Branch 3.7

2023-12-14 Thread Stanislav Kozlovski
pending <https://github.com/apache/kafka/pull/14984>. With that, I remind you that there are only 6 days to code freeze! It's not long until we will have our very first Apache Kafka 3.7 RC! Let's get this shipped. Best, Stanislav On Tue, Dec 12, 2023 at 2:56 PM Stanislav Kozlovski wrote: &g

Re: Apache Kafka 3.7.0 Release

2023-12-12 Thread Stanislav Kozlovski
Hey! Just notifying everybody on this thread that I have cut the 3.7 branch and sent a new email thread titled "New Release Branch 3.7" to the mailing list <https://lists.apache.org/thread/4j87m12fm3bgq01fgphtkfb41s56w6hh>. Best, Stanislav On Wed, Dec 6, 2023 at 11:10 AM Sta

New Release Branch 3.7

2023-12-12 Thread Stanislav Kozlovski
Hello Kafka developers and friends, As promised, we now have a release branch for 3.7 release. Trunk is being bumped to 3.8.0-SNAPSHOT (please help review the PR ). I'll be going over the JIRAs to move every non-blocker from this release to the next

Re: Apache Kafka 3.7.0 Release

2023-12-06 Thread Stanislav Kozlovski
-mail thread titled "New release branch 3.7.0" to notify you, so be on the lookout for that. I will also notify this thread. Thank you for your contributions. Let's get this release shipped! Best, Stanislav On Fri, Nov 24, 2023 at 6:11 PM Stanislav Kozlovski wrote: > Hey all, >

Re: Apache Kafka 3.7.0 Release

2023-11-24 Thread Stanislav Kozlovski
working on rebasing the implementation against trunk > before I open a PR. > > Regards, > Nick > > On Tue, 21 Nov 2023 at 11:27, Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Hi Stan > > > > Can you include KIP-951 to the 3.7 release plan

Re: Apache Kafka 3.7.0 Release

2023-11-15 Thread Stanislav Kozlovski
smael Juma wrote: > > > > > Thanks Stanislav, +1 > > > > > > Ismael > > > > > > On Thu, Nov 2, 2023 at 7:01 AM Stanislav Kozlovski > > > wrote: > > > > > > > Hi all, > > > > > > > > Given the dis

Re: Apache Kafka 3.7.0 Release

2023-11-02 Thread Stanislav Kozlovski
ten 1-2 day slips and it's better if that doesn't require people > > working through the weekend. > > > > (From this mailing list thread > <https://lists.apache.org/thread/dv1rym2jkf0141sfsbkws8ckkzw7st5h>) > > Thanks for driving the release! > Sophie > > On Wed,

Re: [VOTE] KIP-975: Docker Image for Apache Kafka

2023-10-27 Thread Stanislav Kozlovski
Thanks for the KIP! Great idea, well thought out and much needed. +1 (binding) On Fri, 27 Oct 2023 at 06:36, Krishna Agarwal wrote: > Hi, > I'd like to call a vote on KIP-975 which aims to publish an official docker > image for Apache Kafka. > > KIP - > >

Re: Apache Kafka 3.7.0 Release

2023-10-25 Thread Stanislav Kozlovski
rs/delays and time away > during the winter holidays > > Thoughts? > > On Mon, Oct 23, 2023 at 3:14 PM Sophie Blee-Goldman > > wrote: > > > Can you add the 3.7 plan to the release schedule page? > > > > (this --> > > https://cwiki.apache.org/confluen

Re: Apache Kafka 3.7.0 Release

2023-10-15 Thread Stanislav Kozlovski
ter-wide+dynamic+log+adjustment+for+Kafka+Connect > ) > to the release plan? The vote thread for it passed last week and I've > published a complete PR (https://github.com/apache/kafka/pull/14538), so > it > shouldn't be too difficult to get things merged in time for 3.7.0. > > Cheers, >

Re: Apache Kafka 3.7.0 Release

2023-10-14 Thread Stanislav Kozlovski
> w: aiven.io > >> > >> e: josep.p...@aiven.io > >> > >> On Mon, Oct 9, 2023, 20:05 Chris Egerton > wrote: > >> > >>> +1, thanks Stanislav! > >>> > >>> On Mon, Oct 9, 2023, 14:02 Bill Bejeck wrote: > >>

[jira] [Resolved] (KAFKA-14175) KRaft Upgrades Part 2

2023-10-14 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-14175. - Resolution: Won't Fix > KRaft Upgrades Par

Apache Kafka 3.7.0 Release

2023-10-09 Thread Stanislav Kozlovski
Hey all! I would like to volunteer to be the release manager driving the next release - Apache Kafka *3.7.0*. If there are no objections, I will start and share a release plan soon enough! Cheers, Stanislav

Re: [DISCUSS] KIP-932: Queues for Kafka

2023-05-22 Thread Stanislav Kozlovski
Hey Andrew! Kudos on the proposal. It is greatly written - a joy to read. It is definitely an interesting solution to the queueing problem - I would not have guessed we could solve it like this. Thank you for working on this. Happy to get the discussion started - I have a few comments/questions

Re: [VOTE] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-09-09 Thread Stanislav Kozlovski
nks, Stan! > > > > On Tue, Aug 23, 2022 at 12:10 PM Luke Chen wrote: > >> > >> Hi Stanislav, > >> > >> Thanks for the KIP. > >> The solution looks reasonable to me. > >> +1 from me. > >> > >> Thank you. > >

Re: [DISCISS] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-09-09 Thread Stanislav Kozlovski
is an important > > use > > > case to address? For example, do we have concrete examples of people > > > running into this? The way the KIP is written, it sounds like a > potential > > > problem but no information is given on whether it's a real problem in

[VOTE] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-08-22 Thread Stanislav Kozlovski
Hello, I'd like to start a vote on KIP-860, which adds a client-provided option to the AlterPartitionReassignmentsRequest that allows the user to guard against an unintentional change in the replication factor during partition reassignments. Discuss Thread:

Re: [DISCISS] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-08-22 Thread Stanislav Kozlovski
plication-factor-change would be a better > name. I don't feel strong about this so I am happy to go with the > quorum here. > > Best, > David > > On Tue, Aug 16, 2022 at 12:31 AM Stanislav Kozlovski > wrote: > > > > Thanks for the discussion all, >

Re: [DISCISS] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-08-15 Thread Stanislav Kozlovski
his nuance and thus requested > to > > put that in KIP as it's not mentioned why the choice was made. I am fine > if > > you choose to not do it in the interest of brevity. > > > > Vikas > > > > On Sun, Aug 7, 2022 at 9:02 AM Stanislav Kozlovski > > wrote: > &

Re: [DISCISS] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-08-07 Thread Stanislav Kozlovski
nt and throws if those > > differ. > > Can you add more detail to this, or clarify what is targetReplicaSet (for > > e.g. why not sourceReplicaSet?) and how the target replica set will be > > calculated? > > > > And what about the reassign partitions CLI? Do we want to

[DISCISS] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

2022-07-28 Thread Stanislav Kozlovski
Hey all, I'd like to start a discussion on a proposal to help API users from inadvertently increasing the replication factor of a topic through the alter partition reassignments API. The KIP describes two fairly easy-to-hit race conditions in which this can happen. The KIP itself is pretty

[jira] [Created] (KAFKA-14121) AlterPartitionReassignments API should allow callers to specify the option of preserving the replication factor

2022-07-28 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-14121: --- Summary: AlterPartitionReassignments API should allow callers to specify the option of preserving the replication factor Key: KAFKA-14121 URL: https

[jira] [Resolved] (KAFKA-8406) kafka-topics throws wrong error on invalid configuration with bootstrap-server and alter config

2021-09-08 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-8406. Fix Version/s: 2.4.0 Resolution: Fixed > kafka-topics throws wrong er

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

2021-04-14 Thread Stanislav Kozlovski
; SerDe issues should probably go into a dead-letter queue. > > Anyway, IMO we need exceptions that will let us tell the difference. > > Gwen > > On Fri, Oct 11, 2019 at 10:05 AM Stanislav Kozlovski > wrote: > > > > Thanks Jason. I've edited the KIP with the lat

[jira] [Created] (KAFKA-12555) Log reason for rolling a segment

2021-03-25 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-12555: --- Summary: Log reason for rolling a segment Key: KAFKA-12555 URL: https://issues.apache.org/jira/browse/KAFKA-12555 Project: Kafka Issue Type

[jira] [Created] (KAFKA-10510) Reassigning partitions should not allow increasing RF of a partition unless configured with it

2020-09-21 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-10510: --- Summary: Reassigning partitions should not allow increasing RF of a partition unless configured with it Key: KAFKA-10510 URL: https://issues.apache.org/jira/browse

Re: [ANNOUNCE] Apache Kafka 2.6.0

2020-08-06 Thread Stanislav Kozlovski
Sax, > maulin-vasavada, Michael Viamari, Michal T, Mickael Maison, Mitch, Navina > Ramesh, Navinder Pal Singh Brar, nicolasguyomar, Nigel Liang, Nikolay, > Okada Haruki, Paul, Piotr Fras, Radai Rosenblatt, Rajini Sivaram, Randall > Hauch, Rens Groothuijsen, Richard Yu, Rigel Bezerra de Melo,

[jira] [Resolved] (KAFKA-10353) Trogdor - Fix RoundTripWorker to not fail when the topic it's trying to create already exists

2020-08-04 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-10353. - Resolution: Duplicate > Trogdor - Fix RoundTripWorker to not fail w

[jira] [Created] (KAFKA-10353) Trogdor - Fix RoundTripWorker to not fail when the topic it's trying to create already exists

2020-08-04 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-10353: --- Summary: Trogdor - Fix RoundTripWorker to not fail when the topic it's trying to create already exists Key: KAFKA-10353 URL: https://issues.apache.org/jira/browse

[jira] [Resolved] (KAFKA-10301) Partition#remoteReplicasMap can be empty in certain race conditions

2020-07-27 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-10301. - Resolution: Fixed > Partition#remoteReplicasMap can be empty in certain r

[jira] [Created] (KAFKA-10302) Ensure thread-safe access to Partition#remoteReplicasMap

2020-07-23 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-10302: --- Summary: Ensure thread-safe access to Partition#remoteReplicasMap Key: KAFKA-10302 URL: https://issues.apache.org/jira/browse/KAFKA-10302 Project: Kafka

[jira] [Created] (KAFKA-10301) RemoteReplicasMap can be empty in certain race conditions

2020-07-23 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-10301: --- Summary: RemoteReplicasMap can be empty in certain race conditions Key: KAFKA-10301 URL: https://issues.apache.org/jira/browse/KAFKA-10301 Project

Re: [VOTE] KIP-627: Expose Trogdor-specific JMX Metrics for Tasks and Agents

2020-06-26 Thread Stanislav Kozlovski
+1 (non-binding). Thanks for the work! I am also happy to see Trogdor being improved Best, Stanislav On Fri, Jun 26, 2020 at 5:34 AM Colin McCabe wrote: > +1 (binding). > > Thanks, Sam. > > best, > Colin > > > On Thu, Jun 25, 2020, at 18:05, Gwen Shapira wrote: > > +1 (binding) > > > > Thank

[jira] [Resolved] (KAFKA-8723) flaky test LeaderElectionCommandTest#testAllTopicPartition

2020-06-02 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski resolved KAFKA-8723. Resolution: Fixed > flaky test LeaderElectionCommandTest#testAllTopicPartit

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-04-24 Thread Stanislav Kozlovski
> > >> > > > > > >> 900. Multi-tenancy - one concern I would have with a cluster and > > > > > >> broker-level configuration is that it is possible for a user to > > > > > >> consume a large pr

[jira] [Created] (KAFKA-9866) Do not attempt to elect preferred leader replicas which are outside ISR

2020-04-14 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-9866: -- Summary: Do not attempt to elect preferred leader replicas which are outside ISR Key: KAFKA-9866 URL: https://issues.apache.org/jira/browse/KAFKA-9866

[jira] [Created] (KAFKA-9617) Replica Fetcher can mark partition as failed when max.message.bytes is changed

2020-02-27 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-9617: -- Summary: Replica Fetcher can mark partition as failed when max.message.bytes is changed Key: KAFKA-9617 URL: https://issues.apache.org/jira/browse/KAFKA-9617

[jira] [Created] (KAFKA-9589) LogValidatorTest#testLogAppendTimeNonCompressedV2 is not executed and does not pass

2020-02-21 Thread Stanislav Kozlovski (Jira)
Stanislav Kozlovski created KAFKA-9589: -- Summary: LogValidatorTest#testLogAppendTimeNonCompressedV2 is not executed and does not pass Key: KAFKA-9589 URL: https://issues.apache.org/jira/browse/KAFKA-9589

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-14 Thread Stanislav Kozlovski
Congratulations to all! Best, Stanislav On Tue, Jan 14, 2020 at 9:30 AM Gwen Shapira wrote: > Hi everyone, > > I'm happy to announce that Colin McCabe, Vahid Hashemian and Manikumar > Reddy are now members of Apache Kafka PMC. > > Colin and Manikumar became committers on Sept 2018 and Vahid on

Re: [VOTE] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2020-01-06 Thread Stanislav Kozlovski
+1 (non-binding) Thanks for the KIP, Brian! On Thu, Jan 2, 2020 at 7:15 PM Brian Byrne wrote: > Hello all, > > After further discussion and improvements, I'd like to reinstate the voting > process. > > The updated KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-526 >

Re: [DISCUSS] KIP-552: Add interface to handle unused config

2020-01-06 Thread Stanislav Kozlovski
n see 3 options with its pros and cons > discussed in the thread: > 1) extra config with interface to handle unused > 2) change unused warn to debug > 3) add extra logger for unused > > Please let me know what do you think. > > Thanks, > Artur > > On Mon,

[VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

2020-01-04 Thread Stanislav Kozlovski
Hey there, I'm restarting the vote thread for KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation We had some discussions on the previous vote thread which I believe were

Re: [DISCUSS] KIP-552: Add interface to handle unused config

2019-12-30 Thread Stanislav Kozlovski
Hi all, Would printing all the unused configurations in one line, versus N lines, be more helpful? I know that it would greatly reduce the verbosity in log visualization tools like Kibana while still allowing us to see all the relevant information without the need for an explicit action (e.g

Re: [DISCUSS] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

2019-12-26 Thread Stanislav Kozlovski
Hey Brian, 1. Could we more explicitly clarify the behavior of the algorithm when `|T| > TARGET_METADATA_FETCH SIZE` ? I assume we ignore the config in that scenario 2. Should `targetMetadataFetchSize = Math.max(topicsPerSec / 10, 20)` be `topicsPerSec * 10` ? 3. When is this new algorithm

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

2019-12-13 Thread Stanislav Kozlovski
throttle rate but it will > be applied to all non-ISR replication traffic. This is can be undesirable > because during reassignment it also applies to non-reassignment replication > and causes a replica to be throttled if it falls out of ISR. Also if > leadership changes during reassign

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

2019-12-10 Thread Stanislav Kozlovski
Hey Viktor, I like your latest idea regarding the replication/reassignment configs interplay - I think it makes sense for replication to always be higher. A small matrix of possibilities in the KIP may be useful to future readers (users) To be extra clear: 1. if reassignment.throttle is -1,

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

2019-12-04 Thread Stanislav Kozlovski
gt; > (3,2,1) > >> and P2: (4,5,6) -> (7,8,9) the P2 reassignment would elect a new leader > but > >> P1 wouldn't and it wouldn't help the goal of normalizing traffic on > broker > >> 1 that much. > >> Again, I'll have to check how the current algorith

Re: [DISCUSS] KIP-548 Add Option to enforce rack-aware custom partition reassignment execution

2019-11-22 Thread Stanislav Kozlovski
Hello Satish, Could you provide a link to the KIP? I am unable to find it in the KIP parent page https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals Thanks, Stanislav On Fri, Nov 22, 2019 at 8:21 AM Satish Bellapu wrote: > Hi All, > > This [KIP-548] is basically

Re: [VOTE] KIP-544: Make metrics exposed via JMX configurable

2019-11-11 Thread Stanislav Kozlovski
+1 (non-binding). Thanks Xavier On Sat, Nov 9, 2019 at 9:54 AM Manikumar wrote: > +1 (binding). Thanks for the KIP. > > > On Sat, Nov 9, 2019 at 3:11 PM Alexandre Dupriez < > alexandre.dupr...@gmail.com> wrote: > > > +1 (non-binding) > > > > Le ven. 8 nov. 2019 à 20:21, Bill Bejeck a écrit :

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

2019-11-04 Thread Stanislav Kozlovski
t; think we should still apply the replication throttle to ensure the broker > won't have problems. What do you think? > > Thanks, > Viktor > > > On Fri, Nov 1, 2019 at 9:57 AM Stanislav Kozlovski > > wrote: > > > Hey Viktor. Thanks for the KIP! > > >

Re: [DISCUSS] KIP-536: Propagate broker timestamp to Admin API

2019-11-01 Thread Stanislav Kozlovski
Hey Noa, KIP-436 added a JMX metric in Kafka for this exact use-case, called `start-time-ms`. Perhaps it would be useful to name this public interface in the same way for consistency. Could you update the KIP to include the specific RPC changes regarding the metadata request/responses? Here is a

Re: [VOTE] KIP-541: Create a fetch.max.bytes configuration for the broker

2019-11-01 Thread Stanislav Kozlovski
+1 (non-binding). Thanks! Stanislav On Fri, Oct 25, 2019 at 2:29 PM David Arthur wrote: > +1 binding, this will be a nice improvement. Thanks, Colin! > > -David > > On Fri, Oct 25, 2019 at 4:33 AM Tom Bentley wrote: > > > +1 nb. Thanks! > > > > On Fri, Oct 25, 2019 at 7:43 AM Ismael Juma

  1   2   3   4   >