Re: [VOTE] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder

+1 (binding) Thanks Frank! Luke On Fri, May 24, 2024 at 5:21 PM Josep Prat wrote: > Hi Frank, > > thanks for the KIP. > +1 (binding) > > Best, > > On Fri, May 24, 2024 at 11:11 AM Kuan Po Tseng > wrote: > > > +1 (non-binding) > > > > On 2024/05/23 16:26:42 Frank Yang wrote: > > > Hi all, > >

Re: Inquire about a bug issue

Hi Jianbin, Thanks for asking. I'll review the PR this week or next week. Let's target this bug fix for v3.7.1 and v3.8.0. Thanks. Luke On Fri, May 24, 2024 at 11:20 AM Jianbin Chen wrote: > I would like to inquire if anyone is paying attention to this issue >

[jira] [Created] (KAFKA-16828) RackAwareTaskAssignorTest failed

Luke Chen created KAFKA-16828: - Summary: RackAwareTaskAssignorTest failed Key: KAFKA-16828 URL: https://issues.apache.org/jira/browse/KAFKA-16828 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder

LGTM! Thanks for raising this improvement. Luke On Thu, May 23, 2024 at 12:52 AM Chia-Ping Tsai wrote: > Thanks for Josep's response > > > We can add this to 3.8.0, but keep in mind the KIP is not voted yet (as > far > as I can see), so I would highly encourage to start the vote thread ASAP >

[jira] [Created] (KAFKA-16814) KRaft broker cannot startup when `partition.metadata` is missing

Luke Chen created KAFKA-16814: - Summary: KRaft broker cannot startup when `partition.metadata` is missing Key: KAFKA-16814 URL: https://issues.apache.org/jira/browse/KAFKA-16814 Project: Kafka

[jira] [Resolved] (KAFKA-16783) Migrate RemoteLogMetadataManagerTest to new test infra

[ https://issues.apache.org/jira/browse/KAFKA-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16783. --- Fix Version/s: 3.8.0 Resolution: Fixed > Migrate RemoteLogMetadataManagerTest to new t

[jira] [Resolved] (KAFKA-16760) alterReplicaLogDirs failed even if responded with none error

[ https://issues.apache.org/jira/browse/KAFKA-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16760. --- Resolution: Not A Problem > alterReplicaLogDirs failed even if responded with none er

Re: [VOTE] KIP-950: Tiered Storage Disablement

Thanks Chia-Ping! Since ZK is going to be removed, I agree the KRaft part has higher priority. But if Christo or the community contributor has spare time, it's good to have ZK part, too! Thanks. Luke On Thu, May 16, 2024 at 5:45 PM Chia-Ping Tsai wrote: > +1 but I prefer to ship it to KRaft

[jira] [Created] (KAFKA-16781) Expose advertised.listeners in controller node

Luke Chen created KAFKA-16781: - Summary: Expose advertised.listeners in controller node Key: KAFKA-16781 URL: https://issues.apache.org/jira/browse/KAFKA-16781 Project: Kafka Issue Type

Re: [DISCUSS] Apache Kafka 3.7.1 release

Hi Igor, Thanks for volunteering! +1 Luke On Wed, May 15, 2024 at 11:15 PM Mickael Maison wrote: > Hi Igor, > > Thanks for volunteering, +1 > > Mickael > > On Thu, Apr 25, 2024 at 11:09 AM Igor Soarez wrote: > > > > Hi everyone, > > > > I'd like to volunteer to be the release manager for a

Re: [VOTE] KIP-950: Tiered Storage Disablement

Hi Christo, In addition to the minor comments left in the discussion thread, it LGTM. +1 from me. Thank you. Luke On Tue, May 14, 2024 at 11:21 PM Christo Lolov wrote: > Heya! > > I would like to start a vote on KIP-950: Tiered Storage Disablement in > order to catch the last Kafka release

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

o start a VOTE tomorrow > so we can get at least part of this KIP in 3.8. > > Best, > Christo > > On Fri, 10 May 2024 at 11:11, Luke Chen wrote: > > > Hi Christo, > > > > > 1. I am not certain I follow the question. From DISABLED you can only > go > &

[jira] [Created] (KAFKA-16760) alterReplicaLogDirs failed even if responded with none error

Luke Chen created KAFKA-16760: - Summary: alterReplicaLogDirs failed even if responded with none error Key: KAFKA-16760 URL: https://issues.apache.org/jira/browse/KAFKA-16760 Project: Kafka

[jira] [Created] (KAFKA-16711) highestOffsetInRemoteStorage is not updated after logDir altering within broker

Luke Chen created KAFKA-16711: - Summary: highestOffsetInRemoteStorage is not updated after logDir altering within broker Key: KAFKA-16711 URL: https://issues.apache.org/jira/browse/KAFKA-16711 Project

[jira] [Created] (KAFKA-16709) move logDir within broker might cause log cleanup hanging

Luke Chen created KAFKA-16709: - Summary: move logDir within broker might cause log cleanup hanging Key: KAFKA-16709 URL: https://issues.apache.org/jira/browse/KAFKA-16709 Project: Kafka Issue

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

> > >> 1. In the ZK mode, how will the transition from DISABLING to DISABLED > > >> state happen? > > >> For the "retain" policy, until we delete all the remote-log segments, > > the > > >> state will be > > >> DISABLING and th

Re: [VOTE] KIP-1018: Introduce max remote fetch timeout config

Hi Kamal, Thanks for the KIP! +1 from me. Thanks. Luke On Mon, May 6, 2024 at 5:03 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi all, > > We would like to start a voting thread for KIP-1018: Introduce > max remote fetch timeout config for DelayedRemoteFetch requests. >

Re: request permissions to contribute to Kafka

Hi Zhisheng, I've granted your permission. Thank you. Luke On Tue, May 7, 2024 at 10:25 AM Zhisheng Zhang <31791909...@gmail.com> wrote: > Hi > > I'd like to request permissions to contribute to Kafka to propose a KIP > > Wiki ID：zhangzhisheng > Jira ID：zhangzhisheng > > Thank you >

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

Hi Kamal, Thanks for the KIP! Sorry for the late review. Overall LGTM! Just 1 question: If one fetch request contains 2 partitions: [p1, p2] fetch.max.wait.ms: 500, remote.fetch.max.wait.ms: 1000 And now, p1 fetch offset is the log end offset and has no new data coming, and p2 fetch offset is

[jira] [Created] (KAFKA-16661) add a lower `log.initial.task.delay.ms` value to integration test framework

Luke Chen created KAFKA-16661: - Summary: add a lower `log.initial.task.delay.ms` value to integration test framework Key: KAFKA-16661 URL: https://issues.apache.org/jira/browse/KAFKA-16661 Project: Kafka

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Also, I think using `stopReplicas` request is a good idea because it won't cause any problems while migrating to KRaft mode. The stopReplicas request is one of the request that KRaft controller will send to ZK brokers during migration. Thanks. Luke On Fri, May 3, 2024 at 11:48 AM Luke Chen

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

l >>> mechanism to handle the remote stray data. >>> >>> The existing topic deletion, for instance, handles stray logs on disk by >>> detecting them on KafkaBroker startup and deleting before the >>> ReplicaManager is started. >>> Maybe we need a

[jira] [Resolved] (KAFKA-16467) Add README to docs folder

[ https://issues.apache.org/jira/browse/KAFKA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16467. --- Fix Version/s: 3.8.0 Resolution: Fixed > Add README to docs fol

[jira] [Resolved] (KAFKA-16563) migration to KRaft hanging after MigrationClientException

[ https://issues.apache.org/jira/browse/KAFKA-16563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16563. --- Fix Version/s: 3.8.0 3.7.1 Resolution: Fixed > migration to KRaft hang

Re: [VOTE] KIP-1023: Follower fetch from tiered offset

Hi Abhijeet, Thanks for the KIP. +1 from me. Thanks. Luke On Fri, Apr 26, 2024 at 5:41 PM Omnia Ibrahim wrote: > Thanks for the KIP. +1 non-binding from me > > > On 26 Apr 2024, at 06:29, Abhijeet Kumar > wrote: > > > > Hi All, > > > > I would like to start the vote for KIP-1023 - Follower

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

section that talked about the drawback when a > > > > node built with last-tiered-offset has become a leader. As Abhijeet > > > > mentioned, we plan to have a follow-up KIP that will address these by > > > > having a deprioritzation of these brokers. The depr

Re: [Confluence] Request for an account

> 8861094754 > > > On Tue, Mar 26, 2024 at 9:11 AM Luke Chen wrote: > > > Hi Johnny, > > > > Currently, there is an infra issue about this: > > https://issues.apache.org/jira/browse/INFRA-25451 , and unfortunately > it's > > not fixed, yet. > > I

[jira] [Created] (KAFKA-16617) Add KRaft info for the `advertised.listeners` doc description

Luke Chen created KAFKA-16617: - Summary: Add KRaft info for the `advertised.listeners` doc description Key: KAFKA-16617 URL: https://issues.apache.org/jira/browse/KAFKA-16617 Project: Kafka

Re: [ANNOUNCE] New committer: Igor Soarez

Congrats, Igor! On Thu, Apr 25, 2024 at 6:10 AM Matthias J. Sax wrote: > Congrats! > > On 4/24/24 2:29 PM, Bill Bejeck wrote: > > Congrats Igor! > > > > -Bill > > > > On Wed, Apr 24, 2024 at 2:37 PM Tom Bentley wrote: > > > >> Congratulations Igor! > >> > >> On Thu, 25 Apr 2024 at 6:27 AM,

Re: [VOTE] KIP-1037: Allow WriteTxnMarkers API with Alter Cluster Permission

Hi Nikhil, Thanks for the KIP. +1 from me. Luke On Mon, Apr 22, 2024 at 7:41 PM Andrew Schofield wrote: > Hi Nikhil, > Thanks for the KIP. Looks good to me. > > +1 (non-binding) > > Thanks, > Andrew > > > On 22 Apr 2024, at 09:17, Christo Lolov wrote: > > > > Heya Nikhil, > > > > Thanks for

[jira] [Resolved] (KAFKA-16424) truncated logs will be left undeleted after alter dir completion

[ https://issues.apache.org/jira/browse/KAFKA-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16424. --- Fix Version/s: 3.8.0 Resolution: Fixed > truncated logs will be left undeleted after al

[jira] [Created] (KAFKA-16563) migration to KRaft hanging after KeeperException

Luke Chen created KAFKA-16563: - Summary: migration to KRaft hanging after KeeperException Key: KAFKA-16563 URL: https://issues.apache.org/jira/browse/KAFKA-16563 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-16552) Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests

Luke Chen created KAFKA-16552: - Summary: Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests Key: KAFKA-16552 URL: https://issues.apache.org/jira/browse/KAFKA-16552

Re: Request for Contributor Permission

Hi Suprem, I've granted you access. Thanks. Luke On Mon, Apr 15, 2024 at 7:15 AM Suprem Vanam wrote: > Hi, I'm interested in contributing to Kafka. I have access to the JIRA > board but cannot assign issues to myself. Could you give contributor > permissions to Jira ID: supremvanam. > > Thank

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

Congrats, Greg! On Sun, Apr 14, 2024 at 7:05 AM Viktor Somogyi-Vass wrote: > Congrats Greg! :) > > On Sun, Apr 14, 2024, 00:35 Bill Bejeck wrote: > > > Congrats Greg! > > > > -Bill > > > > On Sat, Apr 13, 2024 at 4:25 PM Boudjelda Mohamed Said < > bmsc...@gmail.com> > > wrote: > > > > >

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

ll. I would then opt for moving information from DisableRemoteTopic > within the StopReplicas API which will then disappear in KRaft world as it > is already scheduled for deprecation. What do you think? > > Best, > Christo > > On Wed, 3 Apr 2024 at 07:59, Luke Chen wrote:

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

Hi Abhijeet, Thanks for the KIP to improve the tiered storage feature! Questions: 1. We could also get the "pending-upload-offset" and epoch via remote log metadata, instead of adding a new API to fetch from the leader. Could you explain why you choose the later approach, instead of the former?

[jira] [Resolved] (KAFKA-16455) Check partition exists before send reassignments to server in ReassignPartitionsCommand

[ https://issues.apache.org/jira/browse/KAFKA-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16455. --- Fix Version/s: 3.8.0 Resolution: Fixed > Check partition exists before send reassignme

[jira] [Resolved] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

[ https://issues.apache.org/jira/browse/KAFKA-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16234. --- Resolution: Fixed > Log directory failure re-creates partitions in another logdir automatica

[jira] [Resolved] (KAFKA-16359) kafka-clients-3.7.0.jar published to Maven Central is defective

[ https://issues.apache.org/jira/browse/KAFKA-16359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16359. --- Fix Version/s: 3.8.0 3.7.1 Resolution: Fixed > kafka-clients-3.7.0.

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

> 2. Why were empty producer snapshots not backfilled for older data > > > when clusters were upgraded from 2.8? > > > 3. Do producer snapshots need to be available contiguously, or can > > > earlier snapshots be empty while later segments do not exist? > > &

Re: [DISCUSS] KIP-1031: Control offset translation in MirrorSourceConnector

Hi Omnia, Thanks for the KIP! It LGTM! But I'm not an expert of MM2, it would be good to see if there is any other comment from MM2 experts. Thanks. Luke On Thu, Mar 14, 2024 at 6:08 PM Omnia Ibrahim wrote: > Hi everyone, I would like to start a discussion thread for KIP-1031: > Control

Re: [DISCUSS] KIP-1028: Docker Official Image for Apache Kafka

Hi Krishna, I also have the same question as Manikumar raised: 1. Will the Docker inventory files/etc are the same for OSS Image and Docker Official Images? If no, then why not? Could we make them identical so that we don't have to build 2 images for each release? Thank you. Luke On Wed, Apr 3,

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

IP-631 deprecated these > > > RPCs. > > > > > > > > > > I will come up with some recommendations on how we could improve > that > > > > one but until then, @Luke please feel free to review the KIP. > > > > > > > > > > @Satish, if we

[jira] [Resolved] (KAFKA-15823) NodeToControllerChannelManager: authentication error prevents controller update

[ https://issues.apache.org/jira/browse/KAFKA-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15823. --- Resolution: Fixed > NodeToControllerChannelManager: authentication error prevents control

[jira] [Resolved] (KAFKA-16323) Failing test: fix testRemoteFetchExpiresPerSecMetric

[ https://issues.apache.org/jira/browse/KAFKA-16323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16323. --- Fix Version/s: 3.8.0 3.7.1 Resolution: Fixed > Failing test:

[jira] [Resolved] (KAFKA-16447) Fix failed ReplicaManagerTest

[ https://issues.apache.org/jira/browse/KAFKA-16447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16447. --- Fix Version/s: 3.8.0 Resolution: Fixed > Fix failed ReplicaManagerT

[jira] [Resolved] (KAFKA-16391) Cleanup .lock file after server is down

[ https://issues.apache.org/jira/browse/KAFKA-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16391. --- Fix Version/s: 3.8.0 Resolution: Fixed > Cleanup .lock file after server is d

[jira] [Resolved] (KAFKA-15949) Improve the KRaft metadata version related messages

[ https://issues.apache.org/jira/browse/KAFKA-15949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15949. --- Fix Version/s: 3.8.0 Resolution: Fixed > Improve the KRaft metadata version related messa

[ANNOUNCE] New committer: Christo Lolov

Hi, Everyone, The PMC of Apache Kafka is pleased to announce a new Kafka committer: Christo Lolov. Christo has been a Kafka contributor since 2021. He has made over 50 commits. He authored KIP-902, KIP-963, and KIP-1005, as well as many tiered storage related tasks. He also co-drives the

[jira] [Created] (KAFKA-16425) wrong output when running log dir movement with kafka-reassign-partitions command

Luke Chen created KAFKA-16425: - Summary: wrong output when running log dir movement with kafka-reassign-partitions command Key: KAFKA-16425 URL: https://issues.apache.org/jira/browse/KAFKA-16425 Project

[jira] [Created] (KAFKA-16424) truncated logs will be left undeleted after alter dir completion

Luke Chen created KAFKA-16424: - Summary: truncated logs will be left undeleted after alter dir completion Key: KAFKA-16424 URL: https://issues.apache.org/jira/browse/KAFKA-16424 Project: Kafka

Re: [Confluence] Request for an account

Hi Johnny, Currently, there is an infra issue about this: https://issues.apache.org/jira/browse/INFRA-25451 , and unfortunately it's not fixed, yet. I think alternatively, maybe you could put your proposal in a shared google doc for discussion. (without comment enabled since we want to keep all

[jira] [Resolved] (KAFKA-16409) kafka-delete-records / DeleteRecordsCommand should use standard exception handling

[ https://issues.apache.org/jira/browse/KAFKA-16409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16409. --- Fix Version/s: 3.8.0 Resolution: Fixed > kafka-delete-records / DeleteRecordsCommand sho

Re: Request to join

Hi, Please send an email to dev-subscr...@kafka.apache.org to subscribe to the group. Thanks. Luke On Mon, Mar 25, 2024 at 4:19 PM durairaj t wrote: > >

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

ank you. Luke On Sat, Mar 23, 2024 at 1:24 PM Arpit Goyal wrote: > Hi Team, > Any further comments or suggestions on the possible approaches discussed > above. > > On Tue, Mar 19, 2024, 09:55 Arpit Goyal wrote: > > > @Luke Chen @Kamal Chandraprakash @Greg > > H

[jira] [Resolved] (KAFKA-16318) Add javadoc to KafkaMetric

[ https://issues.apache.org/jira/browse/KAFKA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16318. --- Fix Version/s: 3.8.0 Resolution: Fixed > Add javadoc to KafkaMet

[jira] [Created] (KAFKA-16399) Add JBOD support in tiered storage

Luke Chen created KAFKA-16399: - Summary: Add JBOD support in tiered storage Key: KAFKA-16399 URL: https://issues.apache.org/jira/browse/KAFKA-16399 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] Different retention semantics for active segment rotation

Hi Jorge, You should check the JIRA: https://issues.apache.org/jira/browse/KAFKA-16385 where we had some discussion. Welcome to provide your thoughts there. Thanks. Luke On Thu, Mar 21, 2024 at 3:33 PM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Hi dev community, > > I'd

[jira] [Resolved] (KAFKA-16222) KRaft Migration: Incorrect default user-principal quota after migration

[ https://issues.apache.org/jira/browse/KAFKA-16222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16222. --- Resolution: Fixed > KRaft Migration: Incorrect default user-principal quota after migrat

[jira] [Created] (KAFKA-16385) Segment is rolled before segment.ms or segment.bytes breached

Luke Chen created KAFKA-16385: - Summary: Segment is rolled before segment.ms or segment.bytes breached Key: KAFKA-16385 URL: https://issues.apache.org/jira/browse/KAFKA-16385 Project: Kafka

[jira] [Resolved] (KAFKA-16342) Fix compressed records

[ https://issues.apache.org/jira/browse/KAFKA-16342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16342. --- Fix Version/s: 3.6.2 3.8.0 3.7.1 Resolution: Fixed

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

Hi Arpit, Thanks for the KIP! I agree with Greg that we should make it clear about backward Compatibility. Since you don't have permission to edit the KIP, you could think about it and write in the email thread directly. Questions: 1. Could you explain what will happen if one partition created

[jira] [Resolved] (KAFKA-15490) Invalid path provided to the log failure channel upon I/O error when writing broker metadata checkpoint

[ https://issues.apache.org/jira/browse/KAFKA-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15490. --- Fix Version/s: 3.6.2 Resolution: Fixed > Invalid path provided to the log failure chan

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Hi Christo, Any update with this KIP? If you don't have time to complete it, I can collaborate with you to work on it. Thanks. Luke On Wed, Jan 17, 2024 at 11:38 PM Satish Duggana wrote: > Hi Christo, > Thanks for volunteering to contribute to the KIP discussion. I suggest > considering this

Re: [VOTE] KIP-956: Tiered Storage Quotas

Thanks for the KIP! +1 from me. Luke On Sun, Mar 10, 2024 at 8:44 AM Satish Duggana wrote: > Thanks Abhijeet for the KIP, +1 from me. > > > On Sat, 9 Mar 2024 at 1:51 AM, Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > +1 (non-binding), Thanks for the KIP, Abhijeet! > > >

Re: [DISCUSS] Apache Kafka 3.6.2 release

+1, Thanks Manikumar! On Thu, Mar 14, 2024 at 3:40 AM Bruno Cadonna wrote: > Thanks Manikumar! > > +1 > > Best, > Bruno > > On 3/13/24 5:56 PM, Josep Prat wrote: > > +1 thanks for volunteering! > > > > Best > > --- > > > > Josep Prat > > Open Source Engineering Director,

Re: [DISCUSS] Minimum constraint for segment.ms

Hi Divij, Thanks for raising this. The valid minimum value 1 for `segment.ms` is completely unreasonable. Similarly for `segment.bytes`, `metadata.log.segment.ms`, `metadata.log.segment.bytes`. In addition to that, there are also some config default values we'd like to propose to change in v4.0.

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

; > use replica.alter.log.dirs.io.max.bytes.per.second. To be > > > > > consistent, > > > > > > > > > perhaps this can be sth like > > > > > > > > remote.log.manager.write.max.bytes.per.seco

[jira] [Resolved] (KAFKA-16209) fetchSnapshot might return null if topic is created before v2.8

[ https://issues.apache.org/jira/browse/KAFKA-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16209. --- Fix Version/s: 3.8.0 3.7.1 Resolution: Fixed > fetchSnapshot might ret

[jira] [Created] (KAFKA-16341) Fix un-compressed records

Luke Chen created KAFKA-16341: - Summary: Fix un-compressed records Key: KAFKA-16341 URL: https://issues.apache.org/jira/browse/KAFKA-16341 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-16342) Fix compressed records

Luke Chen created KAFKA-16342: - Summary: Fix compressed records Key: KAFKA-16342 URL: https://issues.apache.org/jira/browse/KAFKA-16342 Project: Kafka Issue Type: Sub-task Reporter

[jira] [Resolved] (KAFKA-16071) NPE in testDescribeUnderReplicatedPartitionsWhenReassignmentIsInProgress

[ https://issues.apache.org/jira/browse/KAFKA-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16071. --- Resolution: Fixed > NPE in testDescribeUnderReplicatedPartitionsWhenReassignmentIsInProgr

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

Hi Ahmed, Thanks for the KIP! Comments: 1. If we all agree with the suggestion from Andrew, you could update the KIP. Otherwise, LGTM! Thanks. Luke On Thu, Feb 29, 2024 at 1:32 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Ahmed, > Could do. Personally, I find the

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

er and user still tries to remove the voter? Any error returned? Thanks. Luke On Thu, Jan 25, 2024 at 7:50 AM José Armando García Sancio wrote: > Thanks for the feedback Luke. See my comments below: > > On Wed, Jan 24, 2024 at 4:20 AM Luke Chen wrote: > > 1. About &quo

Re: [ANNOUNCE] Apache Kafka 3.7.0

ann, Gantigmaa Selenge, Gaurav Narula, gongzhongqiang, > > Greg Harris, Guozhang Wang, Gyeongwon, Do, Hailey Ni, Hanyu Zheng, Hao > > Li, Hector Geraldino, hudeqi, Ian McDonald, Iblis Lin, Igor Soarez, > > iit2009060, Ismael Juma, Jakub Scholz, James Cheng, Jason Gustafson, >

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

Hi Assane, I also share the same concern as Greg has, which is that the KIP is not kafka ecosystem friendly. And this will make the kafka client and broker have high dependencies that once you use the pluggable compression interface, the producer must be java client. This seems to go against the

Re: [VOTE] 3.7.0 RC4

Hi all, I found there is a bug (KAFKA-16283 ) in the built-in `RoundRobinPartitioner`, and it will cause only half of the partitions receiving the records. (I'm surprised our tests didn't catch it!?) After further testing, I found this issue

[jira] [Created] (KAFKA-16283) RoundRobinPartitioner will only send to half of the partitions in a topic

Luke Chen created KAFKA-16283: - Summary: RoundRobinPartitioner will only send to half of the partitions in a topic Key: KAFKA-16283 URL: https://issues.apache.org/jira/browse/KAFKA-16283 Project: Kafka

[jira] [Created] (KAFKA-16282) Allow to get last stable offset (LSO) in kafka-get-offsets.sh

Luke Chen created KAFKA-16282: - Summary: Allow to get last stable offset (LSO) in kafka-get-offsets.sh Key: KAFKA-16282 URL: https://issues.apache.org/jira/browse/KAFKA-16282 Project: Kafka

[jira] [Resolved] (KAFKA-15140) Improve TopicCommandIntegrationTest to be less flaky

[ https://issues.apache.org/jira/browse/KAFKA-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15140. --- Fix Version/s: (was: 3.5.1) Resolution: Fixed > Improve TopicCommandIntegrationT

[jira] [Resolved] (KAFKA-16247) replica keep out-of-sync after migrating broker to KRaft

[ https://issues.apache.org/jira/browse/KAFKA-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16247. --- Resolution: Fixed Fixed in 3.7.0 RC4 > replica keep out-of-sync after migrating broker to KR

[jira] [Created] (KAFKA-16247) 1 replica keep out-of-sync after migrating broker to KRaft

Luke Chen created KAFKA-16247: - Summary: 1 replica keep out-of-sync after migrating broker to KRaft Key: KAFKA-16247 URL: https://issues.apache.org/jira/browse/KAFKA-16247 Project: Kafka Issue

Re: Improve flaky test reporting (KAFKA-12216)

Hi David, I agree it'll be great if we could distinguish between "flakyFailure" and "alwaysFailure". And I also agree we should still show the "flakyFailure" tests because there could be some potential bugs there. Thanks for working on it. Luke On Tue, Feb 13, 2024 at 12:57 AM Ismael Juma

[jira] [Created] (KAFKA-16232) kafka hangs forever in the starting process if the authorizer future is not returned

Luke Chen created KAFKA-16232: - Summary: kafka hangs forever in the starting process if the authorizer future is not returned Key: KAFKA-16232 URL: https://issues.apache.org/jira/browse/KAFKA-16232

[jira] [Resolved] (KAFKA-16157) Topic recreation with offline disk doesn't update leadership/shrink ISR correctly

[ https://issues.apache.org/jira/browse/KAFKA-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16157. --- Resolution: Fixed > Topic recreation with offline disk doesn't update leadership/shrink

[jira] [Created] (KAFKA-16214) No user info when SASL authentication failure

Luke Chen created KAFKA-16214: - Summary: No user info when SASL authentication failure Key: KAFKA-16214 URL: https://issues.apache.org/jira/browse/KAFKA-16214 Project: Kafka Issue Type: Bug

[jira] [Resolved] (KAFKA-16101) KRaft migration rollback documentation is incorrect

[ https://issues.apache.org/jira/browse/KAFKA-16101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16101. --- Resolution: Fixed > KRaft migration rollback documentation is incorr

Re: Broken build

Thanks for the quick fix, Josep! On Tue, Jan 30, 2024 at 7:21 PM Josep Prat wrote: > Hi Kafka Devs! > I noticed the current trunk build is broken under Scala 2.12 only. > I found the faulty error and sent a PR to fix it: > https://github.com/apache/kafka/pull/15290 > > If someone has the time

[jira] [Created] (KAFKA-16209) fetchSnapshot might return null if topic is created before v2.8

Luke Chen created KAFKA-16209: - Summary: fetchSnapshot might return null if topic is created before v2.8 Key: KAFKA-16209 URL: https://issues.apache.org/jira/browse/KAFKA-16209 Project: Kafka

[jira] [Resolved] (KAFKA-16085) remote copy lag bytes/segments metrics don't update all topic value

[ https://issues.apache.org/jira/browse/KAFKA-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16085. --- Fix Version/s: 3.7.0 Assignee: Luke Chen Resolution: Fixed > remote copy lag by

Re: [DISCUSS] KIP-877: Mechanism for plugins and connectors to register metrics

Hi Mickael, Thanks for the KIP. The motivation and solution makes sense to me. Just one question: If we could extend `closable` for Converter plugin, why don't we do that for the "Unsupported Plugins" without close method? I don't say we must do that in this KIP, but maybe you could add the

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hi José, Thanks for the great KIP. It took me quite a lot of time reading and digesting! Some questions and comments: 1. About "VotersRecord": > When a KRaft voter becomes leader it will write a KRaftVersionRecord and VotersRecord to the log if the log or the latest snapshot doesn't contain any

[jira] [Resolved] (KAFKA-16144) Controller leader checkQuorum timer should skip only 1 controller case

[ https://issues.apache.org/jira/browse/KAFKA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16144. --- Resolution: Fixed > Controller leader checkQuorum timer should skip only 1 controller c

Re: [VOTE] KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

Hi Ziming, +1(binding) from me. Thanks. Luke On Mon, Jan 22, 2024 at 11:50 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (non-binding) > > On Mon, Jan 22, 2024 at 8:34 AM ziming deng > wrote: > >> Hello everyone, >> I'd like to initiate a vote for KIP-1011. >> This KIP

[jira] [Resolved] (KAFKA-16163) Constant resignation/reelection of controller when starting a single node in combined mode

[ https://issues.apache.org/jira/browse/KAFKA-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16163. --- Resolution: Duplicate > Constant resignation/reelection of controller when starting a single n

Re: [VOTE] 3.7.0 RC2

DIRS, > >>>>>>> apiVersion=0, clientId=1000, correlationId=14, headerVersion=2), > >>>>>>> connectionId='172.16.14.219:9090-172.16.14.217:53590-7', > >>>> clientAddress=/ > >>>>>>> 172.16.14.217, principal=U

[jira] [Created] (KAFKA-16162) New created topics are unavailable after upgrading to 3.7

Luke Chen created KAFKA-16162: - Summary: New created topics are unavailable after upgrading to 3.7 Key: KAFKA-16162 URL: https://issues.apache.org/jira/browse/KAFKA-16162 Project: Kafka Issue

[jira] [Resolved] (KAFKA-16132) Upgrading from 3.6 to 3.7 in KRaft will have seconds of partitions unavailable

[ https://issues.apache.org/jira/browse/KAFKA-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16132. --- Resolution: Duplicate > Upgrading from 3.6 to 3.7 in KRaft will have seconds of partiti

[jira] [Created] (KAFKA-16144) Controller leader checkQuorum timer should skip only 1 controller case

Luke Chen created KAFKA-16144: - Summary: Controller leader checkQuorum timer should skip only 1 controller case Key: KAFKA-16144 URL: https://issues.apache.org/jira/browse/KAFKA-16144 Project: Kafka

  1   2   3   4   5   6   7   8   9   >