Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-26 Thread Kamal Chandraprakash
Hi Ahmed, Thanks for the KIP! It will be a useful addition to know the last-stable-offset (LSO) for a partition. Motivation section is not clear to me. Can you document the scenarios on how the exposed LSO can be used for txn support/debug? -- Kamal On Thu, Mar 21, 2024 at 11:30 PM Justine

[jira] [Resolved] (KAFKA-16391) Cleanup .lock file after server is down

2024-03-26 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16391. --- Fix Version/s: 3.8.0 Resolution: Fixed > Cleanup .lock file after server is down >

[jira] [Resolved] (KAFKA-16385) Segment is rolled before segment.ms or segment.bytes breached

2024-03-26 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16385. Fix Version/s: 3.8.0 Resolution: Fixed > Segment is rolled before segment.ms or

[jira] [Created] (KAFKA-16428) Fix bug where config change notification znode may not get created during migration

2024-03-26 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-16428: Summary: Fix bug where config change notification znode may not get created during migration Key: KAFKA-16428 URL: https://issues.apache.org/jira/browse/KAFKA-16428

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-26 Thread Colin McCabe
Hi all, Thanks for the discussion. Let me respond to a few questions I saw in the thread (hope I didn't miss any!) Feature level 0 is "special" because it effectively means that the feature hasn't been set up. So I could create a foo feature, a bar feature, and a baz feature

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-26 Thread Justine Olshan
Hi all, I've added the notes about requiring 3.3-IV0 and that the tool/rpc will fail if the metadata version is too low. I will wait for Colin's response on the deprecation. I am not opposed to deprecating it but want everyone to agree. Justine On Tue, Mar 26, 2024 at 12:26 PM José Armando

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Christo Lolov
Thank you everyone! It wouldn't have been possible without quite a lot of reviews and extremely helpful inputs from you and the rest of the community! I am looking forward to working more closely with you going forward :) On Tue, 26 Mar 2024 at 14:31, Kirk True wrote: > Congratulations

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-26 Thread José Armando García Sancio
Hi Justine, On Mon, Mar 25, 2024 at 5:09 PM Justine Olshan wrote: > The reason it is not removed is purely for backwards > compatibility. Colin had strong feelings about not removing any flags. We are not saying that we should remove that flag. That would break backward compatibility of 3.8

Re: [DISCUSS] Jakarta and Java EE 9/10 support in Kafka 4.0

2024-03-26 Thread Christopher Shannon
Greg, I definitely understand that part, it's always tricky for users that have older software and haven't or can't upgrade yet. The most important thing to me is getting to the new Jakarta APIs and Java EE updates as that causes incompatibilities with code and requires code changes. So if that

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

2024-03-26 Thread Kamal Chandraprakash
Hi, Thanks for the KIP! Overall the KIP looks good and covered most of the items. 1. Could you explain how the brokers will handle the DisableRemoteTopic API request? 2. Who will initiate the controller interaction sequence? Does the controller listens for topic config updates and initiate the

[jira] [Created] (KAFKA-16427) KafkaConsumer#position() does not respect timeout when group protocol is CONSUMER

2024-03-26 Thread Alyssa Huang (Jira)
Alyssa Huang created KAFKA-16427: Summary: KafkaConsumer#position() does not respect timeout when group protocol is CONSUMER Key: KAFKA-16427 URL: https://issues.apache.org/jira/browse/KAFKA-16427

Permission to assign tickets in Jira

2024-03-26 Thread Pavel Pozdeev
Hi Team,   Would it be possible to get a permission to assign tickets in Jira? I've got a Jira account, but can only leave a comments on tickets, can not assign a ticket to myself. My Jira username: pasharik   Best, Pavel Pozdeev  

Re: [DISCUSS] KIP-1022 Formatting and Updating Features

2024-03-26 Thread Jun Rao
Hi, Justine, Thanks for the reply. 10. If we are exposing group.coordinator.version in this KIP, I think we need to document what RPCs/records it controls or if it has dependencies on MV. 15. "For transaction version specifically, I have no metadata version dependencies besides requiring 3.3

Re: [DISCUSS] Jakarta and Java EE 9/10 support in Kafka 4.0

2024-03-26 Thread Greg Harris
Hey Christopher, Thank you for explaining that jetty version dependency, I was not aware of that. > So are we really locked into JDK 11 for Connect for version 4.0? Until we adopt a KIP to change the minimum version, we must support 11 in 4.0. By default we will continue to support Java

[jira] [Resolved] (KAFKA-16416) Use NetworkClientTest to replace RequestResponseTest to be the example of log4j output

2024-03-26 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16416. Fix Version/s: 3.8.0 Resolution: Fixed > Use NetworkClientTest to replace

Re: [DISCUSS] Jakarta and Java EE 9/10 support in Kafka 4.0

2024-03-26 Thread Christopher Shannon
Hi Greg, You are right that KIP-1013 and the JDK 17 upgrade is not directly relevant because JDK 11 can still be used with Jakarta APIs. However, it's still somewhat relevant and important because if we are stuck at JDK 11 then we can't upgrade to certain versions. For Connect, there is a Jetty

Re: [DISCUSS] Jakarta and Java EE 9/10 support in Kafka 4.0

2024-03-26 Thread Greg Harris
Hi Christopher! Thanks so much for raising this. I agree that we should move to the new namespace in 4.0, and not doing so would be a mistake. This breaking change has a lot of benefits, and the only cost I am aware of is that ConnectRestExtensions will need to be migrated, rebuilt, and

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

2024-03-26 Thread Almog Gavra
Thanks for the thoughts Bruno! > Do you mean a API to configure restoration instead of boolean flag reprocessOnRestore? Yes, this is exactly the type of thing I was musing (but I don't have any concrete suggestions). It feels like that would give the flexibility to do things like the motivation

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Kirk True
Congratulations Christo! > On Mar 26, 2024, at 7:27 AM, Satish Duggana wrote: > > Congratulations Christo! > > On Tue, 26 Mar 2024 at 19:20, Ivan Yurchenko wrote: >> >> Congrats! >> >> On Tue, Mar 26, 2024, at 14:48, Lucas Brutschy wrote: >>> Congrats! >>> >>> On Tue, Mar 26, 2024 at 2:44 

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Satish Duggana
Congratulations Christo! On Tue, 26 Mar 2024 at 19:20, Ivan Yurchenko wrote: > > Congrats! > > On Tue, Mar 26, 2024, at 14:48, Lucas Brutschy wrote: > > Congrats! > > > > On Tue, Mar 26, 2024 at 2:44 PM Federico Valeri > > wrote: > > > > > > Congrats! > > > > > > On Tue, Mar 26, 2024 at 2:27 

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Ivan Yurchenko
Congrats! On Tue, Mar 26, 2024, at 14:48, Lucas Brutschy wrote: > Congrats! > > On Tue, Mar 26, 2024 at 2:44 PM Federico Valeri wrote: > > > > Congrats! > > > > On Tue, Mar 26, 2024 at 2:27 PM Mickael Maison > > wrote: > > > > > > Congratulations Christo! > > > > > > On Tue, Mar 26, 2024 at

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Lucas Brutschy
Congrats! On Tue, Mar 26, 2024 at 2:44 PM Federico Valeri wrote: > > Congrats! > > On Tue, Mar 26, 2024 at 2:27 PM Mickael Maison > wrote: > > > > Congratulations Christo! > > > > On Tue, Mar 26, 2024 at 2:26 PM Chia-Ping Tsai wrote: > > > > > > Congrats Christo! > > > > > > Chia-Ping

Re: [DISCUSS] KIP-477: Add PATCH method for connector config in Connect REST API

2024-03-26 Thread Ivan Yurchenko
Speaking of the Chris' comment > One thought that comes to mind is that sometimes it may be useful to > explicitly remove properties from a connector configuration. We might > permit this by allowing users to specify null (the JSON literal, not a > string containing the characters "null") as the

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Federico Valeri
Congrats! On Tue, Mar 26, 2024 at 2:27 PM Mickael Maison wrote: > > Congratulations Christo! > > On Tue, Mar 26, 2024 at 2:26 PM Chia-Ping Tsai wrote: > > > > Congrats Christo! > > > > Chia-Ping

[jira] [Created] (KAFKA-16426) Initialise KafkaMetricsGroup statically

2024-03-26 Thread Gaurav Narula (Jira)
Gaurav Narula created KAFKA-16426: - Summary: Initialise KafkaMetricsGroup statically Key: KAFKA-16426 URL: https://issues.apache.org/jira/browse/KAFKA-16426 Project: Kafka Issue Type:

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Mickael Maison
Congratulations Christo! On Tue, Mar 26, 2024 at 2:26 PM Chia-Ping Tsai wrote: > > Congrats Christo! > > Chia-Ping

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Chia-Ping Tsai
Congrats Christo! Chia-Ping

RE: Re: [DISCUSS] KIP-477: Add PATCH method for connector config in Connect REST API

2024-03-26 Thread Ivan Yurchenko
Hi all, This KIP is a bit old now :) but I think its context hasn't changed much since then and the KIP is still valid. I would like to finally bring it to some conclusion. Best, Ivan On 2021/07/12 14:49:47 Chris Egerton wrote: > Hi all, > > Know it's been a while for this KIP but in my

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Yash Mayya
Congratulations Christo! On Tue, Mar 26, 2024 at 5:34 PM Luke Chen wrote: > Hi, Everyone, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer: > Christo Lolov. > > Christo has been a Kafka contributor since 2021. He has made over 50 > commits. He authored KIP-902, KIP-963,

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Sagar
Congrats Christo! Sagar. On Tue, 26 Mar 2024 at 6:04 PM, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Congrats Christo!! > > On Tue 26. Mar 2024 at 14.33, Apoorv Mittal > wrote: > > > Congrats Christo! > > > > Regards, > > Apoorv Mittal > > +44 7721681581 > > > > > > On

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Jorge Esteban Quilcate Otoya
Congrats Christo!! On Tue 26. Mar 2024 at 14.33, Apoorv Mittal wrote: > Congrats Christo! > > Regards, > Apoorv Mittal > +44 7721681581 > > > On Tue, Mar 26, 2024 at 12:05 PM Luke Chen wrote: > > > Hi, Everyone, > > > > The PMC of Apache Kafka is pleased to announce a new Kafka committer: > >

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Apoorv Mittal
Congrats Christo! Regards, Apoorv Mittal +44 7721681581 On Tue, Mar 26, 2024 at 12:05 PM Luke Chen wrote: > Hi, Everyone, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer: > Christo Lolov. > > Christo has been a Kafka contributor since 2021. He has made over 50 >

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Viktor Somogyi-Vass
Congrats Christo! Well deserved! On Tue, Mar 26, 2024, 13:27 Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Congrats Christo! > > On Tue, Mar 26, 2024, 17:42 Bruno Cadonna wrote: > > > Congrats, Christo! > > > > Well deserved! > > > > Best, > > Bruno > > > > On 3/26/24 1:04 PM,

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Kamal Chandraprakash
Congrats Christo! On Tue, Mar 26, 2024, 17:42 Bruno Cadonna wrote: > Congrats, Christo! > > Well deserved! > > Best, > Bruno > > On 3/26/24 1:04 PM, Luke Chen wrote: > > Hi, Everyone, > > > > The PMC of Apache Kafka is pleased to announce a new Kafka committer: > > Christo Lolov. > > > >

[jira] [Resolved] (KAFKA-16423) Ignoring offset partition key with an unexpected format for the first element in the partition key list. Expected type: java.lang.String, actual type: null"

2024-03-26 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-16423. --- Fix Version/s: (was: 3.6.2) (was: 3.8.0)

[jira] [Resolved] (KAFKA-16423) Ignoring offset partition key with an unexpected format for the first element in the partition key list. Expected type: java.lang.String, actual type: null"

2024-03-26 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-16423. --- Resolution: Duplicate > Ignoring offset partition key with an unexpected format for the

[jira] [Reopened] (KAFKA-16423) Ignoring offset partition key with an unexpected format for the first element in the partition key list. Expected type: java.lang.String, actual type: null"

2024-03-26 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton reopened KAFKA-16423: --- Assignee: (was: johndoe) > Ignoring offset partition key with an unexpected format

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Bruno Cadonna
Congrats, Christo! Well deserved! Best, Bruno On 3/26/24 1:04 PM, Luke Chen wrote: Hi, Everyone, The PMC of Apache Kafka is pleased to announce a new Kafka committer: Christo Lolov. Christo has been a Kafka contributor since 2021. He has made over 50 commits. He authored KIP-902, KIP-963,

[jira] [Resolved] (KAFKA-15949) Improve the KRaft metadata version related messages

2024-03-26 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15949. --- Fix Version/s: 3.8.0 Resolution: Fixed > Improve the KRaft metadata version related messages

[DISCUSS] Jakarta and Java EE 9/10 support in Kafka 4.0

2024-03-26 Thread Christopher Shannon
Is this already being planned for version 4.0? If not, I strongly thing it should be. Kafka is currently using the old long deprecated javax apis which is going to continue to cause issues [1] for people as more and more things are updated to use Jakarta. With the bump to require JDK 17 for

[ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Luke Chen
Hi, Everyone, The PMC of Apache Kafka is pleased to announce a new Kafka committer: Christo Lolov. Christo has been a Kafka contributor since 2021. He has made over 50 commits. He authored KIP-902, KIP-963, and KIP-1005, as well as many tiered storage related tasks. He also co-drives the

[jira] [Resolved] (KAFKA-16386) NETWORK_EXCEPTIONs from transaction verification are not translated

2024-03-26 Thread Sean Quah (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Quah resolved KAFKA-16386. --- Fix Version/s: 3.8.0 Resolution: Fixed > NETWORK_EXCEPTIONs from transaction verification

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

2024-03-26 Thread Bruno Cadonna
Hi Almog, Do you mean a API to configure restoration instead of boolean flag reprocessOnRestore? Do you already have an idea? The proposal in the KIP is focused on the processor that updates the global state whereas in the case of GlobalKTable and source KTable the issues lies in the

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

2024-03-26 Thread Satish Duggana
Thanks Mehari, Divij, Christo etal for the KIP. I had an initial review of the KIP and left the below comments. 101. For remote.log.disable.policy=delete: Does it delete the remote log data immediately and the data in remote storage will not be taken into account by any replica? That means

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

2024-03-26 Thread Kamal Chandraprakash
Hi, Sorry for the late reply. Greg has raised some good points: > Does an empty producer snapshot have the same behavior as a non-existent snapshot when restored? Producer snapshots maintain the status about the ongoing txn to either COMMIT/ABORT the transaction. In the older version (<2.8), we

[jira] [Created] (KAFKA-16425) wrong output when running log dir movement with kafka-reassign-partitions command

2024-03-26 Thread Luke Chen (Jira)
Luke Chen created KAFKA-16425: - Summary: wrong output when running log dir movement with kafka-reassign-partitions command Key: KAFKA-16425 URL: https://issues.apache.org/jira/browse/KAFKA-16425

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

2024-03-26 Thread Bruno Cadonna
Hi Walker, I have follow-up comments. 1. I think, we should add an overload to the StreamsBuilder class that allows to name the processor with Named. That makes that processor consistent with all other processors in the DSL regarding naming. 2. I do not understand what you mean with

[jira] [Created] (KAFKA-16424) truncated logs will be left undeleted after alter dir completion

2024-03-26 Thread Luke Chen (Jira)
Luke Chen created KAFKA-16424: - Summary: truncated logs will be left undeleted after alter dir completion Key: KAFKA-16424 URL: https://issues.apache.org/jira/browse/KAFKA-16424 Project: Kafka

[jira] [Created] (KAFKA-16423) Spurious log warnings: "Ignoring offset partition key with an unexpected format for the second element in the partition key list. Expected type: java.lang.String, actual

2024-03-26 Thread johndoe (Jira)
johndoe created KAFKA-16423: --- Summary: Spurious log warnings: "Ignoring offset partition key with an unexpected format for the second element in the partition key list. Expected type: java.lang.String, actual type: null" Key: