Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Hi Greg, I'm sorry if anything I said sounded like I'm trying to minimize the concerns that's definitely not my intention. My background is in databases, and I share your concerns of muddying transaction semantics (which is already, as you pointed out, is different in Kafka). If calling some kin

[jira] [Created] (KAFKA-17191) close RLMM and expire task after empty remote storage in "retain" policy

Luke Chen created KAFKA-17191: - Summary: close RLMM and expire task after empty remote storage in "retain" policy Key: KAFKA-17191 URL: https://issues.apache.org/jira/browse/KAFKA-17191 Project: Kafka

[jira] [Created] (KAFKA-17192) MirrorMaker2 worker config does not pass config.providers value to connectors

Bertalan Kondrat created KAFKA-17192: Summary: MirrorMaker2 worker config does not pass config.providers value to connectors Key: KAFKA-17192 URL: https://issues.apache.org/jira/browse/KAFKA-17192

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi, I echo the commend about symmetry. I expect additional use of the background events in AK 4.0, so adding these metrics while we are in here makes a lot of sense for that reason too. Thanks, Andrew > On 23 Jul 2024, at 22:18, Lianet M. wrote: > > Hey all, > > Follow-up on Bruno's point BC2.

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #74

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.7 #187

See

Re: [PR] Add blog post for 3.8 release [kafka-site]

soarez commented on code in PR #614: URL: https://github.com/apache/kafka-site/pull/614#discussion_r1689466531 ## 38/upgrade.html: ## @@ -28,7 +28,8 @@ Notable changes in 3 If READ is not authorized, checkpointing is limited to offsets mirrorred after the start of

Re: [PR] Add blog post for 3.8 release [kafka-site]

jlprat commented on code in PR #614: URL: https://github.com/apache/kafka-site/pull/614#discussion_r1689481100 ## 38/upgrade.html: ## @@ -28,7 +28,8 @@ Notable changes in 3 If READ is not authorized, checkpointing is limited to offsets mirrorred after the start of

Re: [VOTE] 3.8.0 RC3

Hi Josep, Thank you once again for running this release. I noticed that site-docs/upgrade.html in kafka_2.13-3.8.0-site-docs.tgz does not include the corrections from https://github.com/apache/kafka-site/pull/614 namely the replacement of version 3.6 with 3.7 in the upgrade notes header, and the

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi, 1. The title is a bit misleading. It's proposing to add new metrics, JMX is just one of the mechanisms to export them. 2. +1 to not register the new metrics when using the classic consumer, instead of setting them to 0. Similarly I assume existing metrics that don't apply to the new consumers

Re: [VOTE] 3.8.0 RC3

Hi Igor, You are right, the corrections we are doing on the open pull request are not in the doc artifact. But as far as I know, this shouldn't be considered a blocker for the release as the site will contain the latest and most correct information for the 3.8.x version. Thank you and also Chris a

Re: [PR] Add blog post for 3.8 release [kafka-site]

soarez commented on code in PR #614: URL: https://github.com/apache/kafka-site/pull/614#discussion_r1689497473 ## 38/upgrade.html: ## @@ -28,7 +28,8 @@ Notable changes in 3 If READ is not authorized, checkpointing is limited to offsets mirrorred after the start of

Re: [PR] Add blog post for 3.8 release [kafka-site]

jlprat commented on code in PR #614: URL: https://github.com/apache/kafka-site/pull/614#discussion_r1689510626 ## 38/upgrade.html: ## @@ -28,7 +28,8 @@ Notable changes in 3 If READ is not authorized, checkpointing is limited to offsets mirrorred after the start of

Re: [PR] MINOR: document how we deal with advisories for dependencies [kafka-site]

divijvaidya merged PR #554: URL: https://github.com/apache/kafka-site/pull/554 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apa

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi Mickael, 1. I agree that the title is misleading. It should be something like "New metrics for the AsyncKafkaConsumer". Maybe it should even be "Metrics for the new consumer". 3. I am not sure I understand this comment. Exposed metrics are public as far as I understand. So adding new metri

[jira] [Created] (KAFKA-17193) Pin external GitHub actions to specific git hash

Mickael Maison created KAFKA-17193: -- Summary: Pin external GitHub actions to specific git hash Key: KAFKA-17193 URL: https://issues.apache.org/jira/browse/KAFKA-17193 Project: Kafka Issue Ty

Re: [VOTE] 3.8.0 RC3

Hi Josep, Thanks for running the release! I am +1 (non binding) Downloaded from sources https://github.com/apache/kafka/archive/refs/tags/3.8.0-rc3.tar.gz and built with java 17. I performed the following validations: 1. I deleted a partition.metadata for a topic manually, and tried starting th

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi, 3. Sorry I misunderstood something and got confused as the KIP mentions AsyncKafkaConsumer as a new consumer implementation. I now see that it's not an alternative to KafkaConsumer but just the "backend" used with the new protocol. So yes any new metrics added are public API and I agree AsyncK

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

Hi Divij, The rare scenario we are discussing is similar to an empty follower trying to catch up with the leader for a topic that is not enabled with tiered storage. Consider the following steps: 1. Follower requests offset 0 from the leader. 2. Offset 0 is no more valid on the leader as its log

[jira] [Resolved] (KAFKA-17104) InvalidMessageCrcRecordsPerSec is not updated in validating LegacyRecord

[ https://issues.apache.org/jira/browse/KAFKA-17104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17104. Fix Version/s: 3.9.0 Resolution: Fixed > InvalidMessageCrcRecordsPerSec is not upda

Re: [VOTE] 3.8.0 RC3

+1 (non-binding). As with the previous RCs, I used the staged Scala 2.13 binaries and the staged Maven artifacts. All seems to work fine. Thanks! Jakub On Tue, Jul 23, 2024 at 12:07 PM Josep Prat wrote: > Hello Kafka users, developers and client-developers, > > This is the fourth candidate for

[jira] [Resolved] (KAFKA-17179) Don't add `integration` tag when using `ClusterTestExtensions`

[ https://issues.apache.org/jira/browse/KAFKA-17179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17179. Fix Version/s: 3.9.0 Resolution: Fixed > Don't add `integration` tag when using `Cl

[jira] [Resolved] (KAFKA-17160) Always import `mockito-inline` if the mockito version is 4.x

[ https://issues.apache.org/jira/browse/KAFKA-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17160. Fix Version/s: 3.9.0 Resolution: Fixed > Always import `mockito-inline` if the mock

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

The difference between the two scenarios you mentioned is that with Tiered Storage, the chances of hitting this scenario increases since a user is likely to have an aggressive setting for local disk data cleanup, which would not be the case in empty followers catching up in a non-tiered storage wor

Re: [VOTE] 3.8.0 RC3

Hi Josep, That makes sense to me, thanks for clarifying. +1 non-binding from me then. -- Igor

Re: [PR] Add blog post for 3.8 release [kafka-site]

soarez commented on code in PR #614: URL: https://github.com/apache/kafka-site/pull/614#discussion_r1689910317 ## blog.html: ## @@ -22,6 +22,98 @@ Blog + + + +Apache Kafka 3.8.0

[jira] [Created] (KAFKA-17194) Don't create cluster for MetadataQuorumCommandTest#testCommandConfig

Chia-Ping Tsai created KAFKA-17194: -- Summary: Don't create cluster for MetadataQuorumCommandTest#testCommandConfig Key: KAFKA-17194 URL: https://issues.apache.org/jira/browse/KAFKA-17194 Project: Kaf

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3135

See

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi Mickael, Thank you for your feedback. 1. I can see that, I will get that changed. 2. Done Thanks, Brenden On Wed, Jul 24, 2024 at 5:06 AM Mickael Maison wrote: > Hi, > > 1. The title is a bit misleading. It's proposing to add new metrics, > JMX is just one of the mechanisms to export them.

[PR] MINOR: Fix plugin path instructions in 3.8 quickstart [kafka-site]

C0urante opened a new pull request, #615: URL: https://github.com/apache/kafka-site/pull/615 This ports a fix contained in https://github.com/apache/kafka/pull/16375 over to 3.8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] MINOR: Fix plugin path instructions in 3.8 quickstart [kafka-site]

chia7712 commented on code in PR #615: URL: https://github.com/apache/kafka-site/pull/615#discussion_r1690139779 ## 38/quickstart.html: ## @@ -237,7 +237,7 @@ Edit the config/connect-standalone.properties file, add or change the plugin.path configuration property

Re: [VOTE] 3.8.0 RC3

Hi Josep, I have: - checked signatures and checksums - run ZK and KRaft quickstarts with the Scala 2.13 binaries and with both the kafka and kafka-native images - built and run tests from source with Java 17 +1 (binding) Thanks, Mickael On Wed, Jul 24, 2024 at 4:22 PM Igor Soarez wrote: > > Hi

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Hi, Mickael, Thanks for the KIP. 10. A common case is only one log dir per broker. Could we support sth like --add-config cordoned.log.dirs=* to make it more convenient for this case? 11. Since we changed the metadata record format, should we gate the new configuration based on a new metadata ve

Re: [PR] MINOR: Fix plugin path instructions in 3.8 quickstart [kafka-site]

C0urante commented on code in PR #615: URL: https://github.com/apache/kafka-site/pull/615#discussion_r1690302020 ## 38/quickstart.html: ## @@ -237,7 +237,7 @@ Edit the config/connect-standalone.properties file, add or change the plugin.path configuration property

Re: [PR] MINOR: Fix plugin path instructions in 3.8 quickstart [kafka-site]

C0urante merged PR #615: URL: https://github.com/apache/kafka-site/pull/615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3136

See

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hello everyone, I believe now this KIP is ready for a vote. I will be starting the vote here momentarily. Thanks again, Brenden On Wed, Jul 24, 2024 at 11:01 AM Brenden Deluna wrote: > Hi Mickael, > Thank you for your feedback. > > 1. I can see that, I will get that changed. > 2. Done > > Than

[VOTE] KIP-1068: New metrics for the new KafkaConsumer

Hello everyone, I would like to start the vote on KIP-1068: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1068%3A+New+metrics+for+the+new+KafkaConsumer This KIP introduces new metrics for the new KafkaConsumer implementation for broader metrics coverage. Thanks, Brenden

[jira] [Created] (KAFKA-17195) transaction-coordinator:test failing with junit error

Greg Harris created KAFKA-17195: --- Summary: transaction-coordinator:test failing with junit error Key: KAFKA-17195 URL: https://issues.apache.org/jira/browse/KAFKA-17195 Project: Kafka Issue Typ

[jira] [Created] (KAFKA-17196) KRaft should consider no using wall-clock for computing timer expiration

José Armando García Sancio created KAFKA-17196: -- Summary: KRaft should consider no using wall-clock for computing timer expiration Key: KAFKA-17196 URL: https://issues.apache.org/jira/browse/KAFKA-171

[jira] [Created] (KAFKA-17197) Return a better message when the user provides invalid cluster ids

José Armando García Sancio created KAFKA-17197: -- Summary: Return a better message when the user provides invalid cluster ids Key: KAFKA-17197 URL: https://issues.apache.org/jira/browse/KAFKA-17197

[jira] [Created] (KAFKA-17198) Partition imbalance after alter due to inconsistent order of broker list

Xuze Yang created KAFKA-17198: - Summary: Partition imbalance after alter due to inconsistent order of broker list Key: KAFKA-17198 URL: https://issues.apache.org/jira/browse/KAFKA-17198 Project: Kafka

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3137

See

[jira] [Created] (KAFKA-17199) add unit tests for TransactionLogConfigs and TransactionStateManagerConfigs

Chia-Ping Tsai created KAFKA-17199: -- Summary: add unit tests for TransactionLogConfigs and TransactionStateManagerConfigs Key: KAFKA-17199 URL: https://issues.apache.org/jira/browse/KAFKA-17199 Proje

Re: [PR] MINOR: Fix plugin path instructions in 3.8 quickstart [kafka-site]

chia7712 commented on code in PR #615: URL: https://github.com/apache/kafka-site/pull/615#discussion_r1690139779 ## 38/quickstart.html: ## @@ -237,7 +237,7 @@ Edit the config/connect-standalone.properties file, add or change the plugin.path configuration property

[jira] [Resolved] (KAFKA-17176) Move FileLock to server-common module

[ https://issues.apache.org/jira/browse/KAFKA-17176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17176. Fix Version/s: 3.9.0 Resolution: Fixed > Move FileLock to server-common module > --