Re: Logging in Kafka

Hi Mickael, I agree it's good to make it clear about what we're going to adopt for the logging library. For keeping reload4j or adopting log4j2, I don't have any preference TBH. But if Viktor is willing to drive log4j2 tasks, then we don't have any reason not to adopt log4j2. (Thanks Viktor!)

[jira] [Reopened] (KAFKA-15538) Client support for java regex based subscription

[ https://issues.apache.org/jira/browse/KAFKA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phuc Hong Tran reopened KAFKA-15538: > Client support for java regex based subscription >

[jira] [Resolved] (KAFKA-15538) Client support for java regex based subscription

[ https://issues.apache.org/jira/browse/KAFKA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phuc Hong Tran resolved KAFKA-15538. Resolution: Fixed > Client support for java regex based subscription >

[jira] [Created] (KAFKA-16114) Fix partiton not retention after cancel alter intra broker log dir task

wangliucheng created KAFKA-16114: Summary: Fix partiton not retention after cancel alter intra broker log dir task Key: KAFKA-16114 URL: https://issues.apache.org/jira/browse/KAFKA-16114 Project:

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2560

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2559

See

[jira] [Created] (KAFKA-16113) AsyncKafkaConsumer: Add missing offset commit metrics

Philip Nee created KAFKA-16113: -- Summary: AsyncKafkaConsumer: Add missing offset commit metrics Key: KAFKA-16113 URL: https://issues.apache.org/jira/browse/KAFKA-16113 Project: Kafka Issue

Re: Apache Kafka 3.7.0 Release

Thanks Colin, With that, I believe we are out of blockers. I was traveling today and couldn't build an RC - expect one to be published tomorrow (barring any problems). In the meanwhile - here is a PR for the 3.7 blog post - https://github.com/apache/kafka-site/pull/578 Best, Stan On Wed, Jan

Re: [PR] 3.7: Add documentation for Kafka 3.7 [kafka-site]

stanislavkozlovski commented on code in PR #578: URL: https://github.com/apache/kafka-site/pull/578#discussion_r1448042147 ## blog.html: ## @@ -22,6 +22,119 @@ Blog + + + +

Re: Kafka 3.0 support Java 8

Hi, We attempt to support the last 3 non-patch versions. This would mean we would try to Backport security vulnerabilities to a 3.x (probably 3.8) for 6 to 9 months after the last release. Best, --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 |

Re: Kafka 3.0 support Java 8

Thanks. And how long Kafka 3.x will be supported. Thanks On Wed, Jan 10, 2024 at 3:40 PM Divij Vaidya wrote: > All versions in the 3.x series of Kafka will support Java 8. > > Starting Kafka 4.0, we will drop support for Java 8. Clients will support > >= JDK 11 and other packages will support

Re: Kafka 3.0 support Java 8

All versions in the 3.x series of Kafka will support Java 8. Starting Kafka 4.0, we will drop support for Java 8. Clients will support >= JDK 11 and other packages will support >= JDK 17. More details about Java in Kafka 4.0 can be found here:

Kafka 3.0 support Java 8

Hi, I wonder how long Kafka 3.0 can support Java 8. Thanks & Regards, *Devinder Singh* P *Please consider the environment before printing this email*

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2558

See

Re: Logging in Kafka

Hi, I think the only thing that would need to be done in 3.8 is the deprecation of the log4j appender (KIP-719). This was a pre-req for migrating to log4j2 due to conflicts when having both log4j and log4j2 in the classpath. I don't know if that's still the case with reload4j but I think we

[jira] [Created] (KAFKA-16112) Review JMX metrics in Async Consumer and determine the missing ones

Kirk True created KAFKA-16112: - Summary: Review JMX metrics in Async Consumer and determine the missing ones Key: KAFKA-16112 URL: https://issues.apache.org/jira/browse/KAFKA-16112 Project: Kafka

[jira] [Created] (KAFKA-16111) Implement tests for tricky rebalance callbacks scenarios

Kirk True created KAFKA-16111: - Summary: Implement tests for tricky rebalance callbacks scenarios Key: KAFKA-16111 URL: https://issues.apache.org/jira/browse/KAFKA-16111 Project: Kafka Issue

[jira] [Created] (KAFKA-16110) Implement consumer performance tests

Kirk True created KAFKA-16110: - Summary: Implement consumer performance tests Key: KAFKA-16110 URL: https://issues.apache.org/jira/browse/KAFKA-16110 Project: Kafka Issue Type: New Feature

[jira] [Created] (KAFKA-16109) Ensure system tests cover the "simple consumer + commit" use case

Kirk True created KAFKA-16109: - Summary: Ensure system tests cover the "simple consumer + commit" use case Key: KAFKA-16109 URL: https://issues.apache.org/jira/browse/KAFKA-16109 Project: Kafka

Re: Logging in Kafka

Hi Mickael, Thanks for bringing this up. If we move to log4j2 in 4.0, is there any work that needs to be done in 3.8? That's probably what we should focus on. P.S. My assumption is that if the log4j2 work misses the train, we'll stick with reload4j in 4.0. Hopefully this won't happen. best,

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

On Wed, Jan 10, 2024, at 09:16, Justine Olshan wrote: > Hmm it seems like Colin and Proven are disagreeing with whether we can swap > unstable metadata versions. > >> When we reorder, we are always allocating a new MV and we are never > reusing an existing MV even if it was also unstable. > >>

Re: [PROPOSAL] Add commercial support page on website

I agree with Divji here and to be more pointed. I worry that if we go down the path of adding vendors to a list it comes off as supporting their product, not to mention could be a huge security risk for novice users. I would rather this be a callout to other purely open source tooling, such as

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #134

See

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

Hmm it seems like Colin and Proven are disagreeing with whether we can swap unstable metadata versions. > When we reorder, we are always allocating a new MV and we are never reusing an existing MV even if it was also unstable. > Given that this is true, there's no reason to have special rules

Re: Logging in Kafka

Hi Viktor, A logging library that requires Java 17 is a deal breaker since we need to log from modules that will only require Java 11 in Apache Kafka 4.0. Ismael On Wed, Jan 10, 2024 at 6:43 PM Viktor Somogyi-Vass wrote: > Hi Mickael, > > Reacting to your points: > 1. I think it's somewhat

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

Bumping this thread since I've not seen any feedback. Thanks, Mickael On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison wrote: > > Hi, > > I'd like to start a vote on KIP-877: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugins+and+connectors+to+register+metrics >

Re: [DISCUSS] KIP-853: KRaft Controller Membership Changes

Hey Jose, One additional thought. It would be helpful to have an example to justify the need for this: > Wait for the fetch offset of the replica (ID, UUID) to catch up to the log end offset of the leader. It is helpful also to explain how this affects the AddVoter RPC. Do we wait indefinitely?

[jira] [Resolved] (KAFKA-16098) State updater may attempt to resume a task that is not assigned anymore

[ https://issues.apache.org/jira/browse/KAFKA-16098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-16098. --- Resolution: Fixed > State updater may attempt to resume a task that is not assigned anymore

[jira] [Resolved] (KAFKA-15747) KRaft support in DynamicConnectionQuotaTest

[ https://issues.apache.org/jira/browse/KAFKA-15747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-15747. Fix Version/s: 3.8.0 Resolution: Fixed > KRaft support in

Re: Logging in Kafka

Hi Mickael, Reacting to your points: 1. I think it's somewhat unfortunate that we provide an appender tied to a chosen logger implementation. I think that this shouldn't be part of the project in its current form. However, there is the sl4fj2 Fluent API which may solve our problem and turn

Re: Logging in Kafka

Hi, A couple of PMC members from Apache Logging replied and they said they plan to keep supporting log4j2 for several years. https://lists.apache.org/thread/6n6bkgwj8tglgdgzz8wxhkx1p1xpwodl Thanks, Mickael On Wed, Jan 10, 2024 at 3:57 PM Mickael Maison wrote: > > I asked for details about the

Re: [PROPOSAL] Add commercial support page on website

I don't see a need for this. What additional information does this provide over what can be found via a quick google search? My primary concern is that we are getting in the business of listing vendors in the project site which brings it's own complications without adding much additional value

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2557

See

[PROPOSAL] Add commercial support page on website

Hi, After starting a first thread on this topic (https://lists.apache.org/thread/kkox33rhtjcdr5zztq3lzj7c5s7k9wsr), I would like to propose a PR: https://github.com/apache/kafka-site/pull/577 The purpose of this proposal is to help users to find support for sla, training,

Re: [PR] Add get support page [kafka-site]

fpapon commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1885101241 @ableegoldman I pushed some changes, feel free to review/comment :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.7 #57

See

Re: Kafka trunk test & build stability

Hey folks We seem to have a handle on the OOM issues with the multiple fixes community members made. In https://issues.apache.org/jira/browse/KAFKA-16052, you can see the "before" profile in the description and the "after" profile in the latest comment to see the difference. To prevent future

Re: Logging in Kafka

I asked for details about the future of log4j2 on the logging user list: https://lists.apache.org/thread/6n6bkgwj8tglgdgzz8wxhkx1p1xpwodl Let's see what they say. Thanks, Mickael On Wed, Jan 10, 2024 at 3:23 PM Ismael Juma wrote: > > Hi Mickael, > > Thanks for starting the discussion and for

[jira] [Resolved] (KAFKA-15866) Refactor OffsetFetchRequestState Error handling to be more consistent with OffsetCommitRequestState

[ https://issues.apache.org/jira/browse/KAFKA-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-15866. Fix Version/s: 3.7.0 (was: 3.8.0) Assignee: (was: Lan

Re: [PR] Add get support page [kafka-site]

fpapon commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884961276 @mimaison ok thanks for your feeedbacks. I will update the PR according to the comments of the people and then resend an email to the mailing for the proposal and see how the community

Re: [DISCUSS] KIP-971 Expose replication-offset-lag MirrorMaker2 metric

Hi Elxan, Thanks for the KIP, it looks like a useful addition. Can you add to the KIP the default value you propose for replication.lag.metric.refresh.interval? In MirrorMaker most interval configs can be set to -1 to disable them, will it be the case for this new feature or will this setting

[jira] [Created] (KAFKA-16108) Backport fix for KAFKA-16093 to 3.7

Chris Egerton created KAFKA-16108: - Summary: Backport fix for KAFKA-16093 to 3.7 Key: KAFKA-16108 URL: https://issues.apache.org/jira/browse/KAFKA-16108 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-16107) Ensure consumer does not start fetching from added partitions until onPartitionsAssgined completes

Lianet Magrans created KAFKA-16107: -- Summary: Ensure consumer does not start fetching from added partitions until onPartitionsAssgined completes Key: KAFKA-16107 URL:

Re: Logging in Kafka

Hi Mickael, Thanks for starting the discussion and for summarizing the state of play. I agree with you that it would be important to understand how long log4j2 will be supported for. An alternative would be sl4fj 2.x and logback. Ismael On Wed, Jan 10, 2024 at 2:17 PM Mickael Maison wrote: >

Re: [VOTE] KIP-971: Expose replication-offset-lag MirrorMaker2 metric

Hi Elxan, +1 (binding). Thanks, Viktor On Mon, Jan 8, 2024 at 5:57 PM Dániel Urbán wrote: > Hi Elxan, > +1 (non-binding) > Thanks for the KIP, this will be a very useful metric for MM! > Daniel > > Elxan Eminov ezt írta (időpont: 2024. jan. 7., > V, > 2:17): > > > Hi all, > > Bumping this

[jira] [Resolved] (KAFKA-16097) State updater removes task without pending action in EOSv2

[ https://issues.apache.org/jira/browse/KAFKA-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy resolved KAFKA-16097. Resolution: Fixed > State updater removes task without pending action in EOSv2 >

Re: [PR] Add get support page [kafka-site]

mimaison commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884814599 No worries, there's no need to blame anybody for such a small issue. The initiative to add this page is, in my opinion, good and contributions are always welcome. It's just the initial

Re: [PR] Add get support page [kafka-site]

jbonofre commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884745111 @mimaison I take the blame on me: I did mistakes with the bright side of things. 1. I thought this kind of change was approved by the community/PMC 2. As the same page exists in

Logging in Kafka

Hi, Starting a new thread to discuss the current logging situation in Kafka. I'll restate everything we know but see the [DISCUSS] Road to Kafka 4.0 if you are interested in what has already been said. [0] Currently Kafka uses SLF4J and reload4j as the logging backend. We had to adopt reload4j

Re: [PR] Add get support page [kafka-site]

fpapon commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884696661 @mimaison I replied to the original thread on the mailing list to rebirth the discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2556

See

Re: [PR] Add get support page [kafka-site]

fpapon commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884677270 > You have to admit this PR raised quite a few red flags! > > First it's directly copied from Camel without replacing mentions to Kafka. Then within minutes it's approved by 2

Re: [PR] Add get support page [kafka-site]

mimaison commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884671597 As I said, I think a page listing commercial offerings is useful. I was just commenting on the approach you took. I don't think we necessarily need a committer to build this page

Re: [DISCUSS] Road to Kafka 4.0

It may be worth starting a new thread with regards to the logging situation. Ismael On Wed, Jan 10, 2024 at 12:00 PM Mickael Maison wrote: > Hi Colin, > > Regarding KIP-719, I think need it to land in 3.8 if we want to remove > the appender in 4.0. I also just noticed the log4j's KafkaAppender

Re: [PR] Add get support page [kafka-site]

rmannibucau commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884622256 @mimaison well I wouldn't say red flags but I fully understand the surprise on kafka side if it was never discussed on/offline first - we had the same debate on TomEE side years

Re: [PR] Add get support page [kafka-site]

mimaison commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884592204 You have to admit this PR raised quite a few red flags! First it's directly copied from Camel without replacing mentions to Kafka. Then within minutes it's approved by 2 other

Re: [DISCUSS] Road to Kafka 4.0

Hi Colin, Regarding KIP-719, I think need it to land in 3.8 if we want to remove the appender in 4.0. I also just noticed the log4j's KafkaAppender is being deprecated in log4j2 and will not be part of log4j3. For KIP-653, as I said, my point was to gauge interest in getting it done. While it

Re: [PR] Add get support page [kafka-site]

fpapon commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884502000 > @fpapon , thanks for the PR. Could you explain the motivation that why we should add `Get Support` page? Why is the `Contact Us` page not enough? Thanks. - This page explain more

Re: [DISCUSS] KIP-1014: Managing Unstable Metadata Versions in Apache Kafka

Hi folks, > If you use an unstable MV, you probably won't be able to upgrade your > software. Because whenever something changes, you'll probably get > serialization exceptions being thrown inside the controller. Fatal ones. Thanks for this clarification. I think this concrete risk should be

Re: [PR] Add get support page [kafka-site]

fpapon commented on PR #577: URL: https://github.com/apache/kafka-site/pull/577#issuecomment-1884372269 @showuon There is a lot of ASF projects that provide a page with community support, it help the adoption and the growth of the project because the users need commercial support for

Re: [PR] Add get support page [kafka-site]

rmannibucau commented on code in PR #577: URL: https://github.com/apache/kafka-site/pull/577#discussion_r1447016147 ## support.html: ## @@ -0,0 +1,54 @@ + + + + + + + Support + Community support + + +