[jira] [Created] (KAFKA-13645) Support the TopologyTestDriver with modular topologies

A. Sophie Blee-Goldman created KAFKA-13645: -- Summary: Support the TopologyTestDriver with modular topologies Key: KAFKA-13645 URL: https://issues.apache.org/jira/browse/KAFKA-13645 Project: Ka

[jira] [Created] (KAFKA-13644) Support global state stores with modular topologies

A. Sophie Blee-Goldman created KAFKA-13644: -- Summary: Support global state stores with modular topologies Key: KAFKA-13644 URL: https://issues.apache.org/jira/browse/KAFKA-13644 Project: Kafka

[jira] [Created] (KAFKA-13643) Replace "NamedTopology" with "ModularTopology" in the codebase

A. Sophie Blee-Goldman created KAFKA-13643: -- Summary: Replace "NamedTopology" with "ModularTopology" in the codebase Key: KAFKA-13643 URL: https://issues.apache.org/jira/browse/KAFKA-13643 Pr

[jira] [Resolved] (KAFKA-13534) Upgrade Log4j to 2.15.0 - CVE-2021-44228

[ https://issues.apache.org/jira/browse/KAFKA-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-13534. --- Resolution: Duplicate > Upgrade Log4j to 2.15.0 - CVE-2021-44228 > -

Re: [VOTE] KIP-591: Add Kafka Streams config to set default state store

Thanks Luke for driving the KIP discussion as well as preparing the POC! The POC PR is especially useful for me to get all the implementation devils in the details :) Guozhang On Thu, Feb 3, 2022 at 7:04 PM Luke Chen wrote: > Hi Matthias and all, > > Thanks for your comments. > I've updated th

Re: [VOTE] KIP-591: Add Kafka Streams config to set default state store

Hi Matthias and all, Thanks for your comments. I've updated the KIP to remove the Stores API change. And I'll also close the vote for this KIP. The KIP is accepted with 3 binding votes (Matthias, Guozhang, and John). Thank you very much. Luke On Tue, Feb 1, 2022 at 4:36 PM Luke Chen wrote: >

[VOTE] KIP-679: Producer will enable the strongest delivery guarantee by default

Hi all, Some context about this thread: In KIP-679, we set the default producer config with `enable.idempotence=true` and `acks=all` in v3.0.0. Here's the discussion and vote thread link: https://lists.apache.org/thread/hfwwpyl3kyj381v45dfnyrw9ddrkyc29 https://lists.apache.org/thread/g6gr9m2pxp5wv

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Hello Nick, Thanks for bringing this up and for the proposed options. I read though your writeup and here are some of my thoughts: 1) When changing the topology of Kafka Streams, the developer need to first decide if the whole topology's persisted state (including both the state store as well as

Re: [DISCUSS] KIP-818: Introduce cache-size-bytes-total Task Level Metric

Thanks Sagar for proposing the KIP, and Sophie for sharing your thoughts. Here're my 2c: I think I agree with Sophie for making the two metrics (both the added and the newly proposed) on INFO level since we are always calculating them anyways. Regarding the level of the cache-size though, I'm thin

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Thanks Guozhang, yes the motivation is to support the s390x platform. It's not a critical bug for other platforms. Any chance that gaining platform support is also a valid reason? I was hoping it would be but I won't submit a PR if it isn't. Thanks, Jon "Guozhang Wang" wrote on 2022-02-03 02

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #660

See

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Hello Jonathan, I think Bruno's point is that we can only upgrade in the bugfix releases if the old version of rocksDB has a critical bug that the new version would fix. For 6.27.3 it seems not fixing a critical bug but for a new feature, and hence unless we change the policy we cannot upgrade in

[jira] [Created] (KAFKA-13642) We should have a command-line tool + API to display fenced / unfenced / etc. brokers

Colin McCabe created KAFKA-13642: Summary: We should have a command-line tool + API to display fenced / unfenced / etc. brokers Key: KAFKA-13642 URL: https://issues.apache.org/jira/browse/KAFKA-13642

Re: [DISCUSS] KIP-818: Introduce cache-size-bytes-total Task Level Metric

Hey Sagar, thanks for the KIP! And yes, all metrics are considered part of the public API and thus require a KIP to add (or modify, etc...) Although in this particular case, you could probably make a good case for just considering it as an update to the original KIP which added the analogous metr

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Thanks for the info Bruno. In that case, if no other concerns, I'll try updating RocksDB to v6.27.3 on the 3.0 and 3.1 branches and file issues and PRs if everything looks good. Jon "Bruno Cadonna" wrote on 2022-02-03 10:40:00 AM: > From: "Bruno Cadonna" > To: dev@kafka.apache.org > Date:

Build failed in Jenkins: Kafka » kafka-2.6-jdk8 #145

See Changes: [github] MINOR: Use ducktape version 0.7.17 (#11730) -- [...truncated 6.36 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldThrowIf

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Hi Jonathan, We had to wait until AK 3.0 to upgrade RocksDB to 6.19.3 due to source compatibility issue. More specifically, we expose RocksDB APIs in Kafka Streams for configuring RocksDB and those RocksDB APIs changed. So upgrading RocksDB was actually a compatibility breaking change. We had

Question: Lazy client connections and status check

Hi All, The admin, consumer and producer client creation are all lazy connections, in the sense that one only knows that the connection is “healthy” when an “action” is carried out (e.g., send a message). There is no API call to check if the connection is healthy or if there are any issues wit

Re: [DISCUSS] KIP-795: Add public APIs for AbstractCoordinator

Thanks David for reading the KIP and sharing your thoughts. I still believe the Group Membership protocol has enormous potential to be used outside of Core Kafka. Even if AbstractCordinator is, as you suggested, not the right abstraction to build upon, there's still merit in the idea of having t

Kafka <= 3.1 upgrade RocksDB to v6.27.3?

The rocksdbjni dependency has been upgraded to v6.27.3 on trunk and I wanted to ask if it would be ok to also upgrade it to v6.27.3 on the 3.1 branch (and possibly earlier branches). I thought I should ask in case there are some policies around changing dependency versions in point releases. The

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #659

See Changes: -- [...truncated 437938 lines...] [2022-02-03T13:49:05.822Z] org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest > shouldMigrateInMemoryKeyValueS

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.1 #72

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.0 #172

See Changes: -- [...truncated 418107 lines...] [2022-02-03T13:15:33.014Z] [2022-02-03T13:15:33.014Z] UpdateFeaturesTest > testPartialSuccessDuringValidFeatureUpgradeAndInval

Re: [DISCUSS] KIP-704: Send a hint to broker if it is an unclean leader

Hi José, Thanks for your reply. The behavior of the leader is clear. However, the part which is not clear to me is how followers which could get fetch requests from consumers as well will handle them. Sorry if I was not clear. Best, David On Thu, Feb 3, 2022 at 5:47 AM José Armando García Sanci

Re: [VOTE] KIP-814: Static membership protocol should let the leader skip assignment

+1 from me as well. The KIP is accepted with 3 binding votes (Jason, Guozhang, and me) and 2 non-binding votes (Luke and Ryan). Thanks for your votes! Best, David On Wed, Feb 2, 2022 at 7:21 PM Guozhang Wang wrote: > > +1. Thanks David! > > On Tue, Feb 1, 2022 at 9:39 AM Ryan Leslie (BLOOMBERG

[jira] [Resolved] (KAFKA-13583) Fix FetchRequestBetweenDifferentIbpTest flaky tests

[ https://issues.apache.org/jira/browse/KAFKA-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-13583. - Fix Version/s: 3.2.0 3.1.1 Reviewer: David Jacot Resolution: F

[jira] [Resolved] (KAFKA-13637) User default.api.timeout.ms config as default timeout for KafkaConsumer.endOffsets

[ https://issues.apache.org/jira/browse/KAFKA-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-13637. - Fix Version/s: 3.0.1 3.2.0 3.1.1 Reviewer: David J