[jira] [Created] (KAFKA-14342) KafkaOffsetBackingStore should clear offsets for source partitions on tombstone messages

Yash Mayya created KAFKA-14342: -- Summary: KafkaOffsetBackingStore should clear offsets for source partitions on tombstone messages Key: KAFKA-14342 URL: https://issues.apache.org/jira/browse/KAFKA-14342

[jira] [Created] (KAFKA-14343) Write upgrade/downgrade tests for enabling the state updater

Lucas Brutschy created KAFKA-14343: -- Summary: Write upgrade/downgrade tests for enabling the state updater Key: KAFKA-14343 URL: https://issues.apache.org/jira/browse/KAFKA-14343 Project: Kafka

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1328

See

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Happy Monday, everyone! I've updated the KIP with the following changes: * Clarified MetadataType metric usages (broker vs controller) * Added ZkMigrationReady tagged field to ApiVersionsResponse (for use by KRaft controller quorum) * Added MigrationRecord with two states: Started and Finished * D

[jira] [Created] (KAFKA-14344) Build EmbeddedKafkaCluster with common configs used for all clients

Omnia Ibrahim created KAFKA-14344: - Summary: Build EmbeddedKafkaCluster with common configs used for all clients Key: KAFKA-14344 URL: https://issues.apache.org/jira/browse/KAFKA-14344 Project: Kafka

Re: [DISCUSS] KIP-876: Time based cluster metadata snapshots

Hi all, During the implementation of this KIP and some offline discussion with Jason Gustafson, I realized now that Kafka is generating a snapshot every hour and the default metadata retention is to delete snapshots after 7 days, every cluster metadata partition will have 168 (1 snapshot per hour

Re: [DISCUSS] KIP-878: Autoscaling for Statically Partitioned Streams

Hi Sophie, Thank you for the KIP! 1. I do not understand how autoscaling should work with a Streams topology with a stateful sub-topology that reads from the input topics. The simplest example is a topology that consists of only one stateful sub-topology. As far as I understand the upstream p

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Luke and Justine, > For (3), you said: > > - I have some concerns about the impact of this option on the > transactional > producers, for example, what will happen to an ongoing transaction > associated with an expired PID? Would this leave the transactions in a > "hanging" state? > > - How wil

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

Hi, David, Thanks for the updated KIP. A few more comments. 30. LeaderAndIsrRequest/StopReplicaRequest both have a controllerId field. Should we add a KRaftControllerId field like UpdateMetadata? 31. "If a migration has been finalized, but the KRaft quroum comes up with kafka.metadata.migration.

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1329

See

Re: [DISCUSS] KIP-866 ZooKeeper to KRaft Migration

30. I think we can keep the single ControllerId field in those requests since they are only used for fencing (as far as I know). Internally, the broker components that handle those requests will compare the ControllerId with that of MetadataCache (which is updated via UMR). The reason we need the

[jira] [Created] (KAFKA-14345) Flakey DynamicConnectionQuotaTest should use correct error bounds

Greg Harris created KAFKA-14345: --- Summary: Flakey DynamicConnectionQuotaTest should use correct error bounds Key: KAFKA-14345 URL: https://issues.apache.org/jira/browse/KAFKA-14345 Project: Kafka

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1331

See Changes: -- [...truncated 418782 lines...] [2022-11-01T05:13:22.359Z] [2022-11-01T05:13:22.359Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 168 >

Re: [DISCUSS] KIP-878: Autoscaling for Statically Partitioned Streams

Hi Sophie, Thanks for the KIP. A very useful proposal! Some questions: 1. the staticPartition method in the interface is commented out. 2. For error handling, as you can imagine, there could be errors happening during partition expansion.That means, the operation would be (1) take long time to c