Re: What does kafka streams groupBy does internally?

2024-01-30 Thread Matthias J. Sax
Did reply on SO. -Matthias On 1/24/24 2:18 AM, warrior2...@gmail.com wrote: Let's say there's a topic in which chunks of different files are all mixed up represented by a tuple |(FileId, Chunk)|. Chunks of a same file also can be a little out of order. The task is to aggregate all files and

Re: Re-key by multiple properties without composite key

2024-01-30 Thread Matthias J. Sax
Both fk1 and fk2 point to the PK of another entity (not shown for brevity, of no relevance to the question). It this two independent FK, or one two-column FK? Ingesting the topic into a Kafka Streams application, how can I re-key the resulting KTable by both fk1 and fk2? If you read the

Re: Kafka consumer group crashing and not able to consume once service is up

2024-01-30 Thread Matthias J. Sax
I am a not sure if I can follow completely. From the figures you show, you have a topic with 4 partitions, and 4 consumer groups. Thus, each consumer group should read all 4 partitions, but the figure indicate that each group would read a single partition only? Can you clarify? Are you using

[jira] [Resolved] (KAFKA-10184) Flaky HighAvailabilityTaskAssignorIntegrationTest.shouldScaleOutWithWarmupTasksAndPersistentStores

2024-01-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10184. - Assignee: (was: John Roesler) Resolution: Cannot Reproduce This ticket

[jira] [Resolved] (KAFKA-10184) Flaky HighAvailabilityTaskAssignorIntegrationTest.shouldScaleOutWithWarmupTasksAndPersistentStores

2024-01-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10184. - Assignee: (was: John Roesler) Resolution: Cannot Reproduce This ticket

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2024-01-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16017: Fix Version/s: 3.5.3 3.4.2 3.6.2 > Checkpoin

[jira] [Updated] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16141: Fix Version/s: 3.7.0 > StreamsStandbyTask##test_standby_tasks_rebalanceArgume

Re: [VOTE] 3.7.0 RC2

2024-01-17 Thread Matthias J. Sax
Stan, thanks for driving this all forward! Excellent job. About StreamsStandbyTask - https://issues.apache.org/jira/browse/KAFKA-16141 StreamsUpgradeTest - https://issues.apache.org/jira/browse/KAFKA-16139 For `StreamsUpgradeTest` it was a test setup issue and should be fixed now in trunk

[jira] [Updated] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16141: Component/s: streams > StreamsStandbyTask##test_standby_tasks_rebalanceArgume

[jira] [Updated] (KAFKA-16139) StreamsUpgradeTest fails consistently in 3.7.0

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16139: Component/s: streams system tests > StreamsUpgradeTest fa

[jira] [Assigned] (KAFKA-15770) org.apache.kafka.streams.integration.ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy is flaky

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15770: --- Assignee: Matthias J. Sax

[jira] [Updated] (KAFKA-16123) KStreamKStreamJoinProcessor forwards null records for left/outer joins unconditionally of the join window.

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16123: Component/s: streams > KStreamKStreamJoinProcessor forwards null records for left/ou

[jira] [Updated] (KAFKA-16159) Prune excessive logging from Telemetry Reporter

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16159: Affects Version/s: 3.7.0 > Prune excessive logging from Telemetry Repor

[jira] [Updated] (KAFKA-16159) Prune excessive logging from Telemetry Reporter

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16159: Fix Version/s: (was: 3.8.0) > Prune excessive logging from Telemetry Repor

[jira] [Resolved] (KAFKA-16139) StreamsUpgradeTest fails consistently in 3.7.0

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16139. - Fix Version/s: 3.7.0 3.6.1 Resolution: Fixed

[jira] [Resolved] (KAFKA-16139) StreamsUpgradeTest fails consistently in 3.7.0

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16139. - Fix Version/s: 3.7.0 3.6.1 Resolution: Fixed

[jira] [Updated] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16141: Issue Type: Bug (was: Test) > StreamsStandbyTask##test_standby_tasks_rebalanceArgume

[jira] [Comment Edited] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807890#comment-17807890 ] Matthias J. Sax edited comment on KAFKA-16141 at 1/17/24 9:15 PM

[jira] [Created] (KAFKA-16158) Cleanup usage of `TimestampedBytesStore` interface

2024-01-17 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-16158: --- Summary: Cleanup usage of `TimestampedBytesStore` interface Key: KAFKA-16158 URL: https://issues.apache.org/jira/browse/KAFKA-16158 Project: Kafka

[jira] [Created] (KAFKA-16158) Cleanup usage of `TimestampedBytesStore` interface

2024-01-17 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-16158: --- Summary: Cleanup usage of `TimestampedBytesStore` interface Key: KAFKA-16158 URL: https://issues.apache.org/jira/browse/KAFKA-16158 Project: Kafka

[jira] [Commented] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807890#comment-17807890 ] Matthias J. Sax commented on KAFKA-16141: - After discussion with Almog (and testing

[jira] [Assigned] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-16141: --- Assignee: Matthias J. Sax (was: Almog Gavra) > StreamsStandbyT

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

2024-01-17 Thread Matthias J. Sax
tream here ???/* On 2024-Jan-13 01:22, Matthias J. Sax wrote: `KGroupedStream` is just an "intermediate representation" to get a better flow in the DSL. It's not a "top level" abstraction like KStream/KTable. For `KTable` there is `transformValue()` -- there is no `tra

[jira] [Commented] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807543#comment-17807543 ] Matthias J. Sax commented on KAFKA-16141: - Thanks. I am wondering if the right fix would

[jira] [Commented] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807411#comment-17807411 ] Matthias J. Sax commented on KAFKA-16141: - Assigned to [~agavra] and marked as blocker. Might

[jira] [Updated] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16141: Priority: Blocker (was: Major) > StreamsStandbyT

[jira] [Assigned] (KAFKA-16141) StreamsStandbyTask##test_standby_tasks_rebalanceArguments:{ “metadata_quorum”: “ISOLATED_KRAFT”, “use_new_coordinator”: false} fails consistently in 3.7

2024-01-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-16141: --- Assignee: Almog Gavra > StreamsStandbyTask##test_standby_tasks_rebalanceArgume

Re: [PROPOSAL] Add commercial support page on website

2024-01-12 Thread Matthias J. Sax
François, thanks for starting this initiative. Personally, I don't think it's necessarily harmful for the project to add such a new page, however, I share the same concerns others raised already. I understand your motivation that people had issues finding commercial support, but I am not

Re: [PROPOSAL] Add commercial support page on website

2024-01-12 Thread Matthias J. Sax
François, thanks for starting this initiative. Personally, I don't think it's necessarily harmful for the project to add such a new page, however, I share the same concerns others raised already. I understand your motivation that people had issues finding commercial support, but I am not

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

2024-01-12 Thread Matthias J. Sax
`KGroupedStream` is just an "intermediate representation" to get a better flow in the DSL. It's not a "top level" abstraction like KStream/KTable. For `KTable` there is `transformValue()` -- there is no `transform()` because keying must be preserved -- if you want to change the keying you

Re: [Kafka Streams], Processor API for KTable and KGroupedStream

2024-01-12 Thread Matthias J. Sax
`KGroupedStream` is just an "intermediate representation" to get a better flow in the DSL. It's not a "top level" abstraction like KStream/KTable. For `KTable` there is `transformValue()` -- there is no `transform()` because keying must be preserved -- if you want to change the keying you

[jira] [Commented] (KAFKA-16089) Kafka Streams still leaking memory

2024-01-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806217#comment-17806217 ] Matthias J. Sax commented on KAFKA-16089: - Great job! > Kafka Streams still leaking mem

[jira] [Resolved] (KAFKA-14949) Add Streams upgrade tests from AK 3.4

2024-01-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14949. - Resolution: Fixed > Add Streams upgrade tests from AK

[jira] [Resolved] (KAFKA-14949) Add Streams upgrade tests from AK 3.4

2024-01-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14949. - Resolution: Fixed > Add Streams upgrade tests from AK

[jira] [Assigned] (KAFKA-14949) Add Streams upgrade tests from AK 3.4

2024-01-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-14949: --- Assignee: Mickael Maison > Add Streams upgrade tests from AK

[jira] [Updated] (KAFKA-16092) Queues for Kafka

2024-01-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16092: Issue Type: New Feature (was: Improvement) > Queues for Ka

[jira] [Commented] (KAFKA-14404) Fix & update docs on client configs controlled by Streams

2024-01-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803543#comment-17803543 ] Matthias J. Sax commented on KAFKA-14404: - It seems [~sujayopensource] lost interest to work

Re: Apache Kafka 3.7.0 Release

2024-01-05 Thread Matthias J. Sax
and merged in trunk, I'll update the 3.7 branch as well. Regards, Apoorv Mittal +44 7721681581 On Thu, Jan 4, 2024 at 12:49 PM Matthias J. Sax wrote: We found a blocker for 3.7: https://issues.apache.org/jira/browse/KAFKA-16077 Already having a PR under review to fix it. -Matthias On 1/3

Re: Apache Kafka 3.7.0 Release

2024-01-04 Thread Matthias J. Sax
We found a blocker for 3.7: https://issues.apache.org/jira/browse/KAFKA-16077 Already having a PR under review to fix it. -Matthias On 1/3/24 10:43 AM, Stanislav Kozlovski wrote: Hey all, happy new year. Thanks for the heads up Almog. Makes sense. To give an update - I haven't been able

[jira] [Updated] (KAFKA-16077) Streams fails to close task after restoration when input partitions are updated

2024-01-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16077: Priority: Blocker (was: Critical) > Streams fails to close task after restoration w

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2024-01-02 Thread Matthias J. Sax
Congrats! On 12/29/23 9:40 AM, Viktor Somogyi-Vass wrote: Congrats Divij, well deserved! On Fri, Dec 29, 2023, 09:36 Lucas Brutschy wrote: Congratulations, Divij! On Fri, Dec 29, 2023 at 1:32 AM Colin McCabe wrote: Congratulations, Divij! best, Colin On Thu, Dec 28, 2023, at 11:38,

Re: [DISCUSS] Connect Jira component name

2024-01-02 Thread Matthias J. Sax
SGTM. I found it always odd that it's named `KafkaConnect` :) -Matthias On 12/26/23 6:03 PM, Greg Harris wrote: Hi Connect Developers, I noticed recently that we had two Jira components: "KafkaConnect" and "connect", one with >1000 issues and one with 20 issues. I merged the two tags,

[jira] [Updated] (KAFKA-16055) Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders

2024-01-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16055: Component/s: streams > Thread unsafe use of HashMap stored in QueryableStoreProvi

[jira] [Updated] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2023-12-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16046: Fix Version/s: 3.7.0 > Stream Stream Joins fail after restoration with deserializat

[jira] [Commented] (KAFKA-16018) KafkaStreams can go into a zombie state if UncaughtExceptionHandler is specified via the deprecated method

2023-12-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798908#comment-17798908 ] Matthias J. Sax commented on KAFKA-16018: - Timing is a little unclear to be fair

[jira] [Resolved] (KAFKA-15448) Streams StandbyTaskUpdateListener

2023-12-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15448. - Fix Version/s: 3.7.0 Assignee: Colt McNealy Resolution: Fixed > Stre

[jira] [Updated] (KAFKA-15448) Streams StandbyTaskUpdateListener

2023-12-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15448: Issue Type: New Feature (was: Improvement) > Streams StandbyTaskUpdateListe

[jira] [Resolved] (KAFKA-15448) Streams StandbyTaskUpdateListener

2023-12-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15448. - Fix Version/s: 3.7.0 Assignee: Colt McNealy Resolution: Fixed > Stre

[jira] [Updated] (KAFKA-15448) Streams StandbyTaskUpdateListener

2023-12-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15448: Labels: kip (was: needs-kip) > Streams StandbyTaskUpdateListe

[jira] [Updated] (KAFKA-15448) Streams StandbyTaskUpdateListener

2023-12-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15448: Description: KIP-869: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-988%3A

[jira] [Commented] (KAFKA-16018) KafkaStreams can go into a zombie state if UncaughtExceptionHandler is specified via the deprecated method

2023-12-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798137#comment-17798137 ] Matthias J. Sax commented on KAFKA-16018: - Seems this will "auto fix" in 4.0 when

[jira] [Resolved] (KAFKA-9545) Flaky Test `RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenDeleted`

2023-12-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-9545. Resolution: Fixed > Flaky T

[jira] [Resolved] (KAFKA-9545) Flaky Test `RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenDeleted`

2023-12-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-9545. Resolution: Fixed > Flaky T

[jira] [Updated] (KAFKA-15027) Implement rack aware assignment for standby tasks

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15027: Labels: kip (was: ) > Implement rack aware assignment for standby ta

[jira] [Updated] (KAFKA-15024) Add cost function for task/client

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15024: Labels: kip (was: ) > Add cost function for task/cli

[jira] [Updated] (KAFKA-15054) Add configs and logic to decide if rack aware assignment should be enabled

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15054: Labels: kip (was: ) > Add configs and logic to decide if rack aware assignment sho

[jira] [Updated] (KAFKA-15025) Implement min-cost flow without balancing tasks for same subtopology

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15025: Labels: kip (was: ) > Implement min-cost flow without balancing tasks for s

[jira] [Closed] (KAFKA-15026) Implement min-cost flow balancing tasks for same subtopology

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax closed KAFKA-15026. --- > Implement min-cost flow balancing tasks for same subtopol

[jira] [Updated] (KAFKA-15023) Get rack information for source topic partitions for a task

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15023: Labels: kip (was: ) > Get rack information for source topic partitions for a t

[jira] [Updated] (KAFKA-15026) Implement min-cost flow balancing tasks for same subtopology

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15026: Labels: kip (was: ) > Implement min-cost flow balancing tasks for same subtopol

[jira] [Resolved] (KAFKA-15026) Implement min-cost flow balancing tasks for same subtopology

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15026. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement min-cost flow balancing ta

[jira] [Resolved] (KAFKA-15022) Support rack aware task assignment in Kafka streams

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15022. - Fix Version/s: 3.7.0 3.6.0 Resolution: Fixed > Support r

[jira] [Resolved] (KAFKA-15026) Implement min-cost flow balancing tasks for same subtopology

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15026. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement min-cost flow balancing ta

[jira] [Resolved] (KAFKA-15022) Support rack aware task assignment in Kafka streams

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15022. - Fix Version/s: 3.7.0 3.6.0 Resolution: Fixed > Support r

[jira] [Updated] (KAFKA-15348) Range IQs with versioned state stores

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15348: Labels: kip (was: ) > Range IQs with versioned state sto

[jira] [Resolved] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15347. - Fix Version/s: 3.7.0 Resolution: Fixed > Single-Key_multi-timestamp

[jira] [Updated] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15346: Labels: kip (was: ) > Single-Key_single-timestamp IQs with versioned state sto

[jira] [Resolved] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15347. - Fix Version/s: 3.7.0 Resolution: Fixed > Single-Key_multi-timestamp

[jira] [Updated] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-12-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15347: Labels: kip (was: ) > Single-Key_multi-timestamp IQs with versioned state sto

[jira] [Resolved] (KAFKA-15957) ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy broken

2023-12-03 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15957. - Resolution: Fixed

[jira] [Resolved] (KAFKA-15957) ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy broken

2023-12-03 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15957. - Resolution: Fixed

[jira] [Assigned] (KAFKA-15957) ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy broken

2023-12-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15957: --- Assignee: Lucas Brutschy

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

2023-12-01 Thread Matthias J. Sax
der, LittleHorse.dev* On Thu, Nov 30, 2023 at 4:43 PM Matthias J. Sax wrote: parameter is somewhat irrelevant to our use case Sounds like a weird justification to change the KIP. Providing more information is usually better than less, so it seems it won't hurt to just keep it (seems useful in gene

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

2023-11-30 Thread Matthias J. Sax
hu, Oct 26, 2023 at 12:17 PM Matthias J. Sax wrote: Thanks. SGTM. On 10/25/23 4:06 PM, Sophie Blee-Goldman wrote: That all sounds good to me! Thanks for the KIP On Wed, Oct 25, 2023 at 3:47 PM Colt McNealy wrote: Hi Sophie, Matthias, Bruno, and Eduwer— Thanks for your patience as I

[jira] [Resolved] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

2023-11-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15527. - Resolution: Fixed > Add reverseRange and reverseAll query over kv-store in I

[jira] [Resolved] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

2023-11-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15527. - Resolution: Fixed > Add reverseRange and reverseAll query over kv-store in I

[jira] [Created] (KAFKA-15951) MissingSourceTopicException should include topic names

2023-11-30 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15951: --- Summary: MissingSourceTopicException should include topic names Key: KAFKA-15951 URL: https://issues.apache.org/jira/browse/KAFKA-15951 Project: Kafka

[jira] [Created] (KAFKA-15951) MissingSourceTopicException should include topic names

2023-11-30 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15951: --- Summary: MissingSourceTopicException should include topic names Key: KAFKA-15951 URL: https://issues.apache.org/jira/browse/KAFKA-15951 Project: Kafka

[jira] [Resolved] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-11-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15629. - Resolution: Fixed > proposal to introduce IQv2 Query Types: TimestampedKeyQu

[jira] [Resolved] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-11-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15629. - Resolution: Fixed > proposal to introduce IQv2 Query Types: TimestampedKeyQu

[jira] [Updated] (KAFKA-15941) Flaky test: shouldRestoreNullRecord() – org.apache.kafka.streams.integration.RestoreIntegrationTest

2023-11-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15941: Labels: flaky-test (was: ) > Flaky test: shouldRestoreNullRec

[jira] [Updated] (KAFKA-15941) Flaky test: shouldRestoreNullRecord() – org.apache.kafka.streams.integration.RestoreIntegrationTest

2023-11-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15941: Component/s: streams unit tests > Flaky test: shouldRestoreNullRec

[jira] [Updated] (KAFKA-15944) Flaky test - verifyStore[cache=false, log=true, supplier=ROCKS_KV, kind=DSL] – org.apache.kafka.streams.integration.PositionRestartIntegrationTest

2023-11-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15944: Component/s: unit tests > Flaky test - verifyStore[cache=false, log=true, suppl

Re: Having a Hard Time Unsubscribing

2023-11-28 Thread Matthias J. Sax
When you send an email to unsubscribe..., there should be a response from the bot. Check your spam folder. You need to ack the unsubscribe by replying the the response email the bot sent. HTH. -Matthias On 11/28/23 6:10 AM, Dasun Nirmitha wrote: Hello Kafka devs, I'm aware that it's a

[jira] [Commented] (KAFKA-15768) StateQueryResult#getOnlyPartitionResult should not throw for FailedQueryResult

2023-11-28 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790835#comment-17790835 ] Matthias J. Sax commented on KAFKA-15768: - Thanks for the details [~vvcephei]! {quote}In other

[jira] [Updated] (KAFKA-15896) Flaky test: shouldQuerySpecificStalePartitionStores() – org.apache.kafka.streams.integration.StoreQueryIntegrationTest

2023-11-27 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15896: Labels: flaky-test (was: ) > Flaky test: shouldQuerySpecificStalePartitionSto

[jira] [Updated] (KAFKA-15896) Flaky test: shouldQuerySpecificStalePartitionStores() – org.apache.kafka.streams.integration.StoreQueryIntegrationTest

2023-11-27 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15896: Component/s: streams unit tests > Flaky t

[jira] [Updated] (KAFKA-8575) Investigate removing EAGER protocol & cleaning up task suspension in Streams rebalancing

2023-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-8575: --- Fix Version/s: 4.0.0 > Investigate removing EAGER protocol & cleaning up task sus

[jira] [Updated] (KAFKA-8088) Deprecate `WindowStoreIterator` interface

2023-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-8088: --- Fix Version/s: 4.0.0 > Deprecate `WindowStoreIterator` interf

[jira] [Updated] (KAFKA-12281) Deprecate org.apache.kafka.streams.errors.BrokerNotFoundException

2023-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12281: Labels: beginner needs-kip newbie (was: needs-kip) > Deprec

[jira] [Updated] (KAFKA-15768) StateQueryResult#getOnlyPartitionResult should not throw for FailedQueryResult

2023-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15768: Description: Calling `StateQueryResult#getOnlyPartitionResult` crashes with an incorrect

[jira] [Updated] (KAFKA-15768) StateQueryResult#getOnlyPartitionResult should not throw for FailedQueryResult

2023-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15768: Description: Calling `StateQueryResult#getOnlyPartitionResult` crashes with an incorrect

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-21 Thread Matthias J. Sax
Thanks Sophie. Overall I agree with you. I think 50% is too high as a general rule, and I believe something like 30% might be more appropriate (going lower, given the infrastructure at hand might become too aggressive)? The difficult part about a policy like this is, that we don't really

Re: [VOTE] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-21 Thread Matthias J. Sax
Matthias J. Sax wrote: Adding an enum is a good idea! Wondering if `UNORDERED` is the best name? Want to avoid bike shedding, just asking. We could also use `UNDEFINED` / `UNSPECIFIED` / `NONE` / `ANY` ? In the end, the result _might_ be ordered, we just don't guarantee any order. -Matthias

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-21 Thread Matthias J. Sax
returns an Optional. Cheers, Alieh On Mon, Nov 20, 2023 at 7:38 PM Matthias J. Sax wrote: I think we should also discuss a little more about `validTo()` method? Given that "latest" version does not have a valid-to TS, should we change the return type to `Optional` and ret

Re: [VOTE] KIP-1000: List Client Metrics Configuration Resources

2023-11-20 Thread Matthias J. Sax
+1 (binding) On 11/20/23 1:42 PM, Jason Gustafson wrote: +1 Thanks for the KIP! On Mon, Nov 20, 2023 at 9:31 AM Jun Rao wrote: Hi, Andrew, Thanks for the KIP. +1 Jun On Thu, Nov 16, 2023 at 9:12 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: Hi Apoorv, Thanks for your

Re: [VOTE] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-20 Thread Matthias J. Sax
Adding an enum is a good idea! Wondering if `UNORDERED` is the best name? Want to avoid bike shedding, just asking. We could also use `UNDEFINED` / `UNSPECIFIED` / `NONE` / `ANY` ? In the end, the result _might_ be ordered, we just don't guarantee any order. -Matthias On 11/20/23 9:17

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-20 Thread Matthias J. Sax
cts of the KIP are quite clear now and we have come to a consensus about them, I think it 's time to vote ;-) I look forward to your votes. Thanks a lot. Cheers, Alieh On Fri, Nov 17, 2023 at 2:27 AM Matthias J. Sax wrote: Thanks, Alieh. Overall SGTM. About `validTo` -- wondering if we should

Re: [DISCUSS] KIP-997 Support fetch(fromKey, toKey, from, to) to WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery

2023-11-16 Thread Matthias J. Sax
Thanks for the KIP. Given how `WindowRangeQuery` works right now, it's really time to improve it. 1) Agree. It's not clear what will be added right now. I think we should deprecate existing `getKey()` w/o an actually replacement? For `getFromKey` and `getToKey` we should actually be

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-11-16 Thread Matthias J. Sax
the no-order-guaranteed option does not cost us anything and it keeps the IQv2 interface flexible. I assume we want to drop the Evolving annotation at some point. Sorry for not having brought this up in the discussion about KIP-985. Best, Bruno On 11/15/23 6:56 AM, Matthias J. Sax wrote: Just

<    1   2   3   4   5   6   7   8   9   10   >