[jira] [Updated] (KAFKA-14076) Fix issues with KafkaStreams.CloseOptions

2022-07-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14076: Priority: Blocker (was: Major) > Fix issues with KafkaStreams.CloseOpti

[jira] [Updated] (KAFKA-14076) Fix issues with KafkaStreams.CloseOptions

2022-07-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14076: Fix Version/s: 3.3.0 > Fix issues with KafkaStreams.CloseOpti

[jira] [Assigned] (KAFKA-14076) Fix issues with KafkaStreams.CloseOptions

2022-07-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-14076: --- Assignee: Jim Hughes > Fix issues with KafkaStreams.CloseOpti

[jira] [Commented] (KAFKA-12887) Do not trigger user-customized ExceptionalHandler for RTE

2022-07-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17568659#comment-17568659 ] Matthias J. Sax commented on KAFKA-12887: - This feature broke some stuff,

[jira] [Updated] (KAFKA-13152) Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-07-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13152: Labels: kip (was: needs-kip) > Replace "buffered.records.per.partiti

[jira] [Reopened] (KAFKA-13152) Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-07-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-13152: - > Replace "buffered.records.per.partition" with "inpu

[jira] [Updated] (KAFKA-13152) Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-07-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13152: Fix Version/s: (was: 3.3.0) > Replace "buffered.records.per.partiti

[jira] [Reopened] (KAFKA-13152) Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-07-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-13152: - > Replace "buffered.records.per.partition" with "inpu

Re: [DISCUSS] KIP-837 Allow MultiCasting a Result Record.

2022-07-07 Thread Matthias J. Sax
ing in this new class devoid of any StreamPartitioner. I just thought it's cleaner to keep it this way as StreamPartitioner already dpes the partitioning. Let me know what you think. Thanks! Sagar. On Wed, Jun 29, 2022 at 5:34 AM Matthias J. Sax wrote: Thanks for the KIP. Overall a good addi

Re: [DISCUSS] KIP-842: Add richer group offset reset mechanisms

2022-07-07 Thread Matthias J. Sax
Jun 29, 2022 at 4:57 AM David Jacot wrote: Thanks for the KIP. I read it and I am also worried by the complexity of the new configurations. They are not easy to grasp. I need to digest it a bit more, I think. Best, David Le mer. 29 juin 2022 à 02:25, Matthias J. Sax a écrit : Thanks for the

[jira] [Commented] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2022-07-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563510#comment-17563510 ] Matthias J. Sax commented on KAFKA-14049: - {quote}Null Values in the Stream

[jira] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2022-07-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049 ] Matthias J. Sax deleted comment on KAFKA-14049: - was (Author: mjsax): Sounds like a duplicate to https://issues.apache.org/jira/browse/KAFKA-12317 ? > Relax Non Null Requirement

[jira] [Commented] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2022-07-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563509#comment-17563509 ] Matthias J. Sax commented on KAFKA-14049: - Sounds like a duplicate to h

[jira] [Updated] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-06-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13936: Fix Version/s: 3.3.0 > Invalid consumer lag when monitoring from a kafka stre

[jira] [Assigned] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-06-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-13936: --- Assignee: Prashanth Joseph Babu > Invalid consumer lag when monitoring fro

[jira] [Resolved] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-06-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13936. - Resolution: Fixed > Invalid consumer lag when monitoring from a kafka stre

[jira] [Resolved] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-06-30 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13936. - Resolution: Fixed > Invalid consumer lag when monitoring from a kafka stre

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-06-30 Thread Matthias J. Sax
1 +1(binding) On Thu, Mar 24, 2022 at 5:13 PM Matthias J. Sax wrote: +1 (binding) On 3/24/22 1:52 PM, Hao Li wrote: I hit reply on my phone in the mail app and changed the title and text hoping it will start a new thread. Apparently it doesn't work... On Thu, Mar 24, 2022 at 12:36 PM

[jira] [Commented] (KAFKA-10493) KTable out-of-order updates are not being ignored

2022-06-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17560622#comment-17560622 ] Matthias J. Sax commented on KAFKA-10493: - There is nothing official yet,

Re: [DISCUSS] KIP-842: Add richer group offset reset mechanisms

2022-06-28 Thread Matthias J. Sax
Thanks for the KIP. I don't think I fully digested the proposal yet, but my first reaction is: this is quite complicated. Frankly, I am worried about complexity and usability. Especially the option `safe_latest` is a "weird" one IMHO, and `nearest` is even more complex. The problem at hand

Re: [DISCUSS] KIP-837 Allow MultiCasting a Result Record.

2022-06-28 Thread Matthias J. Sax
Thanks for the KIP. Overall a good addition. I am actually not sure if we need to add a new class? From my understanding, if there is exactly one abstract method, the interface is still functional? Thus, we could add a new method to `StreamsPartitioner` with a default implementation (that call

[jira] [Commented] (KAFKA-13531) Flaky test NamedTopologyIntegrationTest

2022-06-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555781#comment-17555781 ] Matthias J. Sax commented on KAFKA-13531: - Could be. We usually also repor

[jira] [Resolved] (KAFKA-13531) Flaky test NamedTopologyIntegrationTest

2022-06-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13531. - Resolution: Cannot Reproduce > Flaky test NamedTopologyIntegrationT

[jira] [Resolved] (KAFKA-13531) Flaky test NamedTopologyIntegrationTest

2022-06-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13531. - Resolution: Cannot Reproduce > Flaky test NamedTopologyIntegrationT

[jira] [Commented] (KAFKA-13939) Memory Leak When Logging Is Disabled In InMemoryTimeOrderedKeyValueBuffer

2022-06-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554742#comment-17554742 ] Matthias J. Sax commented on KAFKA-13939: - Thanks for the PR. I added yo

[jira] [Assigned] (KAFKA-13939) Memory Leak When Logging Is Disabled In InMemoryTimeOrderedKeyValueBuffer

2022-06-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-13939: --- Assignee: Jackson Newhouse > Memory Leak When Logging Is Disabled

Re: How it is safe to break message ordering but not idempotency after getting an OutOfOrderSequenceException?

2022-06-15 Thread Matthias J. Sax
messages, I mean if both batches include a message being written to topic A, could messages from batch with sn X+1 end up being persisted with an offset lesser than the ones from the batch with sn X? Does this question make sense? El mar, 7 jun 2022 a las 16:13, Matthias J. Sax () escribió: Yes, t

[jira] [Commented] (SUREFIRE-2076) BufferOverflowException when encoding message with null runMode

2022-06-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/SUREFIRE-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554376#comment-17554376 ] Matthias J. Sax commented on SUREFIRE-2076: --- Thanks for the update!

[jira] [Commented] (KAFKA-13963) Topology Description ignores context.forward

2022-06-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551333#comment-17551333 ] Matthias J. Sax commented on KAFKA-13963: - {quote}Is it worth updating the

[jira] [Commented] (KAFKA-13963) Topology Description ignores context.forward

2022-06-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551271#comment-17551271 ] Matthias J. Sax commented on KAFKA-13963: - TopologyDescription only descr

Re: How it is safe to break message ordering but not idempotency after getting an OutOfOrderSequenceException?

2022-06-07 Thread Matthias J. Sax
Yes, the broker de-dupes using the sequence number. But for example, if a sequence number is skipped, you could get this exception: the current batch of messages cannot be appended to the log, as one batch is missing, and the producer would need to re-send the previous/missing batch with lower

Re: Newbie how to get key/value pojo out of a stream?

2022-06-07 Thread Matthias J. Sax
`enable.auto.commit` is a Consumer config and does not apply to Kafka Stream. In Kafka Streams, you basically always have auto commit enabled, and you can control how frequently commits happen via `commit.interval.ms`. Also on `close()` Kafka Streams would commit offsets. -Matthias On 5/31

[jira] [Commented] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-06-03 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17546407#comment-17546407 ] Matthias J. Sax commented on KAFKA-13936: - The docs are in the same reposi

[jira] [Commented] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-06-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545686#comment-17545686 ] Matthias J. Sax commented on KAFKA-13936: - Might be worth to document for

[jira] [Updated] (KAFKA-13945) Add task-level metrics to Streams for bytes/records Produced

2022-05-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13945: Description: KIP-846: [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId

[jira] [Commented] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-05-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544647#comment-17544647 ] Matthias J. Sax commented on KAFKA-13936: - As mentioned above, offsets ar

[jira] [Commented] (KAFKA-13939) Memory Leak When Logging Is Disabled In InMemoryTimeOrderedKeyValueBuffer

2022-05-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544645#comment-17544645 ] Matthias J. Sax commented on KAFKA-13939: - Thanks for reporting this i

[jira] [Updated] (KAFKA-13939) Memory Leak When Logging Is Disabled In InMemoryTimeOrderedKeyValueBuffer

2022-05-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13939: Component/s: streams > Memory Leak When Logging Is Disabled

[jira] [Updated] (KAFKA-13939) Memory Leak When Logging Is Disabled In InMemoryTimeOrderedKeyValueBuffer

2022-05-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13939: Priority: Blocker (was: Major) > Memory Leak When Logging Is Disabled

[jira] [Commented] (KAFKA-13936) Invalid consumer lag when monitoring from a kafka streams application

2022-05-25 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17542359#comment-17542359 ] Matthias J. Sax commented on KAFKA-13936: - > we see in the kafka UI  T

Re: kafka stream - sliding window - getting unexpected output

2022-05-20 Thread Matthias J. Sax
ane wrote: @Matthias J. Sax / All Have added below line : .suppress(Suppressed.untilWindowCloses(Suppressed.BufferConfig.unbounded())) Here is the output : (for uuid (*2cbef750-325b-4a2f-ac39-b2c23fa0313f)*, expecting single output but that is not the case here. Which 1 is the final output from tho

Re: Are timestamps available for records stored in Kafka Streams state stores?

2022-05-20 Thread Matthias J. Sax
Added. Feel free to update wiki pages with smaller details like this directly. -Matthias On 5/19/22 2:27 PM, James Cheng wrote: Thanks Guozhang! Based on your comment, I searched through the repo and found the associated pull requests and JIRAs. It looks like most of the support was added i

[jira] [Commented] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-05-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540294#comment-17540294 ] Matthias J. Sax commented on KAFKA-13817: - Thanks for the PR! I put it int

[jira] [Commented] (KAFKA-13913) Provide builders for KafkaProducer/KafkaConsumer and KafkaStreams

2022-05-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539718#comment-17539718 ] Matthias J. Sax commented on KAFKA-13913: - There was some discussion about

[jira] [Updated] (KAFKA-13913) Provide builders for KafkaProducer/KafkaConsumer and KafkaStreams

2022-05-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13913: Labels: kip (was: ) > Provide builders for KafkaProducer/KafkaConsumer and KafkaStre

[jira] [Updated] (KAFKA-13913) Provide builders for KafkaProducer/KafkaConsumer and KafkaStreams

2022-05-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13913: Component/s: streams > Provide builders for KafkaProducer/KafkaConsumer and KafkaStre

Re: kafka stream - sliding window - getting unexpected output

2022-05-18 Thread Matthias J. Sax
Emitting intermediate result is by-design. If you don't want to get intermediate result, you can add `suppress()` after the aggregation and configure it to only "emit on window close". -Matthias On 5/17/22 3:20 AM, Shankar Mane wrote: Hi All, Our use case is to use sliding window. (for e.g.

[jira] [Commented] (KAFKA-6520) When a Kafka Stream can't communicate with the server, it's Status stays RUNNING

2022-05-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537860#comment-17537860 ] Matthias J. Sax commented on KAFKA-6520: It's still an open item –

[jira] [Commented] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-05-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17536926#comment-17536926 ] Matthias J. Sax commented on KAFKA-13817: - > as it will eventually self-

[jira] [Commented] (SUREFIRE-2076) BufferOverflowException when encoding message with null runMode

2022-05-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/SUREFIRE-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17535072#comment-17535072 ] Matthias J. Sax commented on SUREFIRE-2076: --- [~tibordigana] [~zoltan.

[jira] [Resolved] (KAFKA-7527) Enable Dependency Injection for Kafka Streams handlers (KIP-378)

2022-05-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7527. Resolution: Fixed > Enable Dependency Injection for Kafka Streams handlers (KIP-

[jira] [Resolved] (KAFKA-7527) Enable Dependency Injection for Kafka Streams handlers (KIP-378)

2022-05-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7527. Resolution: Fixed > Enable Dependency Injection for Kafka Streams handlers (KIP-

Re: [VOTE] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-10 Thread Matthias J. Sax
I had one minor question on the discuss thread. It's mainly about clarifying and document the user contract. I am fine either way. +1 (binding) -Matthias On 5/10/22 12:32 PM, Sophie Blee-Goldman wrote: Thanks for the KIP! +1 (binding) On Tue, May 10, 2022, 12:24 PM Bruno Cadonna wrote: T

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-10 Thread Matthias J. Sax
ably better to also commit the processing before we pause it. Guozhang On Fri, May 6, 2022 at 6:12 PM Jim Hughes wrote: Hi Matthias, Since the only thing which will be paused is processing the topology, I think we can let commits happen naturally. Good point about getting the paused stat

Re: [DISCUSS] KIP-834: Pause / Resume KafkaStreams Topologies

2022-05-06 Thread Matthias J. Sax
I think it's tricky to propagate a pauseAll() via the rebalance protocol. New members joining the group would need to get paused, too? Could there be weird race conditions with overlapping pauseAll() and resumeAll() calls on different instanced while there could be a errors / network partitions

[jira] [Commented] (KAFKA-8769) Consider computing stream time independently per key

2022-05-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17532427#comment-17532427 ] Matthias J. Sax commented on KAFKA-8769: Just a "random" comment,

[jira] [Updated] (KAFKA-13873) Add ability to Pause / Resume KafkaStreams Topologies

2022-05-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13873: Description: In order to reduce resources used or modify data pipelines, users may want

[jira] [Updated] (KAFKA-13873) Add ability to Pause / Resume KafkaStreams Topologies

2022-05-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13873: Component/s: streams > Add ability to Pause / Resume KafkaStreams Topolog

[jira] [Updated] (KAFKA-13873) Add ability to Pause / Resume KafkaStreams Topologies

2022-05-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13873: Labels: kip (was: ) > Add ability to Pause / Resume KafkaStreams Topolog

[jira] [Updated] (KAFKA-13870) support both Suppressed untilTimeLimit and maxBytes without using emitEarlyWhenFull()

2022-05-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13870: Labels: needs-kip (was: ) > support both Suppressed untilTimeLimit and maxBytes with

[jira] [Commented] (KAFKA-13870) support both Suppressed untilTimeLimit and maxBytes without using emitEarlyWhenFull()

2022-05-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531836#comment-17531836 ] Matthias J. Sax commented on KAFKA-13870: - I think you mix up two conc

[jira] [Updated] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with spurious record fix

2022-05-03 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13813: Summary: left/outer joins can wait indefinitely for emitted record with spurious record

[jira] [Created] (KAFKA-13866) Support more advanced time retention policies

2022-05-02 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13866: --- Summary: Support more advanced time retention policies Key: KAFKA-13866 URL: https://issues.apache.org/jira/browse/KAFKA-13866 Project: Kafka Issue

[jira] [Created] (KAFKA-13866) Support more advanced time retention policies

2022-05-02 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13866: --- Summary: Support more advanced time retention policies Key: KAFKA-13866 URL: https://issues.apache.org/jira/browse/KAFKA-13866 Project: Kafka Issue

[jira] [Commented] (SUREFIRE-2076) BufferOverflowException when encoding message with null runMode

2022-04-28 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/SUREFIRE-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17529670#comment-17529670 ] Matthias J. Sax commented on SUREFIRE-2076: --- Seems we are hitting

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2022-04-27 Thread Matthias J. Sax
king it up. Levani On 23. Apr 2022, at 02:43, Matthias J. Sax wrote: Ivan, are you still interested in this KIP? I think it would be a good addition. -Matthias On 8/16/21 5:30 PM, Matthias J. Sax wrote: Your point about the IQ problem is an interesting one. I missed the point that the "

[jira] [Commented] (KAFKA-13647) RocksDb metrics 'number-open-files' is not correct

2022-04-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528288#comment-17528288 ] Matthias J. Sax commented on KAFKA-13647: - Should we close this ticket?

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2022-04-22 Thread Matthias J. Sax
Ivan, are you still interested in this KIP? I think it would be a good addition. -Matthias On 8/16/21 5:30 PM, Matthias J. Sax wrote: Your point about the IQ problem is an interesting one. I missed the point that the "new key" would be a "superkey", and thus, it should a

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-04-22 Thread Matthias J. Sax
ml#asList-T...- [3] - https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html#toArray-T:A- Cheers, Chris On Tue, Mar 29, 2022 at 11:14 PM Matthias J. Sax wrote: My understand was, that the original proposal was to have: merge(KStream stream); merge(KStream... streams); Mayb

[jira] [Comment Edited] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522502#comment-17522502 ] Matthias J. Sax edited comment on KAFKA-12909 at 4/21/22 9:1

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17526086#comment-17526086 ] Matthias J. Sax commented on KAFKA-12909: - Ah sorry. That was my bad... F

[jira] [Updated] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13842: Description: Kafka Streams follows a continuous refinement model for aggregation. For

[jira] [Created] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13842: --- Summary: Add per-aggregation step before repartitioning Key: KAFKA-13842 URL: https://issues.apache.org/jira/browse/KAFKA-13842 Project: Kafka Issue

[jira] [Created] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13842: --- Summary: Add per-aggregation step before repartitioning Key: KAFKA-13842 URL: https://issues.apache.org/jira/browse/KAFKA-13842 Project: Kafka Issue

[jira] [Comment Edited] (KAFKA-13787) Failed to delete state store directory for it is not empty

2022-04-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516068#comment-17516068 ] Matthias J. Sax edited comment on KAFKA-13787 at 4/15/22 8:5

[jira] [Comment Edited] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522502#comment-17522502 ] Matthias J. Sax edited comment on KAFKA-12909 at 4/14/22 7:1

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522502#comment-17522502 ] Matthias J. Sax commented on KAFKA-12909: - {quote}It makes sense but i

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17521985#comment-17521985 ] Matthias J. Sax commented on KAFKA-12909: - This ticket is about left/outer

[jira] [Commented] (KAFKA-8318) Session Window Aggregations generate an extra tombstone

2022-04-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17521866#comment-17521866 ] Matthias J. Sax commented on KAFKA-8318: It seem [~ipasynkov] does no wor

[jira] [Updated] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13817: Labels: beginner newbie (was: ) > Schedule nextTimeToEmit to system time every t

[jira] [Updated] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13817: Component/s: streams > Schedule nextTimeToEmit to system time every time instead of j

[jira] [Commented] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17521243#comment-17521243 ] Matthias J. Sax commented on KAFKA-13813: - The change is intentionally

[jira] [Commented] (KAFKA-13820) Add debug-level logs to explain why a store is filtered out during interactive query

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17521234#comment-17521234 ] Matthias J. Sax commented on KAFKA-13820: - Sounds related to [h

[jira] [Updated] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13813: Issue Type: Improvement (was: Bug) > left/outer joins can wait indefinitely for emit

[jira] [Updated] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13813: Component/s: streams > left/outer joins can wait indefinitely for emitted record w

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519199#comment-17519199 ] Matthias J. Sax commented on KAFKA-7509: Thanks for staying on top of this.

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519175#comment-17519175 ] Matthias J. Sax commented on KAFKA-12909: - Not that _inner_ join result

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-04-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516979#comment-17516979 ] Matthias J. Sax commented on KAFKA-7509: Thanks for following up. – I bri

[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty

2022-04-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516068#comment-17516068 ] Matthias J. Sax commented on KAFKA-13787: - `StateDirectory` should have

Re: [VOTE] KIP-813 Shared State Stores

2022-04-01 Thread Matthias J. Sax
+1 (binding) On 4/1/22 6:47 AM, John Roesler wrote: Thanks for the KIP, Daan! I’m +1 (binding) -John On Tue, Mar 29, 2022, at 06:01, Daan Gertis wrote: I would like to start a vote on this one: https://cwiki.apache.org/confluence/display/KAFKA/KIP-813%3A+Shareable+State+Stores Cheers, D.

[DISCUSS] KIP-0422: Add Record Footers

2022-04-01 Thread Matthias J. Sax
Hi, we added record header support to Kafka via KIP-82 many years ago. I think it's time to complement this feature with record footers. Looking forward to your feedback. https://tinyurl.com/43jubbaj -Matthias

[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty

2022-03-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515648#comment-17515648 ] Matthias J. Sax commented on KAFKA-13787: - What file system are you u

[jira] [Updated] (KAFKA-13785) Support emit final result for windowed aggregation

2022-03-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13785: Labels: kip (was: ) > Support emit final result for windowed aggregat

[jira] [Updated] (KAFKA-13785) Support emit final result for windowed aggregation

2022-03-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13785: Component/s: streams > Support emit final result for windowed aggregat

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-03-29 Thread Matthias J. Sax
ng a variant that accepts a Named parameter? Might be tricky to accommodate since variadic arguments have to be last. [1] - https://stackoverflow.com/a/48850722 Cheers, Chris On Mon, Mar 28, 2022 at 11:46 PM Matthias J. Sax wrote: I think Java does not allow to have both overloads, bec

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-03-28 Thread Matthias J. Sax
a single n-ary merge is what you're really modelling. Regards, Nick On Thu, 24 Mar 2022 at 21:24, Matthias J. Sax wrote: Thanks for proposing this KIP. I feel a little bit torn by the idea. In general, we try to keep the surface area small, and only add APIs that delivery (significant) v

[jira] [Comment Edited] (KAFKA-4609) KTable/KTable join followed by groupBy and aggregate/count can result in duplicated results

2022-03-25 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959455#comment-16959455 ] Matthias J. Sax edited comment on KAFKA-4609 at 3/25/22, 5:3

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-03-24 Thread Matthias J. Sax
Thanks for proposing this KIP. I feel a little bit torn by the idea. In general, we try to keep the surface area small, and only add APIs that delivery (significant) value. It seems the current proposal is more or less about syntactic sugar, what can still be valuable, but I am not really sur

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Matthias J. Sax
+1 (binding) On 3/24/22 1:52 PM, Hao Li wrote: I hit reply on my phone in the mail app and changed the title and text hoping it will start a new thread. Apparently it doesn't work... On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna wrote: Hi Hao, Actually, this is the VOTE thread. Do you use

[jira] [Commented] (KAFKA-13542) Utilize the new Consumer#enforceRebalance(reason) API in Streams

2022-03-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512018#comment-17512018 ] Matthias J. Sax commented on KAFKA-13542: - SGTM. > Utilize the new C

<    13   14   15   16   17   18   19   20   21   22   >