[jira] [Created] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13842: --- Summary: Add per-aggregation step before repartitioning Key: KAFKA-13842 URL: https://issues.apache.org/jira/browse/KAFKA-13842 Project: Kafka Issue

[jira] [Created] (KAFKA-13842) Add per-aggregation step before repartitioning

2022-04-20 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-13842: --- Summary: Add per-aggregation step before repartitioning Key: KAFKA-13842 URL: https://issues.apache.org/jira/browse/KAFKA-13842 Project: Kafka Issue

[jira] [Comment Edited] (KAFKA-13787) Failed to delete state store directory for it is not empty

2022-04-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516068#comment-17516068 ] Matthias J. Sax edited comment on KAFKA-13787 at 4/15/22 8:56 PM

[jira] [Comment Edited] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522502#comment-17522502 ] Matthias J. Sax edited comment on KAFKA-12909 at 4/14/22 7:12 PM

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522502#comment-17522502 ] Matthias J. Sax commented on KAFKA-12909: - {quote}It makes sense but it is still kind of hard

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521985#comment-17521985 ] Matthias J. Sax commented on KAFKA-12909: - This ticket is about left/outer join in particular

[jira] [Commented] (KAFKA-8318) Session Window Aggregations generate an extra tombstone

2022-04-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521866#comment-17521866 ] Matthias J. Sax commented on KAFKA-8318: It seem [~ipasynkov] does no work on this any longer

[jira] [Updated] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13817: Labels: beginner newbie (was: ) > Schedule nextTimeToEmit to system time every t

[jira] [Updated] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13817: Component/s: streams > Schedule nextTimeToEmit to system time every time instead of j

[jira] [Commented] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521243#comment-17521243 ] Matthias J. Sax commented on KAFKA-13813: - The change is intentionally, as users wanted to get

[jira] [Commented] (KAFKA-13820) Add debug-level logs to explain why a store is filtered out during interactive query

2022-04-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521234#comment-17521234 ] Matthias J. Sax commented on KAFKA-13820: - Sounds related to [https://cwiki.apache.org

[jira] [Updated] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13813: Issue Type: Improvement (was: Bug) > left/outer joins can wait indefinitely for emit

[jira] [Updated] (KAFKA-13813) left/outer joins can wait indefinitely for emitted record with suprious record fix

2022-04-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13813: Component/s: streams > left/outer joins can wait indefinitely for emitted rec

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519199#comment-17519199 ] Matthias J. Sax commented on KAFKA-7509: Thanks for staying on top of this. The filtering code

[jira] [Commented] (KAFKA-12909) Allow users to opt-into spurious left/outer stream-stream join improvement

2022-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519175#comment-17519175 ] Matthias J. Sax commented on KAFKA-12909: - Not that _inner_ join result are still instant, even

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-04-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516979#comment-17516979 ] Matthias J. Sax commented on KAFKA-7509: Thanks for following up. – I briefly took a look. My

[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty

2022-04-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17516068#comment-17516068 ] Matthias J. Sax commented on KAFKA-13787: - `StateDirectory` should have all relevant code

Re: [VOTE] KIP-813 Shared State Stores

2022-04-01 Thread Matthias J. Sax
+1 (binding) On 4/1/22 6:47 AM, John Roesler wrote: Thanks for the KIP, Daan! I’m +1 (binding) -John On Tue, Mar 29, 2022, at 06:01, Daan Gertis wrote: I would like to start a vote on this one: https://cwiki.apache.org/confluence/display/KAFKA/KIP-813%3A+Shareable+State+Stores Cheers, D.

[DISCUSS] KIP-0422: Add Record Footers

2022-04-01 Thread Matthias J. Sax
Hi, we added record header support to Kafka via KIP-82 many years ago. I think it's time to complement this feature with record footers. Looking forward to your feedback. https://tinyurl.com/43jubbaj -Matthias

[jira] [Commented] (KAFKA-13787) Failed to delete state store directory for it is not empty

2022-03-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515648#comment-17515648 ] Matthias J. Sax commented on KAFKA-13787: - What file system are you using? > Failed to del

[jira] [Updated] (KAFKA-13785) Support emit final result for windowed aggregation

2022-03-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13785: Labels: kip (was: ) > Support emit final result for windowed aggregat

[jira] [Updated] (KAFKA-13785) Support emit final result for windowed aggregation

2022-03-31 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13785: Component/s: streams > Support emit final result for windowed aggregat

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-03-29 Thread Matthias J. Sax
a Named parameter? Might be tricky to accommodate since variadic arguments have to be last. [1] - https://stackoverflow.com/a/48850722 Cheers, Chris On Mon, Mar 28, 2022 at 11:46 PM Matthias J. Sax wrote: I think Java does not allow to have both overloads, because it would result in a

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-03-28 Thread Matthias J. Sax
-ary merge is what you're really modelling. Regards, Nick On Thu, 24 Mar 2022 at 21:24, Matthias J. Sax wrote: Thanks for proposing this KIP. I feel a little bit torn by the idea. In general, we try to keep the surface area small, and only add APIs that delivery (significant) value. It seems

[jira] [Comment Edited] (KAFKA-4609) KTable/KTable join followed by groupBy and aggregate/count can result in duplicated results

2022-03-25 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959455#comment-16959455 ] Matthias J. Sax edited comment on KAFKA-4609 at 3/25/22, 5:31 PM

Re: [DISCUSS] KIP-819: Merge multiple KStreams in one operation

2022-03-24 Thread Matthias J. Sax
Thanks for proposing this KIP. I feel a little bit torn by the idea. In general, we try to keep the surface area small, and only add APIs that delivery (significant) value. It seems the current proposal is more or less about syntactic sugar, what can still be valuable, but I am not really

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Matthias J. Sax
+1 (binding) On 3/24/22 1:52 PM, Hao Li wrote: I hit reply on my phone in the mail app and changed the title and text hoping it will start a new thread. Apparently it doesn't work... On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna wrote: Hi Hao, Actually, this is the VOTE thread. Do you

[jira] [Commented] (KAFKA-13542) Utilize the new Consumer#enforceRebalance(reason) API in Streams

2022-03-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512018#comment-17512018 ] Matthias J. Sax commented on KAFKA-13542: - SGTM. > Utilize the new Consumer#enforceRebala

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-23 Thread Matthias J. Sax
d to other non-windowed operators in the future. 2. To be consistent with other control class names, I feel maybe we can name it "Emitted", not "EmitConfig". 3. Following the first comment, I think we can have the static constructor names as "onWindowClose" an

[jira] [Commented] (KAFKA-13542) Utilize the new Consumer#enforceRebalance(reason) API in Streams

2022-03-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511537#comment-17511537 ] Matthias J. Sax commented on KAFKA-13542: - Curious: did we figure out why the patch resulted

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-03-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17510997#comment-17510997 ] Matthias J. Sax commented on KAFKA-7509: Thanks for the detailed explanation. I don't know

Re: [VOTE} KIP-796: Interactive Query v2

2022-03-22 Thread Matthias J. Sax
The old API IQv1 is not deprecated yet, so I don't see a reason to revert. We might not want to "advertise" the IQv2 in the release announcement though if it's not complete and unstable right now. We might also not want to mention it in the docs? Not sure if there was already docs PR. If

[jira] [Commented] (KAFKA-13739) Sliding window without grace not working

2022-03-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17510043#comment-17510043 ] Matthias J. Sax commented on KAFKA-13739: - Thank you! – Let us know if you need any help

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-03-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509095#comment-17509095 ] Matthias J. Sax commented on KAFKA-7509: {quote} # Breaking backwards compatibility for a lot

[jira] [Commented] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

2022-03-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507833#comment-17507833 ] Matthias J. Sax commented on KAFKA-7509: Personally, I think the best way would be to tackle

[jira] [Updated] (KAFKA-13739) Sliding window without grace not working

2022-03-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13739: Labels: beginner newbie (was: ) > Sliding window without grace not work

[jira] [Commented] (KAFKA-13739) Sliding window without grace not working

2022-03-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507825#comment-17507825 ] Matthias J. Sax commented on KAFKA-13739: - Thanks for chiming in [~lct45] \ {quote}_if_ we want

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-16 Thread Matthias J. Sax
but how we output to `KTable`. That's why I feel option 1 makes more sense as it applies to `TimeWindowedKStream`. But I'm also OK with option 2. Hao On Tue, Mar 15, 2022 at 4:48 PM Matthias J. Sax wrote: Thanks for the KIP. A general comment: it seem that we won't need any new

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-15 Thread Matthias J. Sax
Thanks for the KIP. A general comment: it seem that we won't need any new `allowedLateness` parameter because the grace-period is defined on the window itself already? (On the other hand, if I think about it once more, maybe the `grace-period` is actually not a property of the window but a

Re: [VOTE] KIP-820: Extend KStream process with new Processor API

2022-03-15 Thread Matthias J. Sax
+1 (binding) Thanks for pushing this through. Was a difficult discussion! -Matthias On 3/15/22 10:01 AM, John Roesler wrote: Thanks for the update, Jorge! I’m still +1 (binding) Thanks, John On Thu, Feb 17, 2022, at 12:57, Guozhang Wang wrote: Thanks Jorge, overall looks good to me.

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-03-15 Thread Matthias J. Sax
and Jon on going forward with the new `FixedKeyRecord` approach. The KIP has been updated accordingly. Feel free to add your vote or amend on the vote thread if needed. Cheers, On Mon, 7 Mar 2022 at 21:57, Matthias J. Sax wrote: I think it's ok that we cannot prevent users from mutating

[jira] [Updated] (KAFKA-13739) Sliding window without grace not working

2022-03-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13739: Priority: Minor (was: Major) > Sliding window without grace not work

[jira] [Commented] (KAFKA-13739) Sliding window without grace not working

2022-03-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507261#comment-17507261 ] Matthias J. Sax commented on KAFKA-13739: - Thanks for reporting this. Good catch. I think you

Re: [VOTE] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2022-03-15 Thread Matthias J. Sax
0:22, Sophie Blee-Goldman wrote: Thanks all, +1 (non-binding) Cheers, Sophie On Wed, Jul 8, 2020 at 4:02 AM Bruno Cadonna wrote: Thanks Will and Piotr, +1 (non-binding) Best, Bruno On Wed, Jul 8, 2020 at 8:12 AM Matthias J. Sax wrote: Thanks for the KIP. +1 (binding) -Matthias On 7/7

[jira] [Updated] (KAFKA-13699) ProcessorContext does not expose Stream Time

2022-03-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13699: Affects Version/s: 3.0.0 (was: 2.7.0) > ProcessorContext d

[jira] [Assigned] (KAFKA-13699) ProcessorContext does not expose Stream Time

2022-03-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-13699: --- Assignee: Matthias J. Sax > ProcessorContext does not expose Stream T

[jira] [Assigned] (KAFKA-13721) Left-join still emit spurious results in stream-stream joins in some cases

2022-03-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-13721: --- Assignee: Matthias J. Sax > Left-join still emit spurious results in stream-str

[jira] [Updated] (KAFKA-13722) Update internal interfaces that use ProcessorContext to use StateStoreContext instead

2022-03-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13722: Component/s: streams > Update internal interfaces that use ProcessorContext to

[jira] [Assigned] (KAFKA-8065) Forwarding modified timestamps does not reset timestamp correctly

2022-03-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-8065: -- Assignee: Matthias J. Sax > Forwarding modified timestamps does not reset timest

[jira] [Commented] (KAFKA-13683) Transactional Producer - Transaction with key xyz went wrong with exception: Timeout expired after 60000milliseconds while awaiting InitProducerId

2022-03-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503218#comment-17503218 ] Matthias J. Sax commented on KAFKA-13683: - {quote}We are using kakfa streams 3.0. {quote} Should

[jira] [Commented] (KAFKA-13699) ProcessorContext does not expose Stream Time

2022-03-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503214#comment-17503214 ] Matthias J. Sax commented on KAFKA-13699: - Seems the only question is, if we need to update

[jira] [Updated] (KAFKA-13699) ProcessorContext does not expose Stream Time

2022-03-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13699: Labels: newbie (was: ) > ProcessorContext does not expose Stream T

[jira] [Commented] (KAFKA-6035) Avoid creating changelog topics for state stores that are directly piped to a sink topic

2022-03-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503213#comment-17503213 ] Matthias J. Sax commented on KAFKA-6035: Even if this ticket is finished, if you call

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-03-07 Thread Matthias J. Sax
evel abstraction, therefore users should be aware of the potential runtime exceptions if the key changes. This is why I'm leaning towards alternative 1. Looking forward to your feedback. As a reminder, the vote thread is still open. Feel free to add your vote or amend if needed. Cheers, On Fri, 4 M

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-03-07 Thread Matthias J. Sax
level abstraction, therefore users should be aware of the potential runtime exceptions if the key changes. This is why I'm leaning towards alternative 1. Looking forward to your feedback. As a reminder, the vote thread is still open. Feel free to add your vote or amend if needed. Cheers, O

Re: [DISCUSS] KIP-813 Shared State Stores

2022-03-07 Thread Matthias J. Sax
a projection of the data. Either way, I would not add that into this KIP since it is a specific use-case pattern. Unless there is anything more to add or change, I would propose moving to a vote? Cheers! D. From: Matthias J. Sax Date: Friday, 18 February 2022 at 03:29 To: dev@kafka.apache.org

Re: How to achieve high availability in a Kafka Streams app during deployment?

2022-03-05 Thread Matthias J. Sax
Hard to answer from a 10,000ft view. In general, a rolling upgrade (ie, bounce one instance at a time) is recommended. If you have state, you would need to ensure that state is not lost during a bounce. As you are using Kubernetes, using stateful sets that allow you to re-attach disk should

Re: How to achieve high availability in a Kafka Streams app during deployment?

2022-03-05 Thread Matthias J. Sax
Hard to answer from a 10,000ft view. In general, a rolling upgrade (ie, bounce one instance at a time) is recommended. If you have state, you would need to ensure that state is not lost during a bounce. As you are using Kubernetes, using stateful sets that allow you to re-attach disk should

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-03-03 Thread Matthias J. Sax
do you mind clarifying these points in the Compatibility section of your KIP? Thanks, -John On Wed, 2022-02-23 at 15:07 -0800, Matthias J. Sax wrote: For this KIP, I also see the value. I was just trying to make a step back and ask if it's a good short term solution. If we believe it is,

[jira] [Commented] (KAFKA-13678) 2nd punctuation using STREAM_TIME does not respect scheduled interval

2022-03-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499753#comment-17499753 ] Matthias J. Sax commented on KAFKA-13678: - We have a different use case in mind: punctuation

[jira] [Commented] (KAFKA-13699) ProcessorContext does not expose Stream Time

2022-03-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499691#comment-17499691 ] Matthias J. Sax commented on KAFKA-13699: - That's my understanding, too. Seems we missed to add

[jira] [Commented] (KAFKA-13678) 2nd punctuation using STREAM_TIME does not respect scheduled interval

2022-02-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497935#comment-17497935 ] Matthias J. Sax commented on KAFKA-13678: - We believe it's important for the data re-processing

[jira] [Commented] (KAFKA-4212) Add a key-value store that is a TTL persistent cache

2022-02-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497934#comment-17497934 ] Matthias J. Sax commented on KAFKA-4212: A windowed-store is basically just a `` kv-store

[jira] [Commented] (KAFKA-10635) Streams application fails with OutOfOrderSequenceException after rolling restarts of brokers

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497214#comment-17497214 ] Matthias J. Sax commented on KAFKA-10635: - Is this fixed via https://issues.apache.org/jira

[jira] [Commented] (KAFKA-12446) Define KGroupedTable#aggregate subtractor + adder order of execution

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497210#comment-17497210 ] Matthias J. Sax commented on KAFKA-12446: - Just discovering the ticket. – I don't think it makes

[jira] [Resolved] (KAFKA-12640) AbstractCoordinator ignores backoff timeout when joining the consumer group

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12640. - Resolution: Duplicate > AbstractCoordinator ignores backoff timeout when join

[jira] [Resolved] (KAFKA-12640) AbstractCoordinator ignores backoff timeout when joining the consumer group

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12640. - Resolution: Duplicate > AbstractCoordinator ignores backoff timeout when join

[jira] [Commented] (KAFKA-13501) Avoid state restore via rebalance if standbys are enabled

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497201#comment-17497201 ] Matthias J. Sax commented on KAFKA-13501: - There is also https://issues.apache.org/jira/browse

[jira] [Commented] (KAFKA-13501) Avoid state restore via rebalance if standbys are enabled

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497196#comment-17497196 ] Matthias J. Sax commented on KAFKA-13501: - I just discovered https://issues.apache.org/jira

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-23 Thread Matthias J. Sax
, at 00:43, Jorge Esteban Quilcate Otoya wrote: On Fri, 18 Feb 2022 at 02:16, Matthias J. Sax wrote: It probably deserves its own thread to start discussing ideas. Yes. My question was: if we think it's time to do a DSL 2.0, should we drop this KIP and just fix via DSL 2.0 instead? Goo

[jira] [Updated] (KAFKA-13685) Seems to get empty event time in some messages.

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13685: Component/s: streams (was: consumer

[jira] [Updated] (KAFKA-13678) 2nd punctuation using STREAM_TIME does not respect scheduled interval

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13678: Issue Type: Improvement (was: Bug) > 2nd punctuation using STREAM_TIME does not resp

[jira] [Commented] (KAFKA-13678) 2nd punctuation using STREAM_TIME does not respect scheduled interval

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17497056#comment-17497056 ] Matthias J. Sax commented on KAFKA-13678: - What [~guozhang] says is right – we align stream-time

[jira] [Updated] (KAFKA-13677) version 3.0.0 Processor implementation uses parameter record (restricted identifier)

2022-02-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13677: Component/s: streams (was: clients) > version 3.0.0 Proces

Re: [DISCUSS] KIP-813 Shared State Stores

2022-02-17 Thread Matthias J. Sax
ctly, there was some checkpoint logic available when restoring, but we are bypassing that since logging is disabled on the statestore, no? As always, thanks for your insights. Cheers, D. From: Matthias J. Sax Date: Wednesday, 16 February 2022 at 02:09 To: dev@kafka.apache.org Subject: Re: [DIS

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-17 Thread Matthias J. Sax
Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: Thank you Matthias, this is great feedback. Adding my comments below. On Wed, 16 Feb 2022 at 00:42, Matthias J. Sax wrote: Thanks for the KIP. In alignment to my reply to KIP-634, I am wondering if we are heading into the right dir

Re: [DISCUSS] KIP-813 Shared State Stores

2022-02-15 Thread Matthias J. Sax
en what's the main difference of adding a read-only store v.s. adding a global store? I think because of the first answer the behavior differs from global stores. Makes sense? Cheers, D. From: Matthias J. Sax Date: Thursday, 20 January 2022 at 21:12 To: dev@kafka.apache.org Subject: Re: [DI

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

2022-02-15 Thread Matthias J. Sax
Thanks for the KIP. In alignment to my reply to KIP-634, I am wondering if we are heading into the right direction, or if we should consider to re-design the DSL from scratch? Even if we don't do a DSL 2.0 right now, I have some concerns about this KIP: (1) I am not sure if the propose

Re: [DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

2022-02-15 Thread Matthias J. Sax
key, final V value, final RecordContext recordContext); } ``` This would require to change `Topology#addSink()` to support this extractor as well. If this is aligned with your proposal, I'm happy to add it to this KIP. Cheers, Jorge. On Fri, Sep 11, 2020 at 11:03 PM Matthias J. Sax < mj..

[jira] [Comment Edited] (KAFKA-13655) Cannot edit clients page

2022-02-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492280#comment-17492280 ] Matthias J. Sax edited comment on KAFKA-13655 at 2/15/22, 12:23 AM

[jira] [Commented] (KAFKA-13655) Cannot edit clients page

2022-02-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492280#comment-17492280 ] Matthias J. Sax commented on KAFKA-13655: - Missed you message from a week ago – sorry about

[jira] [Commented] (KAFKA-6647) KafkaStreams.cleanUp creates .lock file in directory its trying to clean (Windows OS)

2022-02-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17491093#comment-17491093 ] Matthias J. Sax commented on KAFKA-6647: Not sure if I can follow the details – it might be best

[jira] [Assigned] (KAFKA-13229) KIP-761: implement a total blocked time metric in Kafka Streams

2022-02-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-13229: --- Assignee: Rohan Desai > KIP-761: implement a total blocked time metric in Ka

[jira] [Commented] (KAFKA-13641) Kafka Streams Scala: Add `Option` to `ValueJoiner` parameters

2022-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489966#comment-17489966 ] Matthias J. Sax commented on KAFKA-13641: - {quote}Hope I didn't come across as rude or imply

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

2022-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489962#comment-17489962 ] Matthias J. Sax commented on KAFKA-13289: - Great news! Thanks for sharing. Happy you got your

Re: [DISCUSS] Should we automatically close stale PRs?

2022-02-09 Thread Matthias J. Sax
I feel setting 6 months for closing and notifying committers on a per-week basis seems sufficient. Guozhang On Sun, Feb 6, 2022 at 9:58 AM Matthias J. Sax wrote: I am +1 to close stale PRs -- not sure to what extend we want to automate it, or just leave it up to the committers to do the cleanup man

[jira] [Resolved] (KAFKA-13655) Cannot edit clients page

2022-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13655. - Resolution: Not A Bug > Cannot edit clients p

[jira] [Comment Edited] (KAFKA-13655) Cannot edit clients page

2022-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489958#comment-17489958 ] Matthias J. Sax edited comment on KAFKA-13655 at 2/10/22, 5:05 AM

[jira] [Resolved] (KAFKA-13655) Cannot edit clients page

2022-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13655. - Resolution: Not A Bug > Cannot edit clients p

[jira] [Commented] (KAFKA-13655) Cannot edit clients page

2022-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489958#comment-17489958 ] Matthias J. Sax commented on KAFKA-13655: - Did you create an account (note the Jira and wiki

Re: [ANNOUNCE] New committer: Luke Chen

2022-02-09 Thread Matthias J. Sax
Congratulations! Glad to have you onboard, Luke! -Matthias On 2/9/22 16:37, Bill Bejeck wrote: Congrats Luke! Well deserved. -Bill On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo wrote: Congratulations Luke! Thank you for your service On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang wrote:

Re: [ANNOUNCE] New committer: Luke Chen

2022-02-09 Thread Matthias J. Sax
Congratulations! Glad to have you onboard, Luke! -Matthias On 2/9/22 16:37, Bill Bejeck wrote: Congrats Luke! Well deserved. -Bill On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo wrote: Congratulations Luke! Thank you for your service On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang wrote:

[jira] [Resolved] (KAFKA-13400) offset() is not supported in standby tasks.

2022-02-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13400. - Resolution: Not A Bug Based on the information provided, the conclusion seems

[jira] [Resolved] (KAFKA-13400) offset() is not supported in standby tasks.

2022-02-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13400. - Resolution: Not A Bug Based on the information provided, the conclusion seems

[jira] [Updated] (KAFKA-13492) IQ Parity: queries for key/value store range and scan

2022-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13492: Fix Version/s: 3.2.0 > IQ Parity: queries for key/value store range and s

[jira] [Updated] (KAFKA-8410) Strengthen the types of Processors, at least in the DSL, maybe in the PAPI as well

2022-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-8410: --- Description: KIP-478: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-478+-+Strongly

[jira] [Updated] (KAFKA-8410) Strengthen the types of Processors, at least in the DSL, maybe in the PAPI as well

2022-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-8410: --- Labels: kip tech-debt (was: tech-debt) > Strengthen the types of Processors, at le

[jira] [Resolved] (KAFKA-8410) Strengthen the types of Processors, at least in the DSL, maybe in the PAPI as well

2022-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8410. Resolution: Fixed This feature (KIP–478: [https://cwiki.apache.org/confluence/display

[jira] [Resolved] (KAFKA-8410) Strengthen the types of Processors, at least in the DSL, maybe in the PAPI as well

2022-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8410. Resolution: Fixed This feature (KIP–478: [https://cwiki.apache.org/confluence/display

[jira] [Updated] (KAFKA-8410) Strengthen the types of Processors, at least in the DSL, maybe in the PAPI as well

2022-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-8410: --- Fix Version/s: 2.7.0 > Strengthen the types of Processors, at least in the DSL, ma

<    8   9   10   11   12   13   14   15   16   17   >