Re: [DISCUSS] 3.5.2 Release

2023-10-13 Thread Matthias J. Sax
Thanks -- there is a few fixed for Kafka Streams we are considering to cherry-pick to get into 3.5.2 release -- can you give us a few more days for this? -Matthias On 10/12/23 6:20 PM, Sophie Blee-Goldman wrote: Thanks for volunteering Luke! On Thu, Oct 12, 2023 at 2:55 AM Levani Kokhreidze

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-13 Thread Matthias J. Sax
to javadocs for range() and reverseRange(): range(): *@return The iterator for this range, from smallest to largest bytes.* reverseRange(): * @return The reverse iterator for this range, from largest to smallest key bytes. Cheers, Alieh On Thu, Oct 12, 2023 at 7:32 AM Matthias J. Sax wrote: Quick

[jira] [Updated] (KAFKA-15601) Client metrics and observability

2023-10-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15601: Component/s: streams > Client metrics and observabil

[jira] [Updated] (KAFKA-15601) Client metrics and observability

2023-10-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15601: Component/s: admin clients consumer

[jira] [Updated] (KAFKA-15601) Client metrics and observability

2023-10-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15601: Labels: kip (was: ) > Client metrics and observabil

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-12 Thread Matthias J. Sax
network:type=RequestMetrics,name=RequestsPerSec,request=*) that report Request rate for every request type? Thanks, Jun On Wed, Oct 11, 2023 at 1:47 PM Matthias J. Sax wrote: Thanks! On 10/10/23 11:31 PM, Andrew Schofield wrote: Matthias, Yes, I think that’s a sensible way forward and the interface

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-12 Thread Matthias J. Sax
or not) could change in the future, but the API can remain the same. Another slight benefit is that there is no need for returning Optional. If the global consumer is not used, it just won't be included in the map. Thanks, Jun On Thu, Oct 12, 2023 at 9:30 AM Matthias J. Sax wrote: Thanks Sophie

[jira] [Updated] (KAFKA-15594) Add 3.6.0 to streams upgrade/compatibility tests

2023-10-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15594: Component/s: streams system tests > Add 3.6.0 to streams upgr

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-12 Thread Matthias J. Sax
le. Meanwhile within the method we would issue separate calls to each of the clients using the default or user-configured value of their `default.api.timeout.ms` as the timeout parameter. So the API as proposed makes sense to me. On Wed, Oct 11, 2023 at 6:48 PM Matthias J. Sax wrote: In ca

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-11 Thread Matthias J. Sax
()`, and and I assume also `prefixScan()` (even if there is no "reverse" version for it). On 10/11/23 10:22 PM, Matthias J. Sax wrote: Thanks for raising this question Hanyu. Great find! My interpretation is as follows (it's actually a warning signal that the API contract is not bett

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-11 Thread Matthias J. Sax
to != null && from.compareTo(to) > 0) { This section performs the necessary checks. Sincerely, Hanyu On Thu, Oct 5, 2023 at 9:52 AM Hanyu (Peter) Zheng < pzh...@confluent.io> wrote: Hi, Hao, In this case, it will return an empty set or list in the end. Sincerely, Ha

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

2023-10-11 Thread Matthias J. Sax
: I do not get what you mean by your second comment. Isn't the KIP already explicit about that? > I assume, results are returned by timestamp for each key. The KIP should be explicit about it. Cheers, Alieh On Tue, Oct 3, 2023 at 6:07 AM Matthias J. Sax wrote: Thanks for updating

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-11 Thread Matthias J. Sax
me". If I do that, then it is not consistent with KIP-960.     Moreover, the input parameter is clearly a timestamp, which explains     enough. What do you think about that?     5. I was asked to add more examples to the example section. My question     is, what is the main purpose of that?

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
return Optional? 132. ClientMetricsSubscriptionRequestCount: Do we need this since we have a set of generic metrics (kafka.network:type=RequestMetrics,name=RequestsPerSec,request=*) that report Request rate for every request type? Thanks, Jun On Wed, Oct 11, 2023 at 1:47 PM Matthias J. Sax wrote

Re: [VOTE] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
+1 (binding) On 9/13/23 5:48 PM, Jason Gustafson wrote: Hey Andrew, +1 on the KIP. For many users of Kafka, it may not be fully understood how much of a challenge client monitoring is. With tens of clients in a cluster, it is already difficult to coordinate metrics collection. When there are

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
Thanks! On 10/10/23 11:31 PM, Andrew Schofield wrote: Matthias, Yes, I think that’s a sensible way forward and the interface you propose looks good. I’ll update the KIP accordingly. Thanks, Andrew On 10 Oct 2023, at 23:01, Matthias J. Sax wrote: Andrew, yes I would like to get

[jira] [Commented] (KAFKA-15571) StateRestoreListener#onRestoreSuspended is never called because wrapper DelegatingStateRestoreListener doesn't implement onRestoreSuspended

2023-10-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773849#comment-17773849 ] Matthias J. Sax commented on KAFKA-15571: - Ups... Thanks to reporting and the PR

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-10 Thread Matthias J. Sax
can also be included in the proposed Kafka Streams follow-on KIP, or would you prefer that I add it to KIP-714? I have a slight preference for the former to put all of the KS enhancements into a separate KIP. Thanks, Andrew On 7 Oct 2023, at 02:12, Matthias J. Sax wrote: Thanks Andrew. SGTM. One

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-09 Thread Matthias J. Sax
it was an accident using getX in the IQv2 KIPs since somebody would have brought it up, otherwise. I am fine with both types of getters. If we think, we need to discuss this in a broader context, let's start a separate thread. Best, Bruno On 10/5/23 7:44 AM, Matthias J. Sax wrote: I agree

Re: [VOTE] KIP-960: Support single-key_single-timestamp interactive queries (IQv2) for versioned state stores

2023-10-09 Thread Matthias J. Sax
One more nit: as discussed on the related KIP-698 thread, we should not use `get` as prefix for the getters. So it should be `K key()` and `Optional asOfTimestamp()`. Otherwise the KIP LGTM. +1 (binding) -Matthias On 10/6/23 2:50 AM, Alieh Saeedi wrote: Hi everyone, Since KIP-960 is

[jira] [Updated] (KAFKA-15569) Update test and add test cases in IQv2StoreIntegrationTest

2023-10-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15569: Issue Type: Test (was: Improvement) > Update test and add test ca

[jira] [Updated] (KAFKA-15569) Update test and add test cases in IQv2StoreIntegrationTest

2023-10-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15569: Component/s: streams unit tests > Update test and add test ca

[jira] [Commented] (KAFKA-15378) Rolling upgrade system tests are failing

2023-10-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772748#comment-17772748 ] Matthias J. Sax commented on KAFKA-15378: - This is a partial fix with regard to versions

[jira] [Assigned] (KAFKA-15378) Rolling upgrade system tests are failing

2023-10-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15378: --- Assignee: Matthias J. Sax > Rolling upgrade system tests are fail

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-06 Thread Matthias J. Sax
not universally support them. I don’t think the KIP is improved by adding one now. I have also added constants for the various Config classes for ENABLE_METRICS_PUSH_CONFIG, including to StreamsConfig. It’s best to be explicit about this. Thanks, Andrew On 2 Oct 2023, at 23:47, Matthias J. Sax wr

[jira] [Resolved] (KAFKA-15437) Add metrics about open iterators

2023-10-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15437. - Resolution: Duplicate > Add metrics about open iterat

[jira] [Resolved] (KAFKA-15437) Add metrics about open iterators

2023-10-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15437. - Resolution: Duplicate > Add metrics about open iterat

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-04 Thread Matthias J. Sax
, if specified     */    public Optional getAsOfTimestamp(); Other than that I didn't have too much to add. Overall I like the direction of the KIP and think the funcatinlyt is all there! best, Walker On Mon, Oct 2, 2023 at 10:46 PM Matthias J. Sax wrote: Thanks for the updated KIP. Overa

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-04 Thread Matthias J. Sax
**(How is this implemented/guaranteed in an `inMemoryKeyValueStore`? I don't know about that implementation). Colt McNealy *Founder, LittleHorse.dev* On Tue, Oct 3, 2023 at 1:35 PM Hanyu (Peter) Zheng wrote: ok, I will update it. Thank you Matthias Sincerely, Hanyu On Tue, Oct 3, 2

[jira] [Commented] (KAFKA-15541) RocksDB Iterator Metrics

2023-10-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772072#comment-17772072 ] Matthias J. Sax commented on KAFKA-15541: - Did not read the KIP yet, but sound very similar

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-03 Thread Matthias J. Sax
Thanks for the KIP Hanyu! I took a quick look and it think the proposal makes sense overall. A few comments about how to structure the KIP. As you propose to not add `ReverseRangQuery` class, the code example should go into "Rejected Alternatives" section, not in the "Proposed Changes"

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

2023-10-02 Thread Matthias J. Sax
Thanks for updating the KIP. Not sure if I agree or not with Bruno's idea to split the query types further? In the end, we split them only because there is three different return types: single value, value-iterator, key-value-iterator. What do we gain by splitting out single-ts-range-key? In

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-10-02 Thread Matthias J. Sax
ility to make their code more readable instead of only relying on the default. Best, Bruno On 8/17/23 4:13 AM, Matthias J. Sax wrote: Thanks for splitting this part into a separate KIP! For `withKey()` we should be explicit that `null` is not allowed. (Looking into existing `KeyQu

Re: Kafka Streams reaching ERROR state during rolling upgrade / restart of brokers

2023-10-02 Thread Matthias J. Sax
the Kafka broker logs? I do not see any other errors logs on the client / application side. On Fri, 29 Sep, 2023, 22:01 Matthias J. Sax, wrote: In general, Kafka Streams should keep running. Can you inspect the logs to figure out why it's going into ERROR state to begin with? Maybe you need

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-10-02 Thread Matthias J. Sax
me complete the code and confirm it for 100%. About the KeyQuery class, I thought the KIP must contain just the newly added stuff. OK, I will paste the whole class in KIP-960. Thanks, Alieh On Thu, Aug 17, 2023 at 3:54 AM Matthias J. Sax wrote: Thanks for updating the KIP and splittin

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-10-02 Thread Matthias J. Sax
Hi, I did not pay attention to this KIP in the past; seems it was on-hold for a while. Overall it sounds very useful, and I think we should extend this with a follow up KIP for Kafka Streams. What is unclear to me at this point is the statement: Kafka Streams applications have an

[jira] [Updated] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15527: Labels: kip (was: ) > Add reverseRange and reverseAll query over kv-store in I

[jira] [Updated] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15527: Component/s: streams > Add reverseRange and reverseAll query over kv-store in I

[jira] [Commented] (KAFKA-15520) Kafka Streams Stateful Aggregation Rebalancing causing processing to pause on all partitions

2023-09-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770539#comment-17770539 ] Matthias J. Sax commented on KAFKA-15520: - {quote}However, I'm seeing that when pods restart

[jira] [Resolved] (KAFKA-15491) RackId doesn't exist error while running WordCountDemo

2023-09-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15491. - Fix Version/s: 3.6.1 3.7.0 Assignee: Hao Li

[jira] [Resolved] (KAFKA-15491) RackId doesn't exist error while running WordCountDemo

2023-09-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15491. - Fix Version/s: 3.6.1 3.7.0 Assignee: Hao Li

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-29 Thread Matthias J. Sax
Congrats! On 9/25/23 7:29 AM, Rajini Sivaram wrote: Congratulations, Justine! Regards, Rajini On Mon, Sep 25, 2023 at 9:40 AM Lucas Brutschy wrote: Congrats, Justine! On Mon, Sep 25, 2023 at 9:20 AM Bruno Cadonna wrote: Congrats, Justine! Well deserved! Best, Bruno On 9/25/23 5:28

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-29 Thread Matthias J. Sax
Congrats! On 9/26/23 1:30 AM, Mayank Shekhar Narula wrote: Congrats Yash! On Fri, Sep 22, 2023 at 5:24 PM Mickael Maison wrote: Congratulations Yash! On Fri, Sep 22, 2023 at 9:25 AM Chaitanya Mukka wrote: Congrats, Yash!! Well deserved. Chaitanya Mukka On 21 Sep 2023 at 8:58 PM +0530,

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-29 Thread Matthias J. Sax
Congrats! On 9/26/23 1:29 AM, Mayank Shekhar Narula wrote: Congratulations Lucas! On Fri, Sep 22, 2023 at 5:24 PM Mickael Maison wrote: Congratulations Lucas! On Fri, Sep 22, 2023 at 7:13 AM Luke Chen wrote: Congratulations, Lukas! Luke On Fri, Sep 22, 2023 at 6:53 AM Tom Bentley

Re: Kafka Streams reaching ERROR state during rolling upgrade / restart of brokers

2023-09-29 Thread Matthias J. Sax
In general, Kafka Streams should keep running. Can you inspect the logs to figure out why it's going into ERROR state to begin with? Maybe you need to increase/change some timeouts/retries configs. The stack trace you shared, is a symptom, but not the root cause. -Matthias On 9/21/23 12:56

Re: Can a message avoid loss occur in Kafka

2023-09-29 Thread Matthias J. Sax
For the config you provide, data loss should not happen (as long as you don't allow for unclean leader election, which is disabled by default). But you might be subject to unavailability for some partitions if a broker fails. -Matthias On 9/17/23 7:49 AM, 陈近南 wrote: Hello, Can a

[jira] [Updated] (KAFKA-15463) StreamsException: Accessing from an unknown node

2023-09-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15463: Priority: Major (was: Blocker) > StreamsException: Accessing from an unknown n

Re: FW: UNSUBSCRIBE

2023-09-12 Thread Matthias J. Sax
To unsubscribe, you need to send an email to users-unsubscr...@storm.apache.org. Bests, -Matthias On 9/12/23 6:37 AM, hmm0403 wrote: UNSUBSCRIBE 내 Galaxy에서 보냄 원본 이메일 발신: Michele Volpe 날짜: 23/9/12 오후 10:26 (GMT+09:00) 받은 사람: user@storm.apache.org 제목: Re:

[jira] [Created] (KAFKA-15443) Upgrade RocksDB dependency

2023-09-07 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15443: --- Summary: Upgrade RocksDB dependency Key: KAFKA-15443 URL: https://issues.apache.org/jira/browse/KAFKA-15443 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-15443) Upgrade RocksDB dependency

2023-09-07 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15443: --- Summary: Upgrade RocksDB dependency Key: KAFKA-15443 URL: https://issues.apache.org/jira/browse/KAFKA-15443 Project: Kafka Issue Type: Task

[jira] [Assigned] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15417: --- Assignee: Victor van den Hoven > JoinWindow does not seem to work prope

[jira] [Commented] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762863#comment-17762863 ] Matthias J. Sax commented on KAFKA-15417: - Thanks a lot! Assigned the ticket to you

[jira] [Comment Edited] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762535#comment-17762535 ] Matthias J. Sax edited comment on KAFKA-15417 at 9/7/23 1:39 AM: - Thanks

[jira] [Commented] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762535#comment-17762535 ] Matthias J. Sax commented on KAFKA-15417: - Thanks. I did look into this and found this PR

Re: kafka streams consumer group reporting lag even on source topics removed from topology

2023-09-05 Thread Matthias J. Sax
t is the case with topics that were consumed previously and not consumed now. Does creation of new consumer group (setting a different application.id) on streams application an option here? On Thu, Aug 17, 2023 at 7:03 AM Matthias J. Sax wrote: Well, it's kinda expected behavior. I

[jira] [Created] (KAFKA-15437) Add metrics about open iterators

2023-09-05 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15437: --- Summary: Add metrics about open iterators Key: KAFKA-15437 URL: https://issues.apache.org/jira/browse/KAFKA-15437 Project: Kafka Issue Type: New

[jira] [Created] (KAFKA-15437) Add metrics about open iterators

2023-09-05 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15437: --- Summary: Add metrics about open iterators Key: KAFKA-15437 URL: https://issues.apache.org/jira/browse/KAFKA-15437 Project: Kafka Issue Type: New

[jira] [Commented] (KAFKA-15309) Add custom error handler to Producer

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761924#comment-17761924 ] Matthias J. Sax commented on KAFKA-15309: - Sure, the ticket is up for grabs. Note, that we

[jira] [Comment Edited] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761918#comment-17761918 ] Matthias J. Sax edited comment on KAFKA-15417 at 9/5/23 1:57 AM: - I

[jira] [Comment Edited] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761918#comment-17761918 ] Matthias J. Sax edited comment on KAFKA-15417 at 9/5/23 1:56 AM: - I

[jira] [Comment Edited] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761918#comment-17761918 ] Matthias J. Sax edited comment on KAFKA-15417 at 9/5/23 1:56 AM: - I

[jira] [Comment Edited] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761918#comment-17761918 ] Matthias J. Sax edited comment on KAFKA-15417 at 9/5/23 1:55 AM: - I

[jira] [Commented] (KAFKA-15417) JoinWindow does not seem to work properly with a KStream - KStream - LeftJoin()

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761918#comment-17761918 ] Matthias J. Sax commented on KAFKA-15417: - I believe it could be related to an internal

[jira] [Updated] (KAFKA-15383) Replace EasyMock with Mockito for KTableImplTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15383: Component/s: streams unit tests > Replace EasyMock with Mock

[jira] [Updated] (KAFKA-15385) Replace EasyMock with Mockito for AbstractStreamTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15385: Component/s: streams unit tests > Replace EasyMock with Mock

[jira] [Updated] (KAFKA-15382) Replace EasyMock with Mockito for KStreamTransformValuesTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15382: Component/s: streams unit tests > Replace EasyMock with Mock

[jira] [Updated] (KAFKA-15384) Replace EasyMock with Mockito for KTableTransformValuesTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15384: Component/s: streams unit tests > Replace EasyMock with Mock

Re: AW: Table updates are not consistent when doing a join with a Stream

2023-09-04 Thread Matthias J. Sax
Your update to the KTable is async when you send data back to the KTable input topic. So your program is subject to race-conditions. So switching to the PAPI was the right move: it make the update to the state store sync and thus fixes the issue. -Matthias On 9/4/23 5:53 AM, Mauricio Lopez

Re: kafka streams consumer group reporting lag even on source topics removed from topology

2023-09-04 Thread Matthias J. Sax
e inactive consumer group is cleared up after 7 days however not sure if that is the case with topics that were consumed previously and not consumed now. Does creation of new consumer group (setting a different application.id) on streams application an option here? On Thu, Aug 17, 2023 at 7:03 AM

[jira] [Updated] (KAFKA-15378) Rolling upgrade system tests are failing

2023-08-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15378: Component/s: system tests > Rolling upgrade system tests are fail

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-08-16 Thread Matthias J. Sax
Thanks for splitting this part into a separate KIP! For `withKey()` we should be explicit that `null` is not allowed. (Looking into existing `KeyQuery` it seems the JavaDocs don't cover this either -- would you like to do a tiny cleanup PR for this, or fix on-the-side in one of your PRs?)

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-08-16 Thread Matthias J. Sax
queries+%28IQv2%29+for+versioned+state+stores> Cheers, Alieh On Thu, Aug 10, 2023 at 2:38 AM Matthias J. Sax wrote: Seems there was a lot of additional feedback. Looking forward to an updated version of the KIP. I also agree to make the queries more composable. I was considering to ra

Re: kafka streams consumer group reporting lag even on source topics removed from topology

2023-08-16 Thread Matthias J. Sax
Well, it's kinda expected behavior. It's a split brain problem. In the end, you use the same `application.id / group.id` and thus the committed offsets for the removed topics are still in `__consumer_offsets` topics and associated with the consumer group. If a tool inspects lags and compares

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755298#comment-17755298 ] Matthias J. Sax commented on KAFKA-15302: - I would need to think more about this. Overall

[jira] [Commented] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-08-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755184#comment-17755184 ] Matthias J. Sax commented on KAFKA-15338: - Sure. Thank a lot! > The metric group documentat

[jira] [Updated] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15346: Component/s: streams > Single-Key_single-timestamp IQs with versioned state sto

[jira] [Updated] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15347: Component/s: streams > Single-Key_multi-timestamp IQs with versioned state sto

[jira] [Updated] (KAFKA-15348) Range IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15348: Component/s: streams > Range IQs with versioned state sto

[jira] [Updated] (KAFKA-15344) Kafka Streams should include the message leader epoch when committing offsets

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15344: Component/s: streams > Kafka Streams should include the message leader epoch w

[jira] [Assigned] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15319: --- Assignee: Lucas Brutschy > Upgrade rocksdb to fix CVE-2022-37

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Fix Version/s: 3.6.0 > Upgrade rocksdb to fix CVE-2022-37

[jira] [Updated] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15338: Labels: beginner newbie (was: ) > The metric group documentation for metrics ad

[jira] [Updated] (KAFKA-15337) Disable *-subscription-store-changelog topic creation for foreign key join in Kafka Streams

2023-08-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15337: Labels: need-kip (was: ) > Disable *-subscription-store-changelog topic creat

[jira] [Updated] (KAFKA-15337) Disable *-subscription-store-changelog topic creation for foreign key join in Kafka Streams

2023-08-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15337: Component/s: streams > Disable *-subscription-store-changelog topic creation for fore

[jira] [Resolved] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13197. - Fix Version/s: 3.6.0 3.5.2 Resolution: Fixed > KStr

[jira] [Resolved] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13197. - Fix Version/s: 3.6.0 3.5.2 Resolution: Fixed > KStr

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-10 Thread Matthias J. Sax
ontrol it with the flag, or maybe create some special aggregate that could be chained after (don't know how to do it yet :-)) Any thoughts? Regards, Igor On Wed, Aug 9, 2023 at 7:19 PM Matthias J. Sax wrote: Thanks for the details. And sorry for being a little bit picky. My goal is to real

Re: [VOTE] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-10 Thread Matthias J. Sax
+1 (binding) On 8/10/23 12:31 PM, Florin Akermann wrote: https://cwiki.apache.org/confluence/display/KAFKA/KIP-962%3A+Relax+non-null+key+requirement+in+Kafka+Streams

[jira] [Updated] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13197: Component/s: documentation streams > KStream-GlobalKTable join semant

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-10 Thread Matthias J. Sax
on the PR. Great. Yes, in general, I hope everybody agrees that we shouldn't add more details to this KIP On Thu, 10 Aug 2023 at 00:16, Matthias J. Sax wrote: Thanks for the KIP. left join KStream-GlobalTable: no longer drop left records with null-key and call KeyValueMapper with 'null' for left

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-08-09 Thread Matthias J. Sax
than the timestamp. I propose to rename the getters to getTimeFrom() and getTimeTo() as in WindowRangeQuery. 5. Please add the Test Plan section. Regarding long vs Instant: Did we miss to use Instant instead of long for all interfaces of the versioned state stores? Best, Bruno On 7/26

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-09 Thread Matthias J. Sax
d be a flag to stream-table foreign key join that would indicate if we want this join to aggregate children or not? What do you think? Regards, Igor On Fri, Aug 4, 2023 at 10:01 PM Matthias J. Sax wrote: Thanks a lot for providing more background. It's getting much clear to me now. Coupl

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-09 Thread Matthias J. Sax
s one good example that the latter can help you distinguish different scenarios whereas the former cannot. On Fri, Aug 4, 2023 at 6:32 PM Matthias J. Sax wrote: Guozhang, thanks for pointing out ValueJoinerWithKey. In the end, it's just a documentation change, ie, point out that the passed in

[jira] [Commented] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752522#comment-17752522 ] Matthias J. Sax commented on KAFKA-14049: - I am still not sure if I understand this ticket

[jira] [Updated] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14049: Labels: (was: kip) > Relax Non Null Requirement for KStreamGlobalKTable Left J

[jira] [Updated] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14049: Labels: kip (was: beginner newbie) > Relax Non Null Requirement for KStreamGlobalKTa

[jira] [Updated] (KAFKA-14748) Relax non-null FK left-join requirement

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14748: Description: Kafka Streams enforces a strict non-null-key policy in the DSL across all

[jira] [Updated] (KAFKA-14748) Relax non-null FK left-join requirement

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14748: Labels: kip (was: ) > Relax non-null FK left-join requirem

[jira] [Updated] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12317: Description: Currently, for a stream-streams and stream-table/globalTable join

[jira] [Updated] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12317: Labels: kip (was: ) > Relax non-null key requirement for left/outer KStream jo

<    1   2   3   4   5   6   7   8   9   10   >