RE: Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Hi Alieh, Thanks for the KIP! 1. As mentioned in the discussion thread for KIP-960, I think it'd be good to add a new method to the VersionedKeyValueStore interface for supporting single-key, multi-timestamp lookups, as part of implementing these new interactive queries so that the IQ

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

e is no need to do everything with a single > >> KIP. > >> > >> About the original v-store KIP and `long` vs `Instance` -- I don't think > >> we forget it. If the store is use inside a `Processor` using `long` is > >> preferred because performan

RE: Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

Hey Alieh, Thanks for the KIP! It looks like the KIP proposes three different types of interactive queries for versioned stores, though they are grouped together into two classes: VersionedKeyQuery adds supports for single-key, single-timestamp lookups, and also for single-key,

Re: [DISCUSS] KIP-923: Add A Grace Period to Stream Table Join

guish between records with the same key and same timestamp. > > > > > > Just an idea! Maybe Walker comes up with something better. > > > > > > Best, > > > Bruno > > > > > > On 05.06.23 20:38, Victoria Xia wrote: > > >> Hi Walker, > >

Re: [VOTE] KIP-923: Add A Grace Period to Stream Table Join

Hi Walker, Thanks for the KIP! Left a clarification question on the discussion thread just now but it's about an implementation detail, so I don't think it changes anything in this vote thread. +1 (non-binding) Cheers, Victoria On Mon, Jun 5, 2023 at 10:23 AM Bill Bejeck wrote: > Hi Walker,

Re: [DISCUSS] KIP-923: Add A Grace Period to Stream Table Join

>>>> path for versioned state stores, and also changing the program to > >>>>>> explicitly materialize using a versioned store. Also note, that we > >> might > >>>>>> not materialize the actual join table, but only an upstream table

Re: [DISCUSS] KIP-923: Add A Grace Period to Stream Table Join

rder. > > But I also see a benefit of re-ordering and emitting out-of-order data > > right away when read (instead of blocking them behind in-order records > > that are not ready yet). -- It might even be a possibility, to let users > > pick a emit strategy eg "EmitStra

Re: [DISCUSS] KIP-923: Add A Grace Period to Stream Table Join

Cool KIP, Walker! Thanks for sharing this proposal. A few clarifications: 1. Is the order that records exit the buffer in necessarily the same as the order that records enter the buffer in, or no? Based on the description in the KIP, it sounds like the answer is no, i.e., records will exit the

[jira] [Created] (KAFKA-14949) Add Streams upgrade tests from AK 3.4

Victoria Xia created KAFKA-14949: Summary: Add Streams upgrade tests from AK 3.4 Key: KAFKA-14949 URL: https://issues.apache.org/jira/browse/KAFKA-14949 Project: Kafka Issue Type: Task

Re: Fwd: [VOTE] KIP-914 Join Processor Semantics for Versioned Stores

with 3.5 as the release deadline is fast approaching. Thanks, Victoria On Tue, Apr 11, 2023 at 2:56 PM Victoria Xia wrote: > Thanks for your comments and suggestions, Matthias, Lucas, and Guozhang! > > I was just in the process of responding when I saw Guozhang's message. I > came up with

Re: Fwd: [VOTE] KIP-914 Join Processor Semantics for Versioned Stores

it the same way from the old record that is passed down to the > > > aggregation after KIP-904? > > > > > > Thanks, > > > Lucas > > > > > > On Tue, Apr 11, 2023 at 5:35 AM Matthias J. Sax > wrote: > > >> > > >> Thank

Fwd: [VOTE] KIP-914 Join Processor Semantics for Versioned Stores

g with the rest of KIP-914 in the 3.5 release. Apologies for not noticing these additional semantics implications earlier, Victoria -- Forwarded message --------- From: Victoria Xia Date: Wed, Mar 22, 2023 at 10:08 AM Subject: Re: [VOTE] KIP-914 Join Processor Semantics for Versione

[jira] [Created] (KAFKA-14864) Memory leak in KStreamWindowAggregate with ON_WINDOW_CLOSE emit strategy

Victoria Xia created KAFKA-14864: Summary: Memory leak in KStreamWindowAggregate with ON_WINDOW_CLOSE emit strategy Key: KAFKA-14864 URL: https://issues.apache.org/jira/browse/KAFKA-14864 Project

[jira] [Created] (KAFKA-14834) Improved stream-table and table-table join semantics for versioned stores

Victoria Xia created KAFKA-14834: Summary: Improved stream-table and table-table join semantics for versioned stores Key: KAFKA-14834 URL: https://issues.apache.org/jira/browse/KAFKA-14834 Project

Re: [VOTE] KIP-914 Join Processor Semantics for Versioned Stores

ing) > > Thanks Victoria! > > Best, > Bruno > > On 20.03.23 17:13, Matthias J. Sax wrote: > > +1 (binding) > > > > On 3/20/23 9:05 AM, Guozhang Wang wrote: > >> +1, thank you Victoria! > >> > >> On Sat, Mar 18, 2023 at 8:27 AM Victoria Xia

Re: [DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

The voting thread for this KIP is open. Thanks, Guozhang and Matthias, for already having voted! On Fri, Mar 17, 2023 at 1:18 PM Guozhang Wang wrote: > Thanks Matthias / Victoria, both bullet points make sense to me. > > On Thu, Mar 16, 2023 at 10:39 AM Victoria Xia > wrote: &g

[VOTE] KIP-914 Join Processor Semantics for Versioned Stores

Hi all, I'd like to start a vote on KIP-914 for updating the Kafka Streams join processors to use proper timestamp-based semantics in applications with versioned stores: https://cwiki.apache.org/confluence/display/KAFKA/KIP-914%3A+Join+Processor+Semantics+for+Versioned+Stores To avoid

Re: [DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

a vote at the end of the day. Happy to continue discussing even after the vote has started. Thanks for all the great suggestions so far! Best, Victoria On Thu, Mar 16, 2023 at 10:38 AM Victoria Xia wrote: > Thanks for your comments, Matthias! > > > For stream-table joins, I t

Re: [DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

change for left-joins either I believe. > > > -Matthias > > > > On 3/15/23 2:00 PM, Guozhang Wang wrote: > > Sounds good to me. Thanks! > > > > On Wed, Mar 15, 2023 at 12:07 PM Victoria Xia > > wrote: > >> > >> Thanks for kicking off

Re: [DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

ks for the KIP, Victoria! > > > > I had some questions/concerns, but you addressed them in the Rejected > Alternatives section. Thanks for the thorough proposal! > > > > -John > > > > On Thu, Mar 9, 2023, at 18:59, Victoria Xia wrote: > > > Hi everyone, &g

[DISCUSS] KIP-914 Join Processor Semantics for Versioned Stores

Hi everyone, I have a proposal for updating Kafka Streams's stream-table join and table-table join semantics for the new versioned key-value state stores introduced in KIP-889 . Would love to hear your thoughts

[jira] [Created] (KAFKA-14723) Do not write expired store records to changelog

Victoria Xia created KAFKA-14723: Summary: Do not write expired store records to changelog Key: KAFKA-14723 URL: https://issues.apache.org/jira/browse/KAFKA-14723 Project: Kafka Issue Type

Re: [VOTE] KIP-889 Versioned State Stores

oria On Wed, Dec 21, 2022 at 8:22 AM Victoria Xia wrote: > Hi everyone, > > We have 3 binding and 1 non-binding vote in favor of this KIP (and no > objections) so KIP-889 is now accepted. > > Thanks for voting, and for your excellent comments in the KIP discussion > thread! >

Re: [VOTE] KIP-889 Versioned State Stores

20:03, Matthias J. Sax wrote: > > > +1 (binding) > > > > > > On 12/15/22 1:27 PM, John Roesler wrote: > > >> Thanks for the thorough KIP, Victoria! > > >> > > >> I'm +1 (binding) > > >> > > >> -John > >

Re: [DISCUSS] KIP-889 Versioned State Stores

> The public constructor would do exactly the same, wouldn't it? > > I would make the constructor public and remove the static method make(). > > Best, > Bruno > > On 15.12.22 20:58, Victoria Xia wrote: > > Thanks again for the great discussion, Sagar, Bruno, and Matth

Re: [DISCUSS] KIP-889 Versioned State Stores

GTM. > > On 12/13/22 3:06 PM, Victoria Xia wrote: > > Hi Matthias, > > > > Thanks for chiming in! Barring objections from anyone on this thread, I > > will start the vote for this KIP on Thursday. That should be enough time > to > > incorporate any lingering

[VOTE] KIP-889 Versioned State Stores

Hi all, I'd like to start a vote on KIP-889 for introducing versioned key-value state stores to Kafka Streams: https://cwiki.apache.org/confluence/display/KAFKA/KIP-889%3A+Versioned+State+Stores The discussion thread has been open for a few weeks now and has converged among the current

[jira] [Created] (KAFKA-14491) Introduce Versioned Key-Value Stores to Kafka Streams

Victoria Xia created KAFKA-14491: Summary: Introduce Versioned Key-Value Stores to Kafka Streams Key: KAFKA-14491 URL: https://issues.apache.org/jira/browse/KAFKA-14491 Project: Kafka Issue

Re: [DISCUSS] KIP-889 Versioned State Stores

side benefit. > (Btw: your code snippet in the KIP shows what `VersionedRecord` would > have a non-null requirement for the value, but I think it would need to > allow null as value?) > > > -Matthias > > On 12/7/22 5:23 PM, Victoria Xia wrote: > > Thanks for the d

Re: [DISCUSS] KIP-889 Versioned State Stores

he new versioned-store, but change it globally in one shot > independent of this KIP. > > > -Matthias > > > > On 12/1/22 10:15 AM, Sagar wrote: > > Thanks Victoria, > > > > I guess an advantage of exposing a method like delete(key, timestamp) > could >

Re: [DISCUSS] KIP-889 Versioned State Stores

e that? I am not sure if we do > that > > in our codebase though. Maybe the experts can chime in. > > > > 3) hmm I would have named it `validUpto` But again not very picky about > it. > > After going through the link and your KIP, it's a lot clearer

Re: [DISCUSS] KIP-889 Versioned State Stores

ny statements about expected performance? Seems to be an implementation > > detail? > > > > (6) validTo timestamp is "exclusive", right? Ie, if I query > > `get(key,ts[=validToV1])` I would get `null` or the "next" record v2 > > with validFr

[DISCUSS] KIP-889 Versioned State Stores

Hi everyone, I have a proposal for introducing versioned state stores in Kafka Streams. Versioned state stores are similar to key-value stores except they can store multiple record versions for a single key. This KIP focuses on interfaces only in order to limit the scope of the KIP.

Re: [VOTE] KIP-775: Custom partitioners in foreign key joins

> > > > > > Glad to see this in here :) > > > > > > On Tue, Sep 28, 2021 at 5:11 PM Bill Bejeck wrote: > > > > > >> +1 (binding) > > >> > > >> On Tue, Sep 28, 2021 at 12:59 PM Matthias J. Sax > > wrote: &g

[VOTE] KIP-775: Custom partitioners in foreign key joins

Hi all, I'd like to start a vote for KIP-775 for adding Kafka Streams support for foreign key joins on tables with custom partitioners: https://cwiki.apache.org/confluence/display/KAFKA/KIP-775%3A+Custom+partitioners+in+foreign+key+joins Thanks, Victoria

Re: [DISCUSS] KIP-775: Custom partitioners in foreign key joins

;>>>> 3. when we issue a key-based query via IQ, we need to know > >>>>> the partitioner, so the IQ interface allows us to pass in a > >>>>> custom partitioner with the query. > >>>>> > >>>>> This is a bit weird. Taking a step back, the parti

Re: [DISCUSS] KIP-775: Custom partitioners in foreign key joins

; >> joins). This new object could also implement `NamedOperation` and thus > >> replace `Named`. To this end, we would deprecate the existing methods > >> using `Named` and replace them with the new methods. Net benefit is, > >> that we don't get more overloads (after w

Re: [DISCUSS] KIP-775: Custom partitioners in foreign key joins

deprecate the existing methods > using `Named` and replace them with the new methods. Net benefit is, > that we don't get more overloads (after we removed the deprecated ones). > > Not sure how we want to call the new object. Maybe `TableJoined` in > alignment to `StreamJoined`? >

[DISCUSS] KIP-775: Custom partitioners in foreign key joins

Hi, I've opened a small KIP for adding Kafka Streams support for foreign key joins on tables with custom partitioners: https://cwiki.apache.org/confluence/display/KAFKA/KIP-775%3A+Custom+partitioners+in+foreign+key+joins Feedback appreciated. Thanks! - Victoria

[jira] [Created] (KAFKA-12366) Performance regression in stream-table joins on trunk

Victoria Xia created KAFKA-12366: Summary: Performance regression in stream-table joins on trunk Key: KAFKA-12366 URL: https://issues.apache.org/jira/browse/KAFKA-12366 Project: Kafka Issue

[jira] [Created] (KAFKA-10008) Symbol not found when running Kafka Streams with RocksDB dependency on MacOS 10.13.6

Victoria Xia created KAFKA-10008: Summary: Symbol not found when running Kafka Streams with RocksDB dependency on MacOS 10.13.6 Key: KAFKA-10008 URL: https://issues.apache.org/jira/browse/KAFKA-10008