Hey Matthias We store data in SSs in key value form. So the queries return value V which is ValueTimestampHeaders in headers-aware SSs case (and ValueAndTimestamp in timestamped SSs). Does returning Record mean that we consider V as an intermittent result and then we convert it to Record for the user?
Thanks -Alieh On Tue, Jun 16, 2026, 8:28 PM Matthias J. Sax <[email protected]> wrote: > Thanks. > > I just had a high level thought. Not sure if this idea is good or bad, > so I just wanted to put it out, to see what people think. > > With the "new" PAPI, we move from KV to `Record` model. For IQ, we > historically also use a KV-based design, and extended it incrementally > via `ValueAndTimestamp` and most recently via `ValueTimestampHeaders`. > > While we need `ValueTimestampHeaders` internally (it's the par that goes > into the RocksDB value), I am wondering if it could make sense for IQv2 > to also use `Record`? > > If we use `Record`, we would avoid the `ValueTimestampHeaders` vs > `AggregationsWithHeaders` (which also have a ts, but just stored in the > RocksDB `SessionKey` object) question, and long term, we might also move > to a `Record` model in the DSL. So we could align and standardize on > `Record` throughout the whole API surface area in the long term? > > > Thoughts? > > > -Matthias > > On 6/16/26 7:19 AM, Jess Jin via dev wrote: > > Thanks Alieh! Updated in the wiki. > > > > - Jess > > > > On Tue, Jun 16, 2026 at 7:49 AM Alieh Saeedi via dev < > [email protected]> > > wrote: > > > >> Hey Jess, > >> I see that you agreed with some of the suggestions and design > alternatives, > >> including the naming. Could you please: > >> > >> - Update the wiki to the committed design? > >> > >> > >> - Fix the motivation wording: existing query types don't "drop" > headers... > >> > >> Thanks > >> - Alieh > >> > >> On Mon, Jun 15, 2026 at 10:02 PM Jess Jin via dev <[email protected] > > > >> wrote: > >> > >>> Hi Lucas, > >>> > >>> Thanks! I agree. The new type only offers withKey(...) (single key → > all > >>> sessions for that key), so "Range" is misleading. I'll rename it to > >>> SessionKeyWithHeadersQuery. That also keeps the "Range" name free for a > >>> future genuine key-range session query. > >>> > >>> - Jess > >>> > >>> On Mon, Jun 15, 2026 at 11:03 AM Lucas Brutschy < > [email protected]> > >>> wrote: > >>> > >>>> Hi Jess, > >>>> > >>>> Thanks for the KIP, and for working through the latest round with > >>>> Matthias and Bill. > >>>> > >>>> LB1: For the session query, the existing IQv2 entry point is > >>>> WindowRangeQuery created via withKey(), which returns all sessions for > >>>> a single key. Is "Range" the right word when only withKey() is > >>>> offered? Seems "Range" previously referred to a range of keys, but > >>>> that range is not part of the new type anymore. > >>>> > >>>> Thanks, > >>>> Lucas > >>>> > >>>> On Mon, Jun 15, 2026 at 4:37 PM Jess Jin via dev < > [email protected] > >>> > >>>> wrote: > >>>>> > >>>>> Hi Bill, > >>>>> > >>>>> Thanks for the review. I agree this needed to be nailed down. > >>>>> > >>>>> - Jess > >>>>> > >>>>> On Fri, Jun 12, 2026 at 2:41 PM Bill Bejeck <[email protected]> > >> wrote: > >>>>> > >>>>>> Hi Jess, > >>>>>> > >>>>>> Thanks for the KIP. > >>>>>> > >>>>>> I'd second the importance of getting the details flushed out in > >>>> Matthias's > >>>>>> 4th comment with regards to session stores > >>>>>> and how we'll support `ValueTimestampeHeader` and > >>>> `AggregationWithHeaders` > >>>>>> and if session-header-stores are supported in this KIP. > >>>>>> > >>>>>> -Bill > >>>>>> > >>>>>> On Thu, Jun 11, 2026 at 8:59 PM Matthias J. Sax <[email protected]> > >>>> wrote: > >>>>>> > >>>>>>> Thanks for the KIP. Very nice to see that IQv2 gets more love. > >>>>>>> > >>>>>>> > >>>>>>> Couple of comments/questions. > >>>>>>> > >>>>>>> > >>>>>>> MJS1: The KIP says. > >>>>>>> > >>>>>>>> surfaced through the public wrapper type > >> ValueTimestampHeaders<V> > >>>>>>> > >>>>>>> Yes, but session-header-store actually uses > >>> `AggregationWithHeaders`. > >>>>>>> Should we mention both cases? Or do we not include > >> session-headers > >>>>>>> stores? Later, the KIP only mentions > >>>>>>> `MeteredTimestampedKeyValueStoreWithHeaders` and > >>>>>>> `MeteredTimestampedWindowStoreWithHeaders`, so maybe that's > >> indeed > >>>> what > >>>>>>> is proposed? If yes, why do exclude session-header-store? Also > >> if, > >>>> yes, > >>>>>>> would be good to call this out explicitly. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> MJS2: The KIP says. > >>>>>>> > >>>>>>>> The existing query types — TimestampedKeyQuery, > >>>> TimestampedRangeQuery, > >>>>>>> WindowKeyQuery, and WindowRangeQuery — all return values without > >>>> headers, > >>>>>>> even when run against a header-aware store. > >>>>>>> > >>>>>>> This sound as if these query types are already supported, but I > >>>> believe > >>>>>>> they are not. Same for their timestamp-less equivalents like > >>>> `KeyQuery`. > >>>>>>> > >>>>>>> I think it would be a good extension to the KIP, to also add > >>> support > >>>> for > >>>>>>> these existing query types on the newly added header-stores. We > >>> don't > >>>>>>> need to define any new classed. The KIP would just say, that we > >>>> extend > >>>>>>> the kv-ts-header store to also support existing KeyQuery and > >>>>>>> TimestampedKeyQuery (and similar for window-header-store; and > >> maybe > >>>> also > >>>>>>> session-header-store, in case the KIP does include > >>>> session-header-stores) > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> MJS3: Question about naming. You propose to add: > >>>>>>> > >>>>>>> - TimestampedKeyWithHeadersQuery > >>>>>>> - TimestampedRangeWithHeadersQuery > >>>>>>> - WindowKeyWithHeadersQuery > >>>>>>> - WindowRangeWithHeadersQuery > >>>>>>> > >>>>>>> But all four query types return some form of > >>> `ValueTimestampHeaders` > >>>>>>> results. So it seems we should align the names, and maybe call > >> the > >>>> later > >>>>>>> two > >>>>>>> > >>>>>>> - TimestampedWindowKeyWithHeadersQuery > >>>>>>> - TimestampedWindowRangeWithHeadersQuery > >>>>>>> > >>>>>>> Or, to avoid very long names, shorten the first two to > >>>>>>> > >>>>>>> - KeyWithHeadersQuery > >>>>>>> - RangeWithHeadersQuery > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> MJS4: About `WindowRangeWithHeadersQuery`. > >>>>>>> > >>>>>>> First some background: The existing `WindowRangeQuery` has two > >>>> methods > >>>>>>> how to setup the query, and depending which one is used, the > >> query > >>>> can > >>>>>>> be used against either a windowed-store or a session-store, ie, > >>> while > >>>>>>> both are a `WindowRangeQuery`, it's technically two different > >>> queries > >>>>>>> for two different stores. > >>>>>>> > >>>>>>> - `withKey(...)` -> only supported by session stores > >>>>>>> - `withWindowStartRange(...)` -> only supported by window store > >>>>>>> > >>>>>>> The KIP proposed to add both methods, too, but at the same time > >>> says, > >>>>>>> that window-store won't accept a query created via `withKey` > >> (what > >>> in > >>>>>>> general aligns to what we have). But the KIP seems to exclude > >>>>>>> session-header-store support. So why would we add `withKey` at > >> all? > >>>>>>> Would it not be simpler to only add the supported > >>>> `withWindowStartRange` > >>>>>>> method? > >>>>>>> > >>>>>>> Of course, if we want to mimic the existing query type, the > >>>>>>> WindowRangeWithHeadersQuery seems to face the challenge that > >>>>>>> window-header-store uses `ValueTimestampeHeader` while > >>>>>>> session-header-stores uses `AggregationWithHeaders` type. I would > >>>> like > >>>>>>> to understand better how this should all fit together. Also for > >> the > >>>>>>> case, that we add session-header-store support later (ie, we > >> should > >>>>>>> avoid to corner ourselves and ensure what we define now, is > >>>> extensible > >>>>>>> in the future). > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -Matthias > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On 6/11/26 6:23 AM, Jess Jin via dev wrote: > >>>>>>>> Hey Alieh, > >>>>>>>> > >>>>>>>> Thanks for the feedback! Updated in the wiki page. > >>>>>>>> > >>>>>>>> - Jess > >>>>>>>> > >>>>>>>> On Wed, Jun 10, 2026 at 3:26 PM Alieh Saeedi < > >>> [email protected] > >>>>> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hey Jess, > >>>>>>>>> > >>>>>>>>> Thanks for putting together the KIP. > >>>>>>>>> > >>>>>>>>> A couple of small nits: > >>>>>>>>> > >>>>>>>>> AS1: Please replace the `getX()` methods with `x()`. For > >>> example, > >>>>>>>>> `getKey()` should become `key()`. As far as I know, that’s the > >>>> usual > >>>>>>>>> convention. > >>>>>>>>> AS2: In the `Usage Example` section, please replace `long` > >> with > >>>>>> `Long`, > >>>>>>>>> since `ValueTimestampHeaders.value` is not a primitive. > >>>>>>>>> > >>>>>>>>> - Alieh > >>>>>>>>> > >>>>>>>>> On Wed, Jun 10, 2026 at 7:02 PM Jess Jin via dev < > >>>>>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> I'd like to start a discussion on KIP-1356: Introduce IQv2 > >> for > >>>>>>>>>> headers-aware state stores. Link > >>>>>>>>>> < > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>> > >>> > >> > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1356*3A*Introduce*IQv2*for*headers-aware*state*stores__;JSsrKysrKw!!Ayb5sqE7!oZFamFyZDrAmJeFHgKc-nOWUWL85bEbfHSdwgrp9jNHVOUCZgoOQ9aiK94L8zPr-tA_zx3WTmNs9vVDlqgQ$ > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>> > >>> > >> > > > >
