---------- Forwarded message --------- From: Soumitra Kumar <[email protected]> Date: Mon, Jun 15, 2026, 12:12 PM Subject: Re: [DISCUSS] FLIP-XXX Support ReducingMergeState and AggregatingMergeState backed by Java based associative merge operators To: <[email protected]>
Hi Roman, I replied to your questions a while back. Let me forward the thread to [email protected] . Best, -Soumitra. On Mon, Jun 15, 2026, 12:48 AM Roman Khachatryan <[email protected]> wrote: > Hello Soumitra Kumar, > > It would be great to get the answers to the questions above I posted - > unless the problem is solved and the FLIP isn't necessary. > > Regards, > Roman > > > On Sun, Jun 14, 2026 at 9:41 PM Soumitra Kumar <[email protected]> > wrote: > > > Hello Members, > > > > Thank you for your review so far. I don't have any open issues at this > > moment. Please let me know if there is any issue for me to > clarify/address. > > > > Best, > > -Soumitra. > > > > On Mon, Jun 8, 2026 at 10:09 PM Soumitra Kumar <[email protected] > > > > wrote: > > > > > Hi Han, > > > > > > I have added a section on TTL of ReducingMergeState and > > > AggregatingMergeState - HERE > > > < > > > https://docs.google.com/document/d/1HwEDRGoSZIUU1SYxTih4qp8FM6LjTdIrDs7CJHm4iB0/edit?tab=t.0#heading=h.mqp1qeixcg45 > > > > , > > > please review. > > > > > > Best, > > > -Soumitra. > > > > > > On Mon, Jun 1, 2026 at 11:02 PM Soumitra Kumar < > [email protected] > > > > > > wrote: > > > > > >> Hi Han, > > >> > > >> Thanks for your review and encouragement! > > >> > > >> #1 - Users can migrate from ReducingState to ReducingMergeState, but > it > > >> has to be a conscious decision knowing the rocksdb implication. We > > should > > >> plan to create a few howto docs monitoring and tuning rocksdb to get > the > > >> best out of the merge operators. Theoretically, it is possible to > build > > an > > >> automatic migration path, but I would not favor that because of the > > >> different runtime characteristics of ReducingState and > > ReducingMergeState. > > >> The checkpoints/savepoints for > ReducingMergeState/AggregatingMergeState > > >> state variables will serialize the Reduce/Aggregate function as well. > > >> > > >> #2 - "Will this introduce different semantics when State TTL is > enabled" > > >> - Can you elaborate on this? TBH, I have not planned the details of > the > > TTL > > >> of ReducingMergeState/AggregatingMergeState variables yet, but the TTL > > >> should be applied on the variable, not on individual operands. I will > > add a > > >> section on TTL of these variables in the FLIP. > > >> > > >> Best, > > >> -Soumitra. > > >> > > >> On Mon, Jun 1, 2026 at 3:03 AM Han Yin <[email protected]> wrote: > > >> > > >>> Hi Sumatra, > > >>> Thanks for the FLIP. The ability to leverage RocksDB merge operators > in > > >>> Reducing/Aggregating state is a really meaningful improvement. > > >>> > > >>> I share similar concerns about the user interface with the previous > > >>> comments: > > >>> • If new state types are introduced, can users migrate their > > >>> existing jobs from ReducingState to ReducingMergeState? Since the > core > > >>> logic of the ReduceFunction remains the same, one would expect a > > >>> straightforward migration path. If yes, will checkpoints/savepoints > > remain > > >>> compatible across this switch (and back)? > > >>> • Will this introduce different semantics when State TTL is > > enabled? > > >>> > > >>> > > >
