Hi Soumitra, We welcome AI, however there are currently no guidelines for using AI in FLIP or design. I think AIs often introduce unnecessary code details or repeated words, which makes the article lose its focus, leaving many people confused and reluctant to participate in the discussion. We need to minimize the reader's cognitive overhead. I suggest not diving into excessive detail, as many code details evolve flexibly during development. It would be best if we could clearly define the flink's public API and the use patterns, as well as present the overall technical architecture and benchmark results.
I have processed your application and you might have access to the wiki now. Please check. Best, Zakelly On Thu, Apr 23, 2026 at 7:41 PM Soumitra Kumar <[email protected]> wrote: > Hi Zakelly, > > I applied for wiki access. I will wait for a few days to get the approval. > If there is a delay, I will use google doc for the next email thread. > I used AI, but not to generate it. I will browse around for the AI use > policy and abide by that. If there is a doc, please send it. Since I have > the prototype, I could provide the details. In the next FLIP, I will be > concise and expand as reviewers ask. > > Looking forward, best, > -Soumitra. > > On Wed, Apr 22, 2026 at 11:13 PM Zakelly Lan <[email protected]> > wrote: > > > Hi Soumitra, > > > > Good to know. It's better to sign up for an account to access the wiki, > but > > google doc is also fine, up to you. And I suggest starting a separate > > discussion thread for this FLIP to attract more attention, since you > need 3 > > binding votes to get it passed. > > > > I still need some time to fully read your FLIP. Also, may I ask if any of > > this content was AI-generated? Could you provide a more concise overview > of > > the architecture and user examples, rather than focusing so heavily on > > minute code details? > > > > > > Best, > > Zakelly > > > > On Wed, Apr 22, 2026 at 12:33 PM Soumitra Kumar < > [email protected]> > > wrote: > > > > > Hi Zakelly, > > > > > > Since I don't have access to the wiki, > > > > > > > > > https://docs.google.com/document/d/1bBbqBC50zIHVRd3JAR4-bSrvmlOLkDGI7A60b2VZRkQ/edit?usp=sharing > > > is the FLIP as a Google doc. It is ready for your review. > > > > > > I am working on getting the JNI overhead and other plots you asked for. > > > > > > Best, > > > -Soumitra. > > > > > > On Thu, Apr 16, 2026 at 3:33 AM Zakelly Lan <[email protected]> > > wrote: > > > > > > > Hi Soumitra, > > > > > > > > It's a good improvement, In general I'm +1 on this. Several > questions / > > > > suggestions: > > > > > > > > 1. Could you please share the benchmark results in brief? I have seen > > > your > > > > perf's README but I could only find a throughput of 1,000 rec/s v.s. > > 500 > > > > rec/s comparison. I would like to know the state size and the usage > of > > > cpu > > > > or I/O across different setups. It would be even better if we could > > > > determine the overhead of the JNI. I'm also thinking that we may > > provide > > > > some built-in merge operators for common primitives like sum of long > > > > written in C on frocksdb side, to save the JNI overhead during > > > compaction. > > > > > > > > 2. I noticed that you are using the DataStream API for testing. So > the > > > > built-in windows cannot benefit from this improvement due to the > > > > retractions or late messages? > > > > > > > > 3. I can see there are some changes to the flink's public API in your > > > > branch, right? I thought that no public API changes would be > required. > > We > > > > need to maintain API signature compatibility. If the changes are > truly > > > > necessary, it is required to file a FLIP under [1] and collect > further > > > > comments from the mailing list. Once the FLIP has been approved > > through a > > > > vote, we can proceed. > > > > > > > > > > > > [1] > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > > > > > > > > Best, > > > > Zakelly > > > > > > > > On Thu, Apr 16, 2026 at 12:38 PM Soumitra Kumar < > > > [email protected]> > > > > wrote: > > > > > > > > > Hello Community, > > > > > > > > > > I wanted to share some work I have been doing on the frocksdb and > > Flink > > > > > that I think is useful for the Flink community. > > > > > > > > > > I have implemented support for Java base associative merge > operators > > in > > > > > https://github.com/ververica/frocksdb and use that to support > > > additional > > > > > reducing and aggregating state variables in Flink. I have used this > > do > > > > > event reordering in a flink app. All the code is in my github repo > ( > > > > > https://github.com/soumitrak) and I will be more than happy to > work > > > with > > > > > the members to contribute the code back to frocksdb and Flink. > > > > > > > > > > I have filed a followup task ( > > > > > https://issues.apache.org/jira/browse/FLINK-39456) to leverage the > > > > support > > > > > in frocksdb to expose the state variables in the Flink. > > > > > > > > > > Code in my forked repos: > > > > > https://github.com/soumitrak/frocksdb/commits/FRocksDB-8.10.0-SK/ > - > > > > > Created > > > > > a branch off FRocksDB-8.10.0 and committed the changes > > > > > https://github.com/soumitrak/flink/commits/rocksdb-merge-operator/ > - > > > > > Created a branch off master and committed the changes > > > > > https://github.com/soumitrak/flink_streaming_event_reordering - > Test > > > > > project used to build, test, perf test, and compare the performance > > of > > > > > heap-based, ValueState, and new MergeState (using associative merge > > > > > operator). > > > > > > > > > > I should have started two email threads, but they are related, so > > added > > > > the > > > > > details in one. > > > > > > > > > > Looking forward to guidance on how to go about this. > > > > > Best, -Soumitra. > > > > > > > > > > > > > > >
