Hi Jim, As discussed:
[3] Regarding the `ArchiveStorage` interface: the return type of the `get` method will remain generic, whereas the `put` method will retain the `String` type, since all Archive information is stored as JSON strings. I will subsequently split this into two separate FLIPs and incorporate the suggestions you outlined above. Best, Zihao Jim Hughes <[email protected]> 于2026年4月29日周三 23:14写道： > Hi Zihao, > > I don't know if you meant to reply off-list. Maybe so since you asked > about contacting Allison... I've tried a few times. I'll ping her again. > I don't think she is prioritizing working on this in her spare time and I > don't think it is related to her new job... > > Thanks for updating around the type. FWIW, you may want to post that on > the dev list for others to see. > > Cheers, > > Jim > > On Tue, Apr 28, 2026 at 10:52 PM zihao chen <[email protected]> wrote: > >> Hi Jim, >> >> Thanks again for helping reach out to the FLIP-505 author. It would be >> great to invite her to join the discussion so we can align the efforts more >> closely. >> >> Regarding the point about declaring the value type of put as String: the >> main reason is that the archive content we write is always JSON text, while >> the generic type parameter T represents the type used when reading. To >> avoid confusion, I can add a clearer explanation in the comments to make >> the intent explicit. If you have other suggestions or think a different >> approach would make the API more intuitive, I’m happy to adjust. >> >> Thanks again for the thoughtful feedback and for helping move this >> forward. >> >> Best, >> Zihao >> >> Jim Hughes <[email protected]> 于2026年4月28日周二 21:11写道： >> >>> Hi Zihao, >>> >>> All that makes sense. I've tried to reach out to Allison Chang about >>> the implementation for FLIP-505 as well. I ran into her at conference and >>> encouraged her to complete the work. >>> >>> [3] In that case, `put` needs to be updated to take a type `T`! >>> >>> Cheers, >>> >>> Jim >>> >>> On Tue, Apr 28, 2026 at 3:24 AM zihao chen <[email protected]> >>> wrote: >>> >>>> Hi Jim, >>>> >>>> Thanks a lot for the detailed feedback and the helpful pointers. >>>> >>>> [1][2] Yes, I have read through FLIP‑505 carefully, but it seems this >>>> work has not progressed further. I also emailed the authors regarding this >>>> discussion, but have not received a reply. >>>> Besides, from my understanding: >>>> >>>> - FLIP-505 focuses on remote store partial caching and per-job >>>> fetching (If the job has an excessive volume of archive files, users >>>> will >>>> still wait for all files are downloaded before visible) >>>> - This FLIP focuses on lazy parsing, overview/detail separation, >>>> and async loading >>>> >>>> These two approaches seem complementary rather than conflicting, and >>>> could potentially work well together. >>>> >>>> [3] Thanks for the suggestion — returning a String would indeed >>>> simplify the current implementation. >>>> However, after re-evaluating the design, I think keeping a generic >>>> return type is more appropriate for a pluggable storage abstraction, >>>> considering different storage backends may return different data types. >>>> >>>> [4] Good catch. In our production environment, the thread count is set >>>> relatively high to ensure the service works fast. However, regarding the >>>> randomness of user requests and applicability, there is indeed no need for >>>> the default configuration to be excessively large. The default thread count >>>> could be reduced to 4 or 8, and users could configure this setting to suit >>>> their specific requirements. What are your thoughts? >>>> >>>> Best regards, >>>> Zihao >>>> >>>> Jim Hughes via dev <[email protected]> 于2026年4月28日周二 02:03写道： >>>> >>>>> Hi Zihao, >>>>> >>>>> [1] Have you read through >>>>> >>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP+505%3A+Flink+History+Server+Scability+Improvements%2C+Remote+Data+Store+Fetch+and+Per+Job+Fetch >>>>> <https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*505*3A*Flink*History*Server*Scability*Improvements*2C*Remote*Data*Store*Fetch*and*Per*Job*Fetch__;KyUrKysrKyUrKysrKysrKw!!Ayb5sqE7!tcnOQwqe5ye5wag-miF6n2P9awG8nsDfweC-ogFUgtevGj1juo8gB_ErDLgJGtJp1j0QoRZ1WuZmH85aTFvWw1UD$> >>>>> ? >>>>> How does it relate to this work? >>>>> >>>>> [2] There are some PRs for the FLIP-505 work which have been open for >>>>> awhile here: >>>>> >>>>> https://github.com/apache/flink/pulls?q=is%3Apr+is%3Aopen+flip-505+37155 >>>>> <https://urldefense.com/v3/__https://github.com/apache/flink/pulls?q=is*3Apr*is*3Aopen*flip-505*37155__;JSslKys!!Ayb5sqE7!tcnOQwqe5ye5wag-miF6n2P9awG8nsDfweC-ogFUgtevGj1juo8gB_ErDLgJGtJp1j0QoRZ1WuZmH85aTF8KEq2Z$> >>>>> >>>>> [3] Shouldn't the `get` method return a `String`? >>>>> >>>>> [4] I'm surprised by the high default thread counts for the lazy >>>>> loader. I >>>>> suppose you have some experience tuning that number? >>>>> >>>>> The rest of it looks reasonable. >>>>> >>>>> Cheers, >>>>> >>>>> Jim >>>>> >>>>> On Mon, Mar 23, 2026 at 8:02 AM zihao chen <[email protected]> >>>>> wrote: >>>>> >>>>> > Hi all, >>>>> > >>>>> > I’d like to start a discussion on *FLIP-XXX: Enhancing HistoryServer >>>>> with >>>>> > Lazy Loading and Pluggable Storage*. >>>>> > >>>>> > This FLIP proposes improving the HistoryServer to address *slow >>>>> > startup*, *delayed >>>>> > job visibility*, and *excessive small files* when handling large >>>>> numbers of >>>>> > archived jobs. >>>>> > >>>>> > The proposal introduces: >>>>> > >>>>> > - *Lazy loading mode* for on-demand job archive fetching >>>>> > - Optional *RocksDB-based storage* to reduce small files >>>>> > >>>>> > Compatibility: Full backward compatibility (EAGER + FILE as default) >>>>> > >>>>> > The detailed designs are described in the FLIP document: >>>>> > >>>>> > >>>>> > >>>>> https://docs.google.com/document/d/1FQn-HQSIdhio3ImeOrbS9VfB6JqRTrRV4ZEZDTf3K2o/edit?usp=sharing >>>>> <https://urldefense.com/v3/__https://docs.google.com/document/d/1FQn-HQSIdhio3ImeOrbS9VfB6JqRTrRV4ZEZDTf3K2o/edit?usp=sharing__;!!Ayb5sqE7!tcnOQwqe5ye5wag-miF6n2P9awG8nsDfweC-ogFUgtevGj1juo8gB_ErDLgJGtJp1j0QoRZ1WuZmH85aTCn0Bfo4$> >>>>> > >>>>> > Looking forward to your feedback. >>>>> > >>>>> > >>>>> > Best regards, >>>>> > >>>>> > Zihao Chen >>>>> > >>>>> >>>>
