Hi, Ramin Thank you for proposing this FLIP—it will indeed enhance convenience.
However, regarding Full mode and Continuous mode, should we set different default values for "freshness"? If a user explicitly selects Full mode, should we configure a longer default freshness period instead? Best, Feng On Sat, Oct 11, 2025 at 12:02 PM Ron Liu <[email protected]> wrote: > Hi, Ramin > > Thanks for your response. The updated FLIP looks good to me. > > Best, > Ron > > Ramin Gharib <[email protected]> 于2025年10月10日周五 20:39写道： > > > Hi Ron, > > > > Thank you for the detailed and constructive feedback on the FLIP. These > are > > all excellent points. > > > > I have updated the FLIP's Confluence page, but I'd like to respond to > each > > of your questions here as well. > > > > 1. On the Default Freshness Value and Configuration > > > > I agree that 3 minutes is a more sensible and safer default than 1 > minute. > > To address this, I've updated the FLIP to include a new configuration > > option, materialized-table.default-freshness, with a default value of 3 > > min. > > This provides the best of both worlds: a safer default for everyone and > the > > flexibility for administrators to configure it. > > > > 2. On the IntervalFreshness API Change (String to int) > > > > The FLIP now clarifies that we will use a non-breaking, "deprecate and > add" > > strategy. We will keep the existing String getInterval() method but mark > it > > as @Deprecated, and a new, type-safe int getIntervalInt() method will be > > added. This ensures existing code continues to compile and function while > > encouraging new development to use the safer method. I have updated the > > FLIP (Section 3.2) to make this explicit. > > > > 3. On the Scope and Discovery of MaterializedTableEnricher > > > > To keep the initial scope of this FLIP focused and deliverable, I've > > proposed that the framework will directly use the > > DefaultMaterializedTableEnricher for now. This provides a complete, > working > > feature out of the box. The mechanism for discovering and configuring > > *custom* enrichers (e.g., via a service loader or environment > > configuration) is a crucial topic, but I believe it warrants its own > > dedicated discussion in a follow-up FLIP. This allows us to land this > > improvement pragmatically. > > > > 4. On RefreshContext > > > > You're right, the FLIP was missing the concrete design for this. I have > > updated the FLIP to include the full class definition for RefreshContext > in > > the "Public Interfaces" section (now Section 3.6). It provides the > enricher > > with the resolved freshness, logical refresh mode, and the freshness > > threshold from the configuration. > > > > Thanks again for helping to improve the proposal. Please let me know if > > these updates and the revised FLIP address your questions. > > > > Best, > > > > Ramin > > > > On Thu, Oct 9, 2025 at 5:34 AM Ron Liu <[email protected]> wrote: > > > > > Hi, Ramin > > > > > > Thanks for initiating this FLIP., This proposal can lower the threshold > > of > > > using Materialized Table, +1. > > > > > > I read through the FLIP, and I have the following questions: > > > > > > 1. Regarding the default freshness default value, you mentioned that it > > is > > > 1min, how is this value determined? In continuous mode, we will convert > > > freshness to checkpoint interval, in our internal best practice, the > > > checkpoint interval is usually 3min. > > > In addition, for data lake scenarios such as stream writing paimon, it > is > > > more reasonable to set the checkpoint interval to 3min. In our internal > > > best practice, the checkpoint interval is usually 3min. > > > I think this default value needs to be discussed, or similar to > > > `materialized-table.refresh-mode.freshness-threshold`, we also provide > an > > > option and provide a default value, which can be configured by the > user. > > > > > > 2. You mentioned that to improve IntervalFreshness, change the current > > > string type to int for `interval`? The `getInterval()` public method > is > > > already used by Paimon and other external systems, if we change the > > return > > > value type, it will cause incompatible behavior. > > > > > > 3. About `MaterializedTableEnricher` interface, how to pass the > > > user-defined implementation to the framework, i.e., how the framework > > > perceives it? Also, does the scope of this interface support > > customization > > > at the table level, or the Catalog level? Or Service level? > > > > > > 4. Regarding RefreshContext, can you give a concrete design of the > > > interface and what information it will provide to > > > MaterializedTableEnricher? > > > > > > > > > Best, > > > Ron > > > > > > Ramin Gharib <[email protected]> 于2025年10月6日周一 20:19写道： > > > > > > > Hi everyone, > > > > > > > > I would like to start a discussion on FLIP-551 [1], which proposes > > making > > > > the FRESHNESS clause optional for Materialized Tables. > > > > > > > > The goal is to simplify the user experience for common streaming > cases > > > and > > > > to enable more powerful, platform-level default logic. > > > > > > > > The proposal achieves this by: > > > > > > > > 1. > > > > > > > > Making the FRESHNESS DDL clause optional, falling back to a > sensible > > > > default of 1 minute. > > > > 2. > > > > > > > > Introducing a new, MaterializedTableEnricher interface for custom > > > logic > > > > to resolve the final Freshness and RefreshMode into the > > > > ResolvedCatalogMaterializedTable when they are omitted. > > > > > > > > This change reduces boilerplate for users while providing a clean > > > extension > > > > point for advanced use cases. The complete architectural details and > > API > > > > are in the FLIP. > > > > > > > > I'm looking forward to hearing your feedback. > > > > > > > > [1] > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-551%3A+Make+FRESHNESS+Optional+for+Materialized+Tables > > > > > > > > Thanks, > > > > > > > > Ramin > > > > > > > > > >
