Hi, Ramin Thanks for your response. The updated FLIP looks good to me.
Best, Ron Ramin Gharib <[email protected]> 于2025年10月10日周五 20:39写道： > Hi Ron, > > Thank you for the detailed and constructive feedback on the FLIP. These are > all excellent points. > > I have updated the FLIP's Confluence page, but I'd like to respond to each > of your questions here as well. > > 1. On the Default Freshness Value and Configuration > > I agree that 3 minutes is a more sensible and safer default than 1 minute. > To address this, I've updated the FLIP to include a new configuration > option, materialized-table.default-freshness, with a default value of 3 > min. > This provides the best of both worlds: a safer default for everyone and the > flexibility for administrators to configure it. > > 2. On the IntervalFreshness API Change (String to int) > > The FLIP now clarifies that we will use a non-breaking, "deprecate and add" > strategy. We will keep the existing String getInterval() method but mark it > as @Deprecated, and a new, type-safe int getIntervalInt() method will be > added. This ensures existing code continues to compile and function while > encouraging new development to use the safer method. I have updated the > FLIP (Section 3.2) to make this explicit. > > 3. On the Scope and Discovery of MaterializedTableEnricher > > To keep the initial scope of this FLIP focused and deliverable, I've > proposed that the framework will directly use the > DefaultMaterializedTableEnricher for now. This provides a complete, working > feature out of the box. The mechanism for discovering and configuring > *custom* enrichers (e.g., via a service loader or environment > configuration) is a crucial topic, but I believe it warrants its own > dedicated discussion in a follow-up FLIP. This allows us to land this > improvement pragmatically. > > 4. On RefreshContext > > You're right, the FLIP was missing the concrete design for this. I have > updated the FLIP to include the full class definition for RefreshContext in > the "Public Interfaces" section (now Section 3.6). It provides the enricher > with the resolved freshness, logical refresh mode, and the freshness > threshold from the configuration. > > Thanks again for helping to improve the proposal. Please let me know if > these updates and the revised FLIP address your questions. > > Best, > > Ramin > > On Thu, Oct 9, 2025 at 5:34 AM Ron Liu <[email protected]> wrote: > > > Hi, Ramin > > > > Thanks for initiating this FLIP., This proposal can lower the threshold > of > > using Materialized Table, +1. > > > > I read through the FLIP, and I have the following questions: > > > > 1. Regarding the default freshness default value, you mentioned that it > is > > 1min, how is this value determined? In continuous mode, we will convert > > freshness to checkpoint interval, in our internal best practice, the > > checkpoint interval is usually 3min. > > In addition, for data lake scenarios such as stream writing paimon, it is > > more reasonable to set the checkpoint interval to 3min. In our internal > > best practice, the checkpoint interval is usually 3min. > > I think this default value needs to be discussed, or similar to > > `materialized-table.refresh-mode.freshness-threshold`, we also provide an > > option and provide a default value, which can be configured by the user. > > > > 2. You mentioned that to improve IntervalFreshness, change the current > > string type to int for `interval`? The `getInterval()` public method is > > already used by Paimon and other external systems, if we change the > return > > value type, it will cause incompatible behavior. > > > > 3. About `MaterializedTableEnricher` interface, how to pass the > > user-defined implementation to the framework, i.e., how the framework > > perceives it? Also, does the scope of this interface support > customization > > at the table level, or the Catalog level? Or Service level? > > > > 4. Regarding RefreshContext, can you give a concrete design of the > > interface and what information it will provide to > > MaterializedTableEnricher? > > > > > > Best, > > Ron > > > > Ramin Gharib <[email protected]> 于2025年10月6日周一 20:19写道： > > > > > Hi everyone, > > > > > > I would like to start a discussion on FLIP-551 [1], which proposes > making > > > the FRESHNESS clause optional for Materialized Tables. > > > > > > The goal is to simplify the user experience for common streaming cases > > and > > > to enable more powerful, platform-level default logic. > > > > > > The proposal achieves this by: > > > > > > 1. > > > > > > Making the FRESHNESS DDL clause optional, falling back to a sensible > > > default of 1 minute. > > > 2. > > > > > > Introducing a new, MaterializedTableEnricher interface for custom > > logic > > > to resolve the final Freshness and RefreshMode into the > > > ResolvedCatalogMaterializedTable when they are omitted. > > > > > > This change reduces boilerplate for users while providing a clean > > extension > > > point for advanced use cases. The complete architectural details and > API > > > are in the FLIP. > > > > > > I'm looking forward to hearing your feedback. > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-551%3A+Make+FRESHNESS+Optional+for+Materialized+Tables > > > > > > Thanks, > > > > > > Ramin > > > > > >
