Hello Ron Liu! Thank you for your FLIP! Here are my considerations:
1. About the Operations interfaces, how can they be empty? Should not they provide at least a `run` or `execute` method (similar to the command pattern)? In this way, their implementation can wrap all the implementations details of particular schedulers, and the scheduler can simply execute the command. In general, I think a simple sequence diagram showcasing the interaction between the interfaces would be awesome to better understand the concept. 2. What about the RefreshHandler, I cannot find a definition of its interface here. Is it out of scope for this FLIP? 3. For the SqlGatewayService arguments: boolean isPeriodic, @Nullable String scheduleTime, @Nullable String scheduleTimeFormat, If it is periodic, where is the period? For the scheduleTime and format, why not simply pass an instance of LocalDateTime or similar? The gateway should not have the responsibility to parse the time. 4. For the REST API: wouldn't it be better (more REST) to move the `mt_identifier` to the URL? E.g.: v3/materialized_tables/<mt_identifier>/refresh Thank you! On Apr 22, 2024 at 08:42 +0200, Ron Liu <ron9....@gmail.com>, wrote: > Hi, Dev > > I would like to start a discussion about FLIP-448: Introduce Pluggable > Workflow Scheduler Interface for Materialized Table. > > In FLIP-435[1], we proposed Materialized Table, which has two types of data > refresh modes: Full Refresh & Continuous Refresh Mode. In Full Refresh > mode, the Materialized Table relies on a workflow scheduler to perform > periodic refresh operation to achieve the desired data freshness. > > There are numerous open-source workflow schedulers available, with popular > ones including Airflow and DolphinScheduler. To enable Materialized Table > to work with different workflow schedulers, we propose a pluggable workflow > scheduler interface for Materialized Table in this FLIP. > > For more details, see FLIP-448 [2]. Looking forward to your feedback. > > [1] https://lists.apache.org/thread/c1gnn3bvbfs8v1trlf975t327s4rsffs > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-448%3A+Introduce+Pluggable+Workflow+Scheduler+Interface+for+Materialized+Table > > Best, > Ron
