Hi, Ron.
Thanks for driving this. After reading the entire flip, I have the following
questions:
1. In the sequence diagram, it appears that there is a missing step for
obtaining the refresh handler from the catalog during the suspend operation.
2. The term "cascade refresh" does not seem to be mentioned in FLIP-435. The
workflow it creates is marked as a "one-time workflow". This is different
from a "periodic workflow," and it appears to be a one-off execution. Is this
actually referring to the Refresh command in FLIP-435?
3. The workflow-scheduler.type has no default value; should it be set to CRON
by default?
4. It appears that in the section on `public interfaces`, within
`WorkflowOperation`, `CreatePeriodicWorkflowOperation` should be changed to
`CreateWorkflowOperation`, right?
--
Best！
Xuyang
At 2024-04-22 14:41:39, "Ron Liu" <ron9....@gmail.com> wrote:
>Hi, Dev
>
>I would like to start a discussion about FLIP-448: Introduce Pluggable
>Workflow Scheduler Interface for Materialized Table.
>
>In FLIP-435[1], we proposed Materialized Table, which has two types of data
>refresh modes: Full Refresh & Continuous Refresh Mode. In Full Refresh
>mode, the Materialized Table relies on a workflow scheduler to perform
>periodic refresh operation to achieve the desired data freshness.
>
>There are numerous open-source workflow schedulers available, with popular
>ones including Airflow and DolphinScheduler. To enable Materialized Table
>to work with different workflow schedulers, we propose a pluggable workflow
>scheduler interface for Materialized Table in this FLIP.
>
>For more details, see FLIP-448 [2]. Looking forward to your feedback.
>
>[1] https://lists.apache.org/thread/c1gnn3bvbfs8v1trlf975t327s4rsffs
>[2]
>https://cwiki.apache.org/confluence/display/FLINK/FLIP-448%3A+Introduce+Pluggable+Workflow+Scheduler+Interface+for+Materialized+Table
>
>Best,
>Ron
