> 4. It appears that in the section on `public interfaces`, within `WorkflowOperation`, `CreatePeriodicWorkflowOperation` should be changed to
`CreateWorkflowOperation`, right? After discussing with Xuyang offline, we need to support periodic workflow and one-time workflow, they need different information, for example, periodic workflow needs cron expression, one-time workflow needs refresh partition, downstream cascade materialized table, etc. Therefore, CreateWorkflowOperation correspondingly will have two different implementation classes, which will be cleaner for both the implementer and the caller. Best, Ron Ron Liu <ron9....@gmail.com> 于2024年5月6日周一 20:48写道： > Hi, Xuyang > > Thanks for joining this discussion > > > 1. In the sequence diagram, it appears that there is a missing step for > obtaining the refresh handler from the catalog during the suspend operation. > > Good catch > > > 2. The term "cascade refresh" does not seem to be mentioned in FLIP-435. > The workflow it creates is marked as a "one-time workflow". This is > different > > from a "periodic workflow," and it appears to be a one-off execution. Is > this actually referring to the Refresh command in FLIP-435? > > The cascade refresh is a future work, we don't propose the corresponding > syntax in FLIP-435. However, intuitively, it would be an extension of the > Refresh command in FLIP-435. > > > 3. The workflow-scheduler.type has no default value; should it be set to > CRON by default? > > Firstly, CRON is not a workflow scheduler. Secondly, I believe that > configuring the Scheduler should be an action that users are aware of, and > default values should not be set. > > > 4. It appears that in the section on `public interfaces`, within > `WorkflowOperation`, `CreatePeriodicWorkflowOperation` should be changed to > > `CreateWorkflowOperation`, right? > > Sorry, I don't get your point. Can you give more description? > > Best, > Ron > > Xuyang <xyzhong...@163.com> 于2024年5月6日周一 20:26写道： > >> Hi, Ron. >> >> Thanks for driving this. After reading the entire flip, I have the >> following questions: >> >> >> >> >> 1. In the sequence diagram, it appears that there is a missing step for >> obtaining the refresh handler from the catalog during the suspend operation. >> >> >> >> >> 2. The term "cascade refresh" does not seem to be mentioned in FLIP-435. >> The workflow it creates is marked as a "one-time workflow". This is >> different >> >> from a "periodic workflow," and it appears to be a one-off execution. Is >> this actually referring to the Refresh command in FLIP-435? >> >> >> >> >> 3. The workflow-scheduler.type has no default value; should it be set to >> CRON by default? >> >> >> >> >> 4. It appears that in the section on `public interfaces`, within >> `WorkflowOperation`, `CreatePeriodicWorkflowOperation` should be changed to >> >> `CreateWorkflowOperation`, right? >> >> >> >> >> -- >> >> Best！ >> Xuyang >> >> >> >> >> >> At 2024-04-22 14:41:39, "Ron Liu" <ron9....@gmail.com> wrote: >> >Hi, Dev >> > >> >I would like to start a discussion about FLIP-448: Introduce Pluggable >> >Workflow Scheduler Interface for Materialized Table. >> > >> >In FLIP-435[1], we proposed Materialized Table, which has two types of >> data >> >refresh modes: Full Refresh & Continuous Refresh Mode. In Full Refresh >> >mode, the Materialized Table relies on a workflow scheduler to perform >> >periodic refresh operation to achieve the desired data freshness. >> > >> >There are numerous open-source workflow schedulers available, with >> popular >> >ones including Airflow and DolphinScheduler. To enable Materialized Table >> >to work with different workflow schedulers, we propose a pluggable >> workflow >> >scheduler interface for Materialized Table in this FLIP. >> > >> >For more details, see FLIP-448 [2]. Looking forward to your feedback. >> > >> >[1] https://lists.apache.org/thread/c1gnn3bvbfs8v1trlf975t327s4rsffs >> >[2] >> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-448%3A+Introduce+Pluggable+Workflow+Scheduler+Interface+for+Materialized+Table >> > >> >Best, >> >Ron >> >
