Hi, Dev
Thank you all for joining this thread and giving your comments and
suggestions, they have helped improve this proposal and I look forward to
further feedback.
If there are no further comments, I'd like to close the discussion and
start the voting one day later.
Best,
Ron
Ron Liu
Hi, dev
Following the recent PoC[1], and drawing on the excellent code design
within Flink, I have made the following optimizations to the Public
Interfaces section of FLIP:
1. I have renamed WorkflowOperation to RefreshWorkflow. This change better
conveys its purpose. RefreshWorkflow is used to
> 4. It appears that in the section on `public interfaces`, within
`WorkflowOperation`, `CreatePeriodicWorkflowOperation` should be changed to
`CreateWorkflowOperation`, right?
After discussing with Xuyang offline, we need to support periodic workflow
and one-time workflow, they need different
Hi, Xuyang
Thanks for joining this discussion
> 1. In the sequence diagram, it appears that there is a missing step for
obtaining the refresh handler from the catalog during the suspend operation.
Good catch
> 2. The term "cascade refresh" does not seem to be mentioned in FLIP-435.
The
Hi, Ron.
Thanks for driving this. After reading the entire flip, I have the following
questions:
1. In the sequence diagram, it appears that there is a missing step for
obtaining the refresh handler from the catalog during the suspend operation.
2. The term "cascade refresh" does not
Hi, Lincoln
Thanks for join this discussion.
After rethinking, I think your suggestion is make sense, although currently
deleting the workflow on the Scheduler and relying only on the
RefreshHandler is enough, if in the future we support cascading deletion,
the DeleteWorkflowOperation can
Thanks Ron for starting this flip! It will complete the user story for
flip-435[1].
Regarding the WorkflowOperation, I have a question about whether we
should add Delete/DropWorkflowOperation as well for when the
Materialized Table is dropped or refresh mode changed from full to
continuous?
[1]
Hello Ron, thank you for your detailed answers!
For the Visitor pattern, I thought about it the other way around, so that
operations visit the scheduler, and not vice-versa :) In this way operations
can get the required information in order to be executed in a tailored way.
Thank you for your
Hi, Lorenzo
> I have a question there: how can the gateway update the refreshHandler in
the Catalog before getting it from the scheduler?
The refreshHandler in CatalogMateriazedTable is null before getting it from
the scheduler, you can look at the CatalogMaterializedTable.Builder[1] for
more
Hi, Shengkai
Thanks for your feedback and suggestion, it looks very useful for this
proposal, regarding your question I made the following optimization:
> *WorkflowScheduler*
> 1. How to get the exception details if `modifyRefreshWorkflow` fails?
> 2. Could you give us an example about how to
Hi, Liu.
Thanks for your proposal. I have some questions about the FLIP:
*WorkflowScheduler*
1. How to get the exception details if `modifyRefreshWorkflow` fails?
2. Could you give us an example about how to configure the scheduler?
*SQL Gateway*
1. SqlGatewayService requires Session as the
Hello Ron!
> I've updated the FLIP, and added the Outline Design section, to introduce how
Materialized Table interacts with the Workflow Scheduler in Full Refresh
mode via a timing diagram, it can help to understand this proposal design.
Thank you for the additions, the sequence diagram says a
Hi, Lorenzo and Feng
Thanks for joining this thread discussing.
Sorry for later response, regarding your question:
> About the Operations interfaces, how can they be empty?
Should not they provide at least a `run` or `execute` method (similar to
the command pattern)?
In this way, their
Hi Ron
Thank you for initiating this FLIP.
My current questions are as follows:
1. From my current understanding, the workflow handle should not be bound
to the Dynamic Table. Therefore, if the workflow is modified, does it mean
that the scheduling information corresponding to the Dynamic Table
Hello Ron Liu! Thank you for your FLIP!
Here are my considerations:
1.
About the Operations interfaces, how can they be empty?
Should not they provide at least a `run` or `execute` method (similar to the
command pattern)?
In this way, their implementation can wrap all the implementations
Hi, Dev
I would like to start a discussion about FLIP-448: Introduce Pluggable
Workflow Scheduler Interface for Materialized Table.
In FLIP-435[1], we proposed Materialized Table, which has two types of data
refresh modes: Full Refresh & Continuous Refresh Mode. In Full Refresh
mode, the
16 matches
Mail list logo
