Hi,
I have drafted the communication messages between core and dispatcher as
below,
// submit sql
execute(RecordRequest) return RecordResponse;
RecordRequest {
String recordSql;
Enum engine; // Spark,Hive,Presto,etc.
String owner;
Integer maxRetryCount;
}
RecordResponse {
Integer code;
Long value;
Enum errorCode; // if code != 200, please tell us what happened:
// 1、 recordSql syntax error
// 2、 internal error, dispatcher self is crashed
// 3、 external error, target engine is crashed when dispatcher
call,etc
Exception ex; // error detail info
}
// validate sql syntax
validateSQL(CheckRequest) return CheckResponse;
ValidateSQLRequest {
String recordSql;
Enum engine; // Spark,Hive,Presto,etc.
}
ValidateSQLResponse {
Integer code;
Enum errorCode; // if code != 200, please tell us what happened:
// 1、 recordSql syntax error
// 2、 internal error, dispatcher self is crashed
// 3、 external error, target engine is crashed when dispatcher
call,etc
Exception ex; // error detail info
}
Could you please review it? Give us your feedback.
Thanks,
Wang
Willem Jiang <[email protected]> 於 2022年10月11日 週二 下午4:53寫道：
> Great, thanks for the update.
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Tue, Oct 11, 2022 at 1:13 PM William Guo <[email protected]> wrote:
> >
> > hi Willem,
> >
> > I'm preparing the roadmap in our jira, will finish it tomorrow.
> >
> > Thanks,
> > William
> >
> > On Tue, Oct 11, 2022 at 10:21 AM Willem Jiang <[email protected]>
> wrote:
> >>
> >> It is good to see we are planning a new version of the griffin.
> >> I think we can write a roadmap to help users and other contributors to
> >> know better about the whole process.
> >>
> >> Willem Jiang
> >>
> >> Twitter: willemjiang
> >> Weibo: 姜宁willem
> >>
> >> On Mon, Oct 10, 2022 at 11:52 PM ding wang <[email protected]>
> wrote:
> >> >
> >> > hi jianhua
> >> >
> >> > I would like to join to implement the core module, and since it needs
> to
> >> > communicate with dispatcher module.
> >> >
> >> > I will provide the communication protocol(interface) between core and
> >> > dispatcher according to the architecture.
> >> >
> >> > Thanks,
> >> >
> >> > Wang
> >> >
> >> > William Guo <[email protected]> 於 2022年10月10日 週一 晚上7:42寫道：
> >> >
> >> > > Cool.
> >> > >
> >> > > Before we discuss, could you please document dispatcher design for
> us to
> >> > > review.
> >> > >
> >> > > Thanks,
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <[email protected]>
> wrote:
> >> > >
> >> > > > hi all,
> >> > > >
> >> > > > I have reviewed the new arch, and will try to implement the
> dispatcher
> >> > > > module in the coming days.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> >> > > >
> >> > > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
> >> > > >
> >> > > > I need to discuss with you the dispatcher interface with the core
> module.
> >> > > >
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Jianhua
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <[email protected]>
> wrote:
> >> > > >
> >> > > > > hi Eugene,
> >> > > > >
> >> > > > > 1. We need to seamlessly migrate from old tasks to new
> workflows, some
> >> > > > > modules will be optional in new workflows since old tasks lack
> them.
> >> > > > > 2. I will draw some diagrams to illustrate it.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > William
> >> > > > >
> >> > > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <[email protected]>
> wrote:
> >> > > > >
> >> > > > > > Hi William
> >> > > > > >
> >> > > > > > Raised a few comments on arch, replaying here
> >> > > > > > 1.Do we need to consider upgrading from old arch to new arch,
> like
> >> > > how
> >> > > > to
> >> > > > > > seamlessly migrate old Griffin tasks into the new one?
> >> > > > > > 2.The arch diagram is focusing on service relationships, can
> you add
> >> > > > > > component or deployment diagram to describe it from different
> angles?
> >> > > > > >
> >> > > > > > Thx
> >> > > > > >
> >> > > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <[email protected]
> >
> >> > > wrote:
> >> > > > > >
> >> > > > > > > Hi Eugene,
> >> > > > > > >
> >> > > > > > > Granted.
> >> > > > > > > Everyone can comment on our space.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > William
> >> > > > > > >
> >> > > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <
> [email protected]>
> >> > > > wrote:
> >> > > > > > >
> >> > > > > > > > Hi William
> >> > > > > > > >
> >> > > > > > > > Could you grant comment permission on the confluence
> page? so it
> >> > > > > will
> >> > > > > > > ease
> >> > > > > > > > the conversations of proposal discussion among the Griffin
> >> > > > community.
> >> > > > > > > >
> >> > > > > > > > Thx
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <
> [email protected]>
> >> > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > hi all,
> >> > > > > > > > >
> >> > > > > > > > > Some dev might also be interested to review our
> proposal for a
> >> > > > new
> >> > > > > > > arch.
> >> > > > > > > > > Your thoughts?
> >> > > > > > > > >
> >> > > > > > > > > Please check this link to review.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > William
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
>
