hi Warden,
Thanks for initiating this.
Since you mentioned interaction between core and dispatcher, I will start a
new thread attached with your protocol.
Thanks,
William
On Wed, Oct 12, 2022 at 10:48 AM Wang Warden <wangd95...@gmail.com> wrote:
> Hi,
>
> I have drafted the communication messages between core and dispatcher as
> below,
>
>
> // submit sql
> execute(RecordRequest) return RecordResponse;
>
> RecordRequest {
> String recordSql;
> Enum engine; // Spark,Hive,Presto,etc.
> String owner;
> Integer maxRetryCount;
> }
>
> RecordResponse {
> Integer code;
> Long value;
> Enum errorCode; // if code != 200, please tell us what happened:
> // 1、 recordSql syntax error
> // 2、 internal error, dispatcher self is crashed
> // 3、 external error, target engine is crashed when dispatcher
> call,etc
> Exception ex; // error detail info
> }
>
> // validate sql syntax
> validateSQL(CheckRequest) return CheckResponse;
>
> ValidateSQLRequest {
> String recordSql;
> Enum engine; // Spark,Hive,Presto,etc.
> }
>
> ValidateSQLResponse {
> Integer code;
> Enum errorCode; // if code != 200, please tell us what happened:
> // 1、 recordSql syntax error
> // 2、 internal error, dispatcher self is crashed
> // 3、 external error, target engine is crashed when dispatcher
> call,etc
> Exception ex; // error detail info
> }
>
>
>
> Could you please review it? Give us your feedback.
>
> Thanks,
> Wang
>
> Willem Jiang <willem.ji...@gmail.com> 於 2022年10月11日 週二 下午4:53寫道：
>
>> Great, thanks for the update.
>>
>> Willem Jiang
>>
>> Twitter: willemjiang
>> Weibo: 姜宁willem
>>
>> On Tue, Oct 11, 2022 at 1:13 PM William Guo <gu...@apache.org> wrote:
>> >
>> > hi Willem,
>> >
>> > I'm preparing the roadmap in our jira, will finish it tomorrow.
>> >
>> > Thanks,
>> > William
>> >
>> > On Tue, Oct 11, 2022 at 10:21 AM Willem Jiang <willem.ji...@gmail.com>
>> wrote:
>> >>
>> >> It is good to see we are planning a new version of the griffin.
>> >> I think we can write a roadmap to help users and other contributors to
>> >> know better about the whole process.
>> >>
>> >> Willem Jiang
>> >>
>> >> Twitter: willemjiang
>> >> Weibo: 姜宁willem
>> >>
>> >> On Mon, Oct 10, 2022 at 11:52 PM ding wang <wangd95...@gmail.com>
>> wrote:
>> >> >
>> >> > hi jianhua
>> >> >
>> >> > I would like to join to implement the core module, and since it
>> needs to
>> >> > communicate with dispatcher module.
>> >> >
>> >> > I will provide the communication protocol(interface) between core and
>> >> > dispatcher according to the architecture.
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Wang
>> >> >
>> >> > William Guo <gu...@apache.org> 於 2022年10月10日 週一 晚上7:42寫道：
>> >> >
>> >> > > Cool.
>> >> > >
>> >> > > Before we discuss, could you please document dispatcher design for
>> us to
>> >> > > review.
>> >> > >
>> >> > > Thanks,
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Mon, Oct 10, 2022 at 6:20 PM jianhua guo <guojhk...@gmail.com>
>> wrote:
>> >> > >
>> >> > > > hi all,
>> >> > > >
>> >> > > > I have reviewed the new arch, and will try to implement the
>> dispatcher
>> >> > > > module in the coming days.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>> >> > > >
>> >> > > > https://github.com/apache/griffin/tree/griffin-1.0.0-dev
>> >> > > >
>> >> > > > I need to discuss with you the dispatcher interface with the
>> core module.
>> >> > > >
>> >> > > >
>> >> > > > Thanks,
>> >> > > >
>> >> > > > Jianhua
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Wed, Aug 31, 2022 at 6:54 AM William Guo <gu...@apache.org>
>> wrote:
>> >> > > >
>> >> > > > > hi Eugene,
>> >> > > > >
>> >> > > > > 1. We need to seamlessly migrate from old tasks to new
>> workflows, some
>> >> > > > > modules will be optional in new workflows since old tasks lack
>> them.
>> >> > > > > 2. I will draw some diagrams to illustrate it.
>> >> > > > >
>> >> > > > > Thanks,
>> >> > > > > William
>> >> > > > >
>> >> > > > > On Sun, Aug 28, 2022 at 5:07 PM Eugene Law <liu...@apache.org>
>> wrote:
>> >> > > > >
>> >> > > > > > Hi William
>> >> > > > > >
>> >> > > > > > Raised a few comments on arch, replaying here
>> >> > > > > > 1.Do we need to consider upgrading from old arch to new
>> arch, like
>> >> > > how
>> >> > > > to
>> >> > > > > > seamlessly migrate old Griffin tasks into the new one?
>> >> > > > > > 2.The arch diagram is focusing on service relationships, can
>> you add
>> >> > > > > > component or deployment diagram to describe it from
>> different angles?
>> >> > > > > >
>> >> > > > > > Thx
>> >> > > > > >
>> >> > > > > > On Sun, Aug 28, 2022 at 2:31 PM William Guo <
>> gu...@apache.org>
>> >> > > wrote:
>> >> > > > > >
>> >> > > > > > > Hi Eugene,
>> >> > > > > > >
>> >> > > > > > > Granted.
>> >> > > > > > > Everyone can comment on our space.
>> >> > > > > > >
>> >> > > > > > > Thanks,
>> >> > > > > > > William
>> >> > > > > > >
>> >> > > > > > > On Sun, Aug 28, 2022 at 10:16 AM Eugene Law <
>> liu...@apache.org>
>> >> > > > wrote:
>> >> > > > > > >
>> >> > > > > > > > Hi William
>> >> > > > > > > >
>> >> > > > > > > > Could you grant comment permission on the confluence
>> page? so it
>> >> > > > > will
>> >> > > > > > > ease
>> >> > > > > > > > the conversations of proposal discussion among the
>> Griffin
>> >> > > > community.
>> >> > > > > > > >
>> >> > > > > > > > Thx
>> >> > > > > > > >
>> >> > > > > > > >
>> >> > > > > > > > On Fri, Aug 26, 2022 at 11:46 AM William Guo <
>> gu...@apache.org>
>> >> > > > > wrote:
>> >> > > > > > > >
>> >> > > > > > > > > hi all,
>> >> > > > > > > > >
>> >> > > > > > > > > Some dev might also be interested to review our
>> proposal for a
>> >> > > > new
>> >> > > > > > > arch.
>> >> > > > > > > > > Your thoughts?
>> >> > > > > > > > >
>> >> > > > > > > > > Please check this link to review.
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> https://cwiki.apache.org/confluence/display/GRIFFIN/The+DQ+workflow+Architecture+Proposal
>> >> > > > > > > > >
>> >> > > > > > > > > Thanks,
>> >> > > > > > > > > William
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>>
>
