---------- Forwarded message --------- From: Tushar Patil <tushar.opensourc...@gmail.com> Date: Tue, Aug 9, 2022 at 9:14 PM Subject: Re: [ROLL CALL] Status of this project To: William Guo <gu...@apache.org>
Hi William, I will be happy to contribute in this new architecture development. How can I pickup development task for the same ? Regards, Tushar > On 09-Aug-2022, at 5:54 PM, William Guo <gu...@apache.org> wrote: > > hi all, > > This is our new arch diagram > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > We will build milestones based on this proposal. > > Thanks, > William > > On Tue, Aug 9, 2022 at 8:22 PM William Guo <gu...@apache.org> wrote: > >> hi >> >> test for new subscriber, please ignore this. >> >> >> Thanks, >> William >> >> On Mon, Aug 8, 2022 at 6:11 PM William Guo <gu...@apache.org> wrote: >> >>> hi All, >>> >>> Since our pmcs have reached an agreement, we will draft our feature >>> roadmap pretty soon. >>> >>> @Willem Jiang <willem.ji...@gmail.com> >>> We will keep you posted. >>> >>> Thanks, >>> William >>> >>> On Mon, Aug 8, 2022 at 5:30 PM Willem Ning Jiang <ningji...@apache.org> >>> wrote: >>> >>>> It looks like there is a new generation of Griffin. >>>> @William would you wrap up a roadmap for it? >>>> >>>> Thanks, >>>> >>>> Willem >>>> >>>> On 2022/08/06 08:16:49 William Guo wrote: >>>>> Any comments from other PMCs? >>>>> >>>>> We need a full discussion before we kick off our next milestone for >>>> apache >>>>> griffin. >>>>> >>>>> Thanks, >>>>> William >>>>> >>>>> >>>>> On Fri, Jul 29, 2022 at 9:55 PM john@apache <john...@apache.org> >>>> wrote: >>>>> >>>>>> Could not agree more with Eugene's summary. >>>>>> >>>>>> Just had a quick meeting with William, talking his arch draft. Now we >>>>>> have a clarified and clear defined goal for the new arch. >>>>>> >>>>>> William and I all agree that it would be great to have a simple and >>>>>> clearly defined goal to go with. >>>>>> >>>>>> ---- >>>>>> >>>>>> # What Griffin should focus >>>>>> >>>>>> Griffin is a generic framework to enable user to measure and monitor >>>>>> the data quality in an easy and extensive manner. >>>>>> >>>>>> # Old Griffin architecture problems >>>>>> >>>>>> What's the painpoints we are facing in the current edition of Griffin >>>>>> from the architectural perspective? >>>>>> >>>>>> 1. To have an abstraction layer of query engine >>>>>> The current spark implementation limited Griffin's adoption, since a >>>>>> company could use other query engine in company level. It's too heavy >>>>>> to set up a spark environment to run Griffin. >>>>>> >>>>>> 2. To support a common data quality workflow: measure-monitor-alert >>>>>> In most of real scenarios, measuring is not the goal, but >>>>>> measure-monitor-alert workflow. >>>>>> User should be able to define an use case, including: >>>>>> 1. to measure a dq metrics >>>>>> 2. to define the monitoring trigger >>>>>> 3. to define the alerting action >>>>>> 4. to integrate the 3-steps into a single job/UoW (by scheduler) >>>>>> >>>>>> On Sun, Jul 24, 2022 at 4:50 PM Eugene Law <liu...@apache.org> >>>> wrote: >>>>>>> >>>>>>> Appreciate William post of new architecture. John & Grant had >>>> raised good >>>>>>> comments as well. >>>>>>> >>>>>>> As an evolving data quality product, we need to catch up to the >>>> rolling >>>>>>> forward of the big data ecosystem. I think there are points we can >>>>>>> consider more. >>>>>>> >>>>>>> Firstly, how to absorb positive/negative experience from existent >>>> jira >>>>>> bugs >>>>>>> and make it apply in new architecture. >>>>>>> Secondly, how to highlight the improvement from the old one to the >>>> new >>>>>>> architecture and convince Grffin's users that the change will >>>> bring more >>>>>>> gains. >>>>>>> Thirdly, we should consider making rule-base architecture evolving >>>> as a >>>>>>> long-term goal, finally becoming a de facto standard in the >>>> vertical >>>>>> area. >>>>>>> >>>>>>> Thx >>>>>>> Eugene >>>>>>> >>>>>>> >>>>>>> On Sun, Jul 24, 2022 at 4:00 PM William Guo <gu...@apache.org> >>>> wrote: >>>>>>> >>>>>>>> just created the GIP pages and moved this proposal under GIP, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>> https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal >>>>>>>> >>>>>>>> Thanks, >>>>>>>> William >>>>>>>> >>>>>>>> On Sat, Jul 23, 2022 at 3:11 PM William Guo <gu...@apache.org> >>>> wrote: >>>>>>>> >>>>>>>>> I have drafted next generation griffin architecture. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal >>>>>>>>> >>>>>>>>> Let's brainstorm it . >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> William >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Jul 23, 2022 at 10:28 AM john@apache < >>>> john...@apache.org> >>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Great to hear that we will have something new in the plan. >>>>>>>>>> >>>>>>>>>> It is so happy to me to see that there will be new arch draft >>>> in 2 >>>>>> days. >>>>>>>>>> >>>>>>>>>> At the same time, I would like to raise up some of my >>>> ideas/comments >>>>>>>>>> related to the new arch before we decided to go >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 1. What’s the good part and bad part for the existing >>>> arch? I >>>>>> agree >>>>>>>>>> that the current arch which were defined 4 years ago is not >>>> good >>>>>>>>>> enough for the ongoing requirement. So I would like to >>>> recommend >>>>>> that >>>>>>>>>> compile a list of expected change from arch level, so we can >>>> start >>>>>>>>>> from there, to discuss the priority and scope. >>>>>>>>>> 2. We might need to define/re-define high level principles >>>>>> helping >>>>>>>>>> us clarify the vision, if it is not clear enough now. We need >>>> some >>>>>>>>>> brain storming to collect more ideas about what griffin >>>> should do >>>>>> and >>>>>>>>>> what should not. >>>>>>>>>> 3. We need to check with what is the main pain points of >>>> current >>>>>>>>>> user, and try to solve them in the arch level. I will go >>>> through all >>>>>>>>>> the jira issues and classify them to several categories. And >>>> I will >>>>>>>>>> keep update the status. >>>>>>>>>> >>>>>>>>>> Please let me know your comments. >>>>>>>>>> >>>>>>>>>> On Sat, Jul 23, 2022 at 12:53 AM Grant <grant.xu...@gmail.com >>>>> >>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Willem, >>>>>>>>>>> >>>>>>>>>>> I am Grant Guo, PMC member of the Griffin project, apache >>>> ID: >>>>>> xuexu >>>>>>>>>>> >>>>>>>>>>> Since last December, William, me, and other PMC members >>>> have been >>>>>>>>>>> discussing the feasibility of the rule-based solution, >>>> which could >>>>>>>> bring >>>>>>>>>>> Griffin to a new level in terms of usability, >>>> extensibility, etc. >>>>>>>>>>> >>>>>>>>>>> A new architectural proposal will be posted soon. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> On Wed, Jul 20, 2022 at 6:08 PM Willem Jiang < >>>>>> willem.ji...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> There has been no development activity on this project >>>> for more >>>>>>>> than 5 >>>>>>>>>>>> months. The PMC has failed to submit several board >>>> reports. >>>>>> Without >>>>>>>> a >>>>>>>>>>>> community or anyone working on the project, there is no >>>> project. >>>>>>>>>>>> >>>>>>>>>>>> If any of the PMC members are still active, please >>>> indicate so >>>>>> by >>>>>>>>>>>> responding to this email. >>>>>>>>>>>> >>>>>>>>>>>> Thank you, all! I hope everyone is safe and healthy. >>>>>>>>>>>> >>>>>>>>>>>> Willem Jiang >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>
