hi Tushar, We will put our development tasks on jira as https://issues.apache.org/jira/browse/GRIFFIN-371
You can pickup task from there, BTW, welcome to contribute! Thanks, William On Wed, Aug 10, 2022 at 7:12 PM William Guo <gu...@apache.org> wrote: > > > ---------- Forwarded message --------- > From: Tushar Patil <tushar.opensourc...@gmail.com> > Date: Tue, Aug 9, 2022 at 9:14 PM > Subject: Re: [ROLL CALL] Status of this project > To: William Guo <gu...@apache.org> > > > Hi William, > > I will be happy to contribute in this new architecture development. How > can I pickup development task for the same ? > > Regards, > Tushar > > > On 09-Aug-2022, at 5:54 PM, William Guo <gu...@apache.org> wrote: > > > > hi all, > > > > This is our new arch diagram > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > We will build milestones based on this proposal. > > > > Thanks, > > William > > > > On Tue, Aug 9, 2022 at 8:22 PM William Guo <gu...@apache.org> wrote: > > > >> hi > >> > >> test for new subscriber, please ignore this. > >> > >> > >> Thanks, > >> William > >> > >> On Mon, Aug 8, 2022 at 6:11 PM William Guo <gu...@apache.org> wrote: > >> > >>> hi All, > >>> > >>> Since our pmcs have reached an agreement, we will draft our feature > >>> roadmap pretty soon. > >>> > >>> @Willem Jiang <willem.ji...@gmail.com> > >>> We will keep you posted. > >>> > >>> Thanks, > >>> William > >>> > >>> On Mon, Aug 8, 2022 at 5:30 PM Willem Ning Jiang <ningji...@apache.org > > > >>> wrote: > >>> > >>>> It looks like there is a new generation of Griffin. > >>>> @William would you wrap up a roadmap for it? > >>>> > >>>> Thanks, > >>>> > >>>> Willem > >>>> > >>>> On 2022/08/06 08:16:49 William Guo wrote: > >>>>> Any comments from other PMCs? > >>>>> > >>>>> We need a full discussion before we kick off our next milestone for > >>>> apache > >>>>> griffin. > >>>>> > >>>>> Thanks, > >>>>> William > >>>>> > >>>>> > >>>>> On Fri, Jul 29, 2022 at 9:55 PM john@apache <john...@apache.org> > >>>> wrote: > >>>>> > >>>>>> Could not agree more with Eugene's summary. > >>>>>> > >>>>>> Just had a quick meeting with William, talking his arch draft. Now > we > >>>>>> have a clarified and clear defined goal for the new arch. > >>>>>> > >>>>>> William and I all agree that it would be great to have a simple and > >>>>>> clearly defined goal to go with. > >>>>>> > >>>>>> ---- > >>>>>> > >>>>>> # What Griffin should focus > >>>>>> > >>>>>> Griffin is a generic framework to enable user to measure and monitor > >>>>>> the data quality in an easy and extensive manner. > >>>>>> > >>>>>> # Old Griffin architecture problems > >>>>>> > >>>>>> What's the painpoints we are facing in the current edition of > Griffin > >>>>>> from the architectural perspective? > >>>>>> > >>>>>> 1. To have an abstraction layer of query engine > >>>>>> The current spark implementation limited Griffin's adoption, since a > >>>>>> company could use other query engine in company level. It's too > heavy > >>>>>> to set up a spark environment to run Griffin. > >>>>>> > >>>>>> 2. To support a common data quality workflow: measure-monitor-alert > >>>>>> In most of real scenarios, measuring is not the goal, but > >>>>>> measure-monitor-alert workflow. > >>>>>> User should be able to define an use case, including: > >>>>>> 1. to measure a dq metrics > >>>>>> 2. to define the monitoring trigger > >>>>>> 3. to define the alerting action > >>>>>> 4. to integrate the 3-steps into a single job/UoW (by scheduler) > >>>>>> > >>>>>> On Sun, Jul 24, 2022 at 4:50 PM Eugene Law <liu...@apache.org> > >>>> wrote: > >>>>>>> > >>>>>>> Appreciate William post of new architecture. John & Grant had > >>>> raised good > >>>>>>> comments as well. > >>>>>>> > >>>>>>> As an evolving data quality product, we need to catch up to the > >>>> rolling > >>>>>>> forward of the big data ecosystem. I think there are points we can > >>>>>>> consider more. > >>>>>>> > >>>>>>> Firstly, how to absorb positive/negative experience from existent > >>>> jira > >>>>>> bugs > >>>>>>> and make it apply in new architecture. > >>>>>>> Secondly, how to highlight the improvement from the old one to the > >>>> new > >>>>>>> architecture and convince Grffin's users that the change will > >>>> bring more > >>>>>>> gains. > >>>>>>> Thirdly, we should consider making rule-base architecture evolving > >>>> as a > >>>>>>> long-term goal, finally becoming a de facto standard in the > >>>> vertical > >>>>>> area. > >>>>>>> > >>>>>>> Thx > >>>>>>> Eugene > >>>>>>> > >>>>>>> > >>>>>>> On Sun, Jul 24, 2022 at 4:00 PM William Guo <gu...@apache.org> > >>>> wrote: > >>>>>>> > >>>>>>>> just created the GIP pages and moved this proposal under GIP, > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>> > >>>> > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> William > >>>>>>>> > >>>>>>>> On Sat, Jul 23, 2022 at 3:11 PM William Guo <gu...@apache.org> > >>>> wrote: > >>>>>>>> > >>>>>>>>> I have drafted next generation griffin architecture. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>> > >>>> > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > >>>>>>>>> > >>>>>>>>> Let's brainstorm it . > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> William > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Sat, Jul 23, 2022 at 10:28 AM john@apache < > >>>> john...@apache.org> > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> Great to hear that we will have something new in the plan. > >>>>>>>>>> > >>>>>>>>>> It is so happy to me to see that there will be new arch draft > >>>> in 2 > >>>>>> days. > >>>>>>>>>> > >>>>>>>>>> At the same time, I would like to raise up some of my > >>>> ideas/comments > >>>>>>>>>> related to the new arch before we decided to go > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> 1. What’s the good part and bad part for the existing > >>>> arch? I > >>>>>> agree > >>>>>>>>>> that the current arch which were defined 4 years ago is not > >>>> good > >>>>>>>>>> enough for the ongoing requirement. So I would like to > >>>> recommend > >>>>>> that > >>>>>>>>>> compile a list of expected change from arch level, so we can > >>>> start > >>>>>>>>>> from there, to discuss the priority and scope. > >>>>>>>>>> 2. We might need to define/re-define high level principles > >>>>>> helping > >>>>>>>>>> us clarify the vision, if it is not clear enough now. We need > >>>> some > >>>>>>>>>> brain storming to collect more ideas about what griffin > >>>> should do > >>>>>> and > >>>>>>>>>> what should not. > >>>>>>>>>> 3. We need to check with what is the main pain points of > >>>> current > >>>>>>>>>> user, and try to solve them in the arch level. I will go > >>>> through all > >>>>>>>>>> the jira issues and classify them to several categories. And > >>>> I will > >>>>>>>>>> keep update the status. > >>>>>>>>>> > >>>>>>>>>> Please let me know your comments. > >>>>>>>>>> > >>>>>>>>>> On Sat, Jul 23, 2022 at 12:53 AM Grant <grant.xu...@gmail.com > >>>>> > >>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hi Willem, > >>>>>>>>>>> > >>>>>>>>>>> I am Grant Guo, PMC member of the Griffin project, apache > >>>> ID: > >>>>>> xuexu > >>>>>>>>>>> > >>>>>>>>>>> Since last December, William, me, and other PMC members > >>>> have been > >>>>>>>>>>> discussing the feasibility of the rule-based solution, > >>>> which could > >>>>>>>> bring > >>>>>>>>>>> Griffin to a new level in terms of usability, > >>>> extensibility, etc. > >>>>>>>>>>> > >>>>>>>>>>> A new architectural proposal will be posted soon. > >>>>>>>>>>> > >>>>>>>>>>> Thanks > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Jul 20, 2022 at 6:08 PM Willem Jiang < > >>>>>> willem.ji...@gmail.com> > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hi, > >>>>>>>>>>>> > >>>>>>>>>>>> There has been no development activity on this project > >>>> for more > >>>>>>>> than 5 > >>>>>>>>>>>> months. The PMC has failed to submit several board > >>>> reports. > >>>>>> Without > >>>>>>>> a > >>>>>>>>>>>> community or anyone working on the project, there is no > >>>> project. > >>>>>>>>>>>> > >>>>>>>>>>>> If any of the PMC members are still active, please > >>>> indicate so > >>>>>> by > >>>>>>>>>>>> responding to this email. > >>>>>>>>>>>> > >>>>>>>>>>>> Thank you, all! I hope everyone is safe and healthy. > >>>>>>>>>>>> > >>>>>>>>>>>> Willem Jiang > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >
