It looks like there is a new generation of Griffin. @William would you wrap up a roadmap for it?
Thanks, Willem On 2022/08/06 08:16:49 William Guo wrote: > Any comments from other PMCs? > > We need a full discussion before we kick off our next milestone for apache > griffin. > > Thanks, > William > > > On Fri, Jul 29, 2022 at 9:55 PM john@apache <john...@apache.org> wrote: > > > Could not agree more with Eugene's summary. > > > > Just had a quick meeting with William, talking his arch draft. Now we > > have a clarified and clear defined goal for the new arch. > > > > William and I all agree that it would be great to have a simple and > > clearly defined goal to go with. > > > > ---- > > > > # What Griffin should focus > > > > Griffin is a generic framework to enable user to measure and monitor > > the data quality in an easy and extensive manner. > > > > # Old Griffin architecture problems > > > > What's the painpoints we are facing in the current edition of Griffin > > from the architectural perspective? > > > > 1. To have an abstraction layer of query engine > > The current spark implementation limited Griffin's adoption, since a > > company could use other query engine in company level. It's too heavy > > to set up a spark environment to run Griffin. > > > > 2. To support a common data quality workflow: measure-monitor-alert > > In most of real scenarios, measuring is not the goal, but > > measure-monitor-alert workflow. > > User should be able to define an use case, including: > > 1. to measure a dq metrics > > 2. to define the monitoring trigger > > 3. to define the alerting action > > 4. to integrate the 3-steps into a single job/UoW (by scheduler) > > > > On Sun, Jul 24, 2022 at 4:50 PM Eugene Law <liu...@apache.org> wrote: > > > > > > Appreciate William post of new architecture. John & Grant had raised good > > > comments as well. > > > > > > As an evolving data quality product, we need to catch up to the rolling > > > forward of the big data ecosystem. I think there are points we can > > > consider more. > > > > > > Firstly, how to absorb positive/negative experience from existent jira > > bugs > > > and make it apply in new architecture. > > > Secondly, how to highlight the improvement from the old one to the new > > > architecture and convince Grffin's users that the change will bring more > > > gains. > > > Thirdly, we should consider making rule-base architecture evolving as a > > > long-term goal, finally becoming a de facto standard in the vertical > > area. > > > > > > Thx > > > Eugene > > > > > > > > > On Sun, Jul 24, 2022 at 4:00 PM William Guo <gu...@apache.org> wrote: > > > > > > > just created the GIP pages and moved this proposal under GIP, > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > > > > > Thanks, > > > > William > > > > > > > > On Sat, Jul 23, 2022 at 3:11 PM William Guo <gu...@apache.org> wrote: > > > > > > > > > I have drafted next generation griffin architecture. > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > > > > > > > Let's brainstorm it . > > > > > > > > > > Thanks, > > > > > William > > > > > > > > > > > > > > > On Sat, Jul 23, 2022 at 10:28 AM john@apache <john...@apache.org> > > wrote: > > > > > > > > > >> Hi all, > > > > >> > > > > >> Great to hear that we will have something new in the plan. > > > > >> > > > > >> It is so happy to me to see that there will be new arch draft in 2 > > days. > > > > >> > > > > >> At the same time, I would like to raise up some of my ideas/comments > > > > >> related to the new arch before we decided to go > > > > >> > > > > >> > > > > >> 1. What’s the good part and bad part for the existing arch? I > > agree > > > > >> that the current arch which were defined 4 years ago is not good > > > > >> enough for the ongoing requirement. So I would like to recommend > > that > > > > >> compile a list of expected change from arch level, so we can start > > > > >> from there, to discuss the priority and scope. > > > > >> 2. We might need to define/re-define high level principles > > helping > > > > >> us clarify the vision, if it is not clear enough now. We need some > > > > >> brain storming to collect more ideas about what griffin should do > > and > > > > >> what should not. > > > > >> 3. We need to check with what is the main pain points of current > > > > >> user, and try to solve them in the arch level. I will go through all > > > > >> the jira issues and classify them to several categories. And I will > > > > >> keep update the status. > > > > >> > > > > >> Please let me know your comments. > > > > >> > > > > >> On Sat, Jul 23, 2022 at 12:53 AM Grant <grant.xu...@gmail.com> > > wrote: > > > > >> > > > > > >> > Hi Willem, > > > > >> > > > > > >> > I am Grant Guo, PMC member of the Griffin project, apache ID: > > xuexu > > > > >> > > > > > >> > Since last December, William, me, and other PMC members have been > > > > >> > discussing the feasibility of the rule-based solution, which could > > > > bring > > > > >> > Griffin to a new level in terms of usability, extensibility, etc. > > > > >> > > > > > >> > A new architectural proposal will be posted soon. > > > > >> > > > > > >> > Thanks > > > > >> > > > > > >> > On Wed, Jul 20, 2022 at 6:08 PM Willem Jiang < > > willem.ji...@gmail.com> > > > > >> wrote: > > > > >> > > > > > >> > > Hi, > > > > >> > > > > > > >> > > There has been no development activity on this project for more > > > > than 5 > > > > >> > > months. The PMC has failed to submit several board reports. > > Without > > > > a > > > > >> > > community or anyone working on the project, there is no project. > > > > >> > > > > > > >> > > If any of the PMC members are still active, please indicate so > > by > > > > >> > > responding to this email. > > > > >> > > > > > > >> > > Thank you, all! I hope everyone is safe and healthy. > > > > >> > > > > > > >> > > Willem Jiang > > > > >> > > > > > > >> > > > > > > > > > > > >
