Any comments from other PMCs? We need a full discussion before we kick off our next milestone for apache griffin.
Thanks, William On Fri, Jul 29, 2022 at 9:55 PM john@apache <[email protected]> wrote: > Could not agree more with Eugene's summary. > > Just had a quick meeting with William, talking his arch draft. Now we > have a clarified and clear defined goal for the new arch. > > William and I all agree that it would be great to have a simple and > clearly defined goal to go with. > > ---- > > # What Griffin should focus > > Griffin is a generic framework to enable user to measure and monitor > the data quality in an easy and extensive manner. > > # Old Griffin architecture problems > > What's the painpoints we are facing in the current edition of Griffin > from the architectural perspective? > > 1. To have an abstraction layer of query engine > The current spark implementation limited Griffin's adoption, since a > company could use other query engine in company level. It's too heavy > to set up a spark environment to run Griffin. > > 2. To support a common data quality workflow: measure-monitor-alert > In most of real scenarios, measuring is not the goal, but > measure-monitor-alert workflow. > User should be able to define an use case, including: > 1. to measure a dq metrics > 2. to define the monitoring trigger > 3. to define the alerting action > 4. to integrate the 3-steps into a single job/UoW (by scheduler) > > On Sun, Jul 24, 2022 at 4:50 PM Eugene Law <[email protected]> wrote: > > > > Appreciate William post of new architecture. John & Grant had raised good > > comments as well. > > > > As an evolving data quality product, we need to catch up to the rolling > > forward of the big data ecosystem. I think there are points we can > > consider more. > > > > Firstly, how to absorb positive/negative experience from existent jira > bugs > > and make it apply in new architecture. > > Secondly, how to highlight the improvement from the old one to the new > > architecture and convince Grffin's users that the change will bring more > > gains. > > Thirdly, we should consider making rule-base architecture evolving as a > > long-term goal, finally becoming a de facto standard in the vertical > area. > > > > Thx > > Eugene > > > > > > On Sun, Jul 24, 2022 at 4:00 PM William Guo <[email protected]> wrote: > > > > > just created the GIP pages and moved this proposal under GIP, > > > > > > > > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > > > Thanks, > > > William > > > > > > On Sat, Jul 23, 2022 at 3:11 PM William Guo <[email protected]> wrote: > > > > > > > I have drafted next generation griffin architecture. > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > > > > > Let's brainstorm it . > > > > > > > > Thanks, > > > > William > > > > > > > > > > > > On Sat, Jul 23, 2022 at 10:28 AM john@apache <[email protected]> > wrote: > > > > > > > >> Hi all, > > > >> > > > >> Great to hear that we will have something new in the plan. > > > >> > > > >> It is so happy to me to see that there will be new arch draft in 2 > days. > > > >> > > > >> At the same time, I would like to raise up some of my ideas/comments > > > >> related to the new arch before we decided to go > > > >> > > > >> > > > >> 1. What’s the good part and bad part for the existing arch? I > agree > > > >> that the current arch which were defined 4 years ago is not good > > > >> enough for the ongoing requirement. So I would like to recommend > that > > > >> compile a list of expected change from arch level, so we can start > > > >> from there, to discuss the priority and scope. > > > >> 2. We might need to define/re-define high level principles > helping > > > >> us clarify the vision, if it is not clear enough now. We need some > > > >> brain storming to collect more ideas about what griffin should do > and > > > >> what should not. > > > >> 3. We need to check with what is the main pain points of current > > > >> user, and try to solve them in the arch level. I will go through all > > > >> the jira issues and classify them to several categories. And I will > > > >> keep update the status. > > > >> > > > >> Please let me know your comments. > > > >> > > > >> On Sat, Jul 23, 2022 at 12:53 AM Grant <[email protected]> > wrote: > > > >> > > > > >> > Hi Willem, > > > >> > > > > >> > I am Grant Guo, PMC member of the Griffin project, apache ID: > xuexu > > > >> > > > > >> > Since last December, William, me, and other PMC members have been > > > >> > discussing the feasibility of the rule-based solution, which could > > > bring > > > >> > Griffin to a new level in terms of usability, extensibility, etc. > > > >> > > > > >> > A new architectural proposal will be posted soon. > > > >> > > > > >> > Thanks > > > >> > > > > >> > On Wed, Jul 20, 2022 at 6:08 PM Willem Jiang < > [email protected]> > > > >> wrote: > > > >> > > > > >> > > Hi, > > > >> > > > > > >> > > There has been no development activity on this project for more > > > than 5 > > > >> > > months. The PMC has failed to submit several board reports. > Without > > > a > > > >> > > community or anyone working on the project, there is no project. > > > >> > > > > > >> > > If any of the PMC members are still active, please indicate so > by > > > >> > > responding to this email. > > > >> > > > > > >> > > Thank you, all! I hope everyone is safe and healthy. > > > >> > > > > > >> > > Willem Jiang > > > >> > > > > > >> > > > > > > > >
