hi All, Since our pmcs have reached an agreement, we will draft our feature roadmap pretty soon.
@Willem Jiang <willem.ji...@gmail.com> We will keep you posted. Thanks, William On Mon, Aug 8, 2022 at 5:30 PM Willem Ning Jiang <ningji...@apache.org> wrote: > It looks like there is a new generation of Griffin. > @William would you wrap up a roadmap for it? > > Thanks, > > Willem > > On 2022/08/06 08:16:49 William Guo wrote: > > Any comments from other PMCs? > > > > We need a full discussion before we kick off our next milestone for > apache > > griffin. > > > > Thanks, > > William > > > > > > On Fri, Jul 29, 2022 at 9:55 PM john@apache <john...@apache.org> wrote: > > > > > Could not agree more with Eugene's summary. > > > > > > Just had a quick meeting with William, talking his arch draft. Now we > > > have a clarified and clear defined goal for the new arch. > > > > > > William and I all agree that it would be great to have a simple and > > > clearly defined goal to go with. > > > > > > ---- > > > > > > # What Griffin should focus > > > > > > Griffin is a generic framework to enable user to measure and monitor > > > the data quality in an easy and extensive manner. > > > > > > # Old Griffin architecture problems > > > > > > What's the painpoints we are facing in the current edition of Griffin > > > from the architectural perspective? > > > > > > 1. To have an abstraction layer of query engine > > > The current spark implementation limited Griffin's adoption, since a > > > company could use other query engine in company level. It's too heavy > > > to set up a spark environment to run Griffin. > > > > > > 2. To support a common data quality workflow: measure-monitor-alert > > > In most of real scenarios, measuring is not the goal, but > > > measure-monitor-alert workflow. > > > User should be able to define an use case, including: > > > 1. to measure a dq metrics > > > 2. to define the monitoring trigger > > > 3. to define the alerting action > > > 4. to integrate the 3-steps into a single job/UoW (by scheduler) > > > > > > On Sun, Jul 24, 2022 at 4:50 PM Eugene Law <liu...@apache.org> wrote: > > > > > > > > Appreciate William post of new architecture. John & Grant had raised > good > > > > comments as well. > > > > > > > > As an evolving data quality product, we need to catch up to the > rolling > > > > forward of the big data ecosystem. I think there are points we can > > > > consider more. > > > > > > > > Firstly, how to absorb positive/negative experience from existent > jira > > > bugs > > > > and make it apply in new architecture. > > > > Secondly, how to highlight the improvement from the old one to the > new > > > > architecture and convince Grffin's users that the change will bring > more > > > > gains. > > > > Thirdly, we should consider making rule-base architecture evolving > as a > > > > long-term goal, finally becoming a de facto standard in the vertical > > > area. > > > > > > > > Thx > > > > Eugene > > > > > > > > > > > > On Sun, Jul 24, 2022 at 4:00 PM William Guo <gu...@apache.org> > wrote: > > > > > > > > > just created the GIP pages and moved this proposal under GIP, > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > > > > > > > Thanks, > > > > > William > > > > > > > > > > On Sat, Jul 23, 2022 at 3:11 PM William Guo <gu...@apache.org> > wrote: > > > > > > > > > > > I have drafted next generation griffin architecture. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/GRIFFIN/Rule+Based+Architecture+Proposal > > > > > > > > > > > > Let's brainstorm it . > > > > > > > > > > > > Thanks, > > > > > > William > > > > > > > > > > > > > > > > > > On Sat, Jul 23, 2022 at 10:28 AM john@apache <john...@apache.org > > > > > wrote: > > > > > > > > > > > >> Hi all, > > > > > >> > > > > > >> Great to hear that we will have something new in the plan. > > > > > >> > > > > > >> It is so happy to me to see that there will be new arch draft > in 2 > > > days. > > > > > >> > > > > > >> At the same time, I would like to raise up some of my > ideas/comments > > > > > >> related to the new arch before we decided to go > > > > > >> > > > > > >> > > > > > >> 1. What’s the good part and bad part for the existing arch? I > > > agree > > > > > >> that the current arch which were defined 4 years ago is not good > > > > > >> enough for the ongoing requirement. So I would like to recommend > > > that > > > > > >> compile a list of expected change from arch level, so we can > start > > > > > >> from there, to discuss the priority and scope. > > > > > >> 2. We might need to define/re-define high level principles > > > helping > > > > > >> us clarify the vision, if it is not clear enough now. We need > some > > > > > >> brain storming to collect more ideas about what griffin should > do > > > and > > > > > >> what should not. > > > > > >> 3. We need to check with what is the main pain points of > current > > > > > >> user, and try to solve them in the arch level. I will go > through all > > > > > >> the jira issues and classify them to several categories. And I > will > > > > > >> keep update the status. > > > > > >> > > > > > >> Please let me know your comments. > > > > > >> > > > > > >> On Sat, Jul 23, 2022 at 12:53 AM Grant <grant.xu...@gmail.com> > > > wrote: > > > > > >> > > > > > > >> > Hi Willem, > > > > > >> > > > > > > >> > I am Grant Guo, PMC member of the Griffin project, apache ID: > > > xuexu > > > > > >> > > > > > > >> > Since last December, William, me, and other PMC members have > been > > > > > >> > discussing the feasibility of the rule-based solution, which > could > > > > > bring > > > > > >> > Griffin to a new level in terms of usability, extensibility, > etc. > > > > > >> > > > > > > >> > A new architectural proposal will be posted soon. > > > > > >> > > > > > > >> > Thanks > > > > > >> > > > > > > >> > On Wed, Jul 20, 2022 at 6:08 PM Willem Jiang < > > > willem.ji...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > > >> > > Hi, > > > > > >> > > > > > > > >> > > There has been no development activity on this project for > more > > > > > than 5 > > > > > >> > > months. The PMC has failed to submit several board reports. > > > Without > > > > > a > > > > > >> > > community or anyone working on the project, there is no > project. > > > > > >> > > > > > > > >> > > If any of the PMC members are still active, please indicate > so > > > by > > > > > >> > > responding to this email. > > > > > >> > > > > > > > >> > > Thank you, all! I hope everyone is safe and healthy. > > > > > >> > > > > > > > >> > > Willem Jiang > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > >
