Hi Atri, I've checked the contribution this year: https://github.com/NetEase/kyuubi/graphs/contributors?from=2021-01-01&to=2021-05-31&type=c According to this page, I don't think one contributor/company is dominating. The number of active developers is 4, which is currently low, but I think it is reasonable. During incubation, we can grow the community and increase the number.
Thanks, Akira On Thu, Jun 3, 2021 at 12:52 PM Atri Sharma <a...@apache.org> wrote: > > I would like to hear from Kyuubi's mentors on the topic before reaching a > conclusion. > > On Thu, 3 Jun 2021, 05:40 Sheng Wu, <wu.sheng.841...@gmail.com> wrote: > > > Kent > > > > Thanks. > > The proposal is generally better this time. > > > > If your initial committers and your mentors agree, you are ready to embrace > > the ASF culture, I am good with that. > > Just from my experience, your community looks a little too young, like > > Atri's concern, which is reasonable. > > But I would like to trust you and your mentors' judgment and determination. > > > > Sheng Wu 吴晟 > > Twitter, wusheng1108 > > > > > > Kent Yao <y...@apache.org> 于2021年6月3日周四 上午12:54写道： > > > > > Hi Sheng, > > > > > > Truly sorry for the format issue. > > > > > > Please trust me I did not mean to hurt anybody and push any contributor > > > away. We drafted this proposal together based on the New Podling > > Proposal. > > > We never tried to use PPMC and committer to measure whose contribution is > > > higher. This mistake has been fixed in the proposal. > > > > > > About releases, we will follow > > > https://www.apache.org/legal/release-policy.html and the guide of our > > > mentors. We will also take the initiative to learn other documents. > > > > > > > This is not a block, the point I mentioned this is, you will need more > > > people. > > > > > > We will keep working on this. > > > > > > Bests, > > > > > > Kent Yao > > > > > > > > > Sheng Wu <wu.sheng.841...@gmail.com> 于2021年6月2日周三 下午10:53写道： > > > > > > > Hi Kent > > > > > > > > Thanks for replying, but to be honest, your mail format is chaos. > > > > Take a look at bellowing, which is what I saw in the gmail. > > > > > > > > This may be an issue of your mail client or some else. > > > > > > > > > The current #.2 - 5 top > > > > > contributors will stay as committers, and they will join the PPMC > > > during > > > > > incubation. > > > > > > > > PPMC usually is the default for all initial committers, considering > > your > > > > community is still small and not very active. > > > > You should invite all people you trusted to join, rather than pushing > > > them > > > > away. > > > > This is not just a too-high bar issue. Excluding 4 out of 5 is strange. > > > > I can't guess what is the root cause, why the other 4 PPMC members > > don't > > > > touch codes but are trusted completely. Meanwhile, people with easily > > > > measurable code contributions are just as committers? > > > > We(ASF) always remind the community to recognize no-code contribution, > > > but > > > > in this case, it seems the opposite? Don't recognize code contributors > > > > enough? > > > > Please reach your mentors, this seems not a typical ASF style. If you > > > have > > > > other people already involved in the ASF, asking them too. > > > > > > > > And from your description of the release process, I think you should > > talk > > > > more with your mentors. > > > > Such as what is a qualified ASF release, what should be checked. > > > > This is not a block, the point I mentioned this is, you will need more > > > > people. > > > > > > > > > > > > Sheng Wu 吴晟 > > > > Twitter, wusheng1108 > > > > > > > > > > > > Kent Yao <y...@apache.org> 于2021年6月2日周三 下午10:37写道： > > > > > > > > > Hi Sheng, > > > > > > > > > > Thanks for your detailed check and valuable comments. They are > > > > > accommodating. I have addressed some of them directly in the > > proposal. > > > > > Please also check the table below for some Q&As. > > > > > QA > > > > > You listed only 5 core maintainers, but clearly only one has > > dominated > > > > > contributions. The distortion and skewness of the contributions are > > > > mainly > > > > > caused by the way how we merge PRs changed(mentioned in another email > > > on > > > > > the list). > > > > > I agree with Atri, orphaned is measurable. It is better to adjust > > this > > > > > statement a little. Address the comment to notify the risk in the > > > Kyuubi > > > > > proposal. > > > > > Also, with this fact, the core maintainers should notice whether this > > > > small > > > > > group of people to make sure your features iteration, bug fix, and > > > > release > > > > > process working by following ASF's requirements.This would not be an > > > easy > > > > > job. Currently, we learn the release process from some Apache > > projects, > > > > > using major.minor.patch for versioning. As we can see, since Kyuubi > > > 1.0, > > > > it > > > > > takes us 2 ~ 3 weeks for a patch release that contains 5-10 bug fixes > > > > until > > > > > EOL. And it takes us about two months to finish a minor release with > > 3 > > > ~ > > > > 5 > > > > > big improvements and various small features/bug fixes. We also will > > > start > > > > > with some release candidates for preview and test before the final. > > > This > > > > is > > > > > how we develop and release Kyuubi for now, but we do have a long way > > to > > > > go > > > > > and learn to match the ASF requirement. > > > > > And your committer list is very strange... 1. Why some of them are > > PPMC > > > > and > > > > > others are committers? Where this hierarchy comes from? The current > > > PPMC > > > > > candidates help us a lot build the Kyuubi project and community, not > > > > > directly to the codebase but as mentors(before Apache). Most of them > > > are > > > > > from Apache mature TLPs, which have close relationships with Kyuubi. > > > They > > > > > have sufficient Apache Way experience to guide us to become a more > > > > > Apache-like project before submitting this proposal in many ways. We > > > will > > > > > continue to promote more contributors to the PPMC/committer based on > > > > their > > > > > code or non-code contributions during incubation. > > > > > 2. Your core maintainers have 5, which only 4 PPMC? The current #.2 > > - 5 > > > > top > > > > > contributors will stay as committers, and they will join the PPMC > > > during > > > > > incubation. > > > > > 3. I think only the top 4 have over 400 LOC, why you declared you > > have > > > 5 > > > > > core developers? I didn't notice that 400 LOC is a bar for core > > > > developer, > > > > > will change 5 to 4 core developers accordingly > > > > > I think you don't have enough no-salaried developers, am I missing > > > > > anything? Yes. > > > > > Changed in the proposal, most of the contributors are salaried, and > > > some > > > > of > > > > > them are willing to help in the spare time. Thanks > > > > > This project belongs to NetEase Inc. clearly, why NO SGA? Added SGA > > in > > > > > Proposal. All work here is approved by NetEase and can be forward. > > > > > At last, you are asking about > > > github.com/apache/incubator-kyuubi-shaded > > > > > repo, > > > > > what does this represent? I can only see you have only one source > > repo > > > > and > > > > > a doc repo. Removed this requirement in the proposal, and we will > > apply > > > > it > > > > > from IPMC if we do need this in the future. > > > > > ------------------------------ > > > > > > > > > > Bests > > > > > > > > > > Kent Yao > > > > > > > > > > Sheng Wu <wu.sheng.841...@gmail.com> 于2021年6月2日周三 下午5:45写道： > > > > > > > > > > > Hi > > > > > > > > > > > > I read the contributor list and the proposal > > > > > > > The risk of the Kyuubi project being abandoned is minimal. Many > > > > > > organizations are using Kyuubi to build critical big data pipelines > > > and > > > > > > willing to help develop Kyuubi's community if it becomes an ASF > > > > project. > > > > > > > > > > > > I think the team should evaluate the status again. You listed only > > 5 > > > > core > > > > > > maintainers, but clearly only one has dominated contributions. I > > > agree > > > > > with > > > > > > Atri, orphaned is measurable. > > > > > > It is better to adjust this statement a little. Also, with this > > fact, > > > > the > > > > > > core maintainers should notice whether this small group of people > > to > > > > make > > > > > > sure your features iteration, bug fix, and release process working > > by > > > > > > following ASF's requirements. > > > > > > This would not be an easy job. > > > > > > > > > > > > And your committer list is very strange > > > > > > ``` > > > > > > Kent Yao (y...@apache.org) (PPMC) > > > > > > Wenchen Fan (wenc...@apache.org) (PPMC) > > > > > > Wanqiang Ji (j...@apache.org) (PPMC) > > > > > > Xun Liu (liu...@apache.org) (PPMC) > > > > > > Fei Wang (cn.feiw...@gmail.com) > > > > > > Ulysses You (ulyssesyo...@gmail.com) > > > > > > Cheng Pan (pan3...@gmail.com) > > > > > > Hongxiang Jiang (jhx1...@gmail.com) > > > > > > Zen Jin(xinjingziranc...@gmail.com) > > > > > > ``` > > > > > > 1. Why some of them are PPMC and others are committers? Where this > > > > > > hierarchy comes from? > > > > > > 2. Your core maintainers have 5, which only 4 PPMC? (Even hierarchy > > > > makes > > > > > > sense, most likely not) > > > > > > 3. I think only the top 4 have over 400 LOC, why you declared you > > > have > > > > 5 > > > > > > core developers? > > > > > > > > > > > > Another re-evaluation request is about `Reliance on Salaried > > > > Developers:` > > > > > > > They are all passionate about the project, and we are confident > > > that > > > > > the > > > > > > project will continue even if no salaried developers contribute to > > > the > > > > > > project. > > > > > > > > > > > > I think you don't have enough no-salaried developers, am I missing > > > > > > anything? > > > > > > > > > > > > About IP clearance, `Initial Source and Intellectual Property > > > > Submission > > > > > > Plan` > > > > > > > As soon as Kyuubi is approved to join Apache Incubator, our > > initial > > > > > > committers will submit iCLA(s) and CCLA(s). The codebase is already > > > > > > licensed under the Apache License 2.0. > > > > > > > > > > > > This project belongs to NetEase Inc. clearly, why NO SGA? > > > > > > > > > > > > At last, you are asking about ` > > > > > github.com/apache/incubator-kyuubi-shaded` > > <http://github.com/apache/incubator-kyuubi-shaded> > > > <http://github.com/apache/incubator-kyuubi-shaded> > > > > <http://github.com/apache/incubator-kyuubi-shaded> > > > > > <http://github.com/apache/incubator-kyuubi-shaded> > > > > > > <http://github.com/apache/incubator-kyuubi-shaded> > > > > > > repo, what does this represent? I can only see you have only one > > > source > > > > > > repo and a doc repo. > > > > > > > > > > > > Sheng Wu 吴晟 > > > > > > Twitter, wusheng1108 > > > > > > > > > > > > > > > > > > Atri Sharma <a...@apache.org> 于2021年6月2日周三 下午4:49写道： > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > I looked at the link you advised on and if I am reading it > > > correctly, > > > > > top > > > > > > > committer has 630ish commits and the next committer has 73 > > commits? > > > > > > > > > > > > > > IMO this is an indication of the community not being active > > enough > > > > and > > > > > > > hence the risk of project getting orphaned is measurable. > > > > > > > > > > > > > > Atri > > > > > > > > > > > > > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, <willem.ji...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > Hi Atri Sharma, > > > > > > > > > > > > > > > > It's easier to check out the recent contribution of kyuubi from > > > > this > > > > > > > > link[1]. > > > > > > > > We already identified the one person contribution issue of > > kyuubi > > > > > when > > > > > > > > Kent came to us and we kept reminding him to put more effort > > into > > > > > > > > building the community. > > > > > > > > Last week I had a long F2F conversation with Kent, I think he > > > > already > > > > > > > > knows about it and there have been some improvements in recent > > > > months > > > > > > > > but there is still a long way to go if Kyuubi is accepted as a > > > > > podling > > > > > > > > in the Apache Incubator. > > > > > > > > > > > > > > > > [1]https://github.com/NetEase/kyuubi/graphs/contributors > > > > > > > > > > > > > > > > Willem Jiang > > > > > > > > > > > > > > > > Twitter: willemjiang > > > > > > > > Weibo: 姜宁willem > > > > > > > > > > > > > > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma <a...@apache.org> > > > > wrote: > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I took a look at commit log and seems like all the commits > > for > > > > > last 3 > > > > > > > > > months have been done by a single person. > > > > > > > > > > > > > > > > > > How active are the remaining committers? > > > > > > > > > > > > > > > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <y...@apache.org> wrote: > > > > > > > > > > > > > > > > > > > Hi Alexander Alten, > > > > > > > > > > > > > > > > > > > > Thanks for your interest and +1 for Kyuubi. > > > > > > > > > > > > > > > > > > > > Not being very familiar with Dremel, I just took a quick > > look > > > > at > > > > > > the > > > > > > > > > > profile of a very similar and exciting project, Apache > > Drill. > > > > If > > > > > I > > > > > > > > > > understand correctly, Kyuubi is built on top of the compute > > > > layer > > > > > > > > > > (Apache Spark), while Apache Drill is built on top of the > > > > storage > > > > > > > > > > layer. > > > > > > > > > > > > > > > > > > > > Apache Spark has a large user base and usage scenarios all > > > over > > > > > the > > > > > > > > > > world. When users have some basic Spark programming and > > > tuning > > > > > > > skills, > > > > > > > > > > things are easy for them. But many times, most users do not > > > > have > > > > > > all > > > > > > > > > > of these abilities when they use Spark directly or > > > indirectly. > > > > We > > > > > > > want > > > > > > > > > > to build a more easy-to-use platform on top of it to help > > > these > > > > > > > users. > > > > > > > > > > On the one hand, we will work on the interface layer to > > make > > > it > > > > > > easy > > > > > > > > > > to access (of course, this effort will make our project > > look > > > > very > > > > > > > > > > similar to other projects, such as HiveServer2). On the > > other > > > > > hand, > > > > > > > we > > > > > > > > > > will work under Kyuubi's multi-tenant architecture to > > extend > > > > our > > > > > > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine) to make it > > more > > > > > > stable > > > > > > > > > > and efficient in our scenario. > > > > > > > > > > > > > > > > > > > > What's more, we are going to add more on-prem Kyuubi engine > > > > > types, > > > > > > > e.g. > > > > > > > > > > Flink. > > > > > > > > > > > > > > > > > > > > Alexander Alten <a...@scalytics.io> 于2021年5月31日周一 下午2:35写道： > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey, > > > > > > > > > > > > > > > > > > > > > > Looks like an interesting project, competition to > > > Starburst. > > > > > How > > > > > > > > does it > > > > > > > > > > > compare to Apache Dremel? > > > > > > > > > > > > > > > > > > > > > > +1 (non-binding) for incubation. > > > > > > > > > > > > > > > > > > > > > > Thanks and stay safe, > > > > > > > > > > > --alex > > > > > > > > > > > > > > > > > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao <y...@apache.org> > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Dear all, > > > > > > > > > > > > > > > > > > > > > > > > We want to propose Kyuubi as a new Apache Incubator > > > > project. > > > > > > > > > > > > > > > > > > > > > > > > Kyuubi is a distributed multi-tenant Thrift JDBC/ODBC > > > > server > > > > > > for > > > > > > > > > > > > large-scale data management, processing, and analytics, > > > > built > > > > > > on > > > > > > > > top > > > > > > > > > > > > of Apache Spark and designed to support more engines > > > (i.e., > > > > > > > Apache > > > > > > > > > > > > Flink). We are aiming to make Kyuubi an > > "out-of-the-box" > > > > tool > > > > > > for > > > > > > > > data > > > > > > > > > > > > warehouses and data lakes. Kyuubi concentrates on > > > improving > > > > > the > > > > > > > > > > > > experience for non-Spark users, making it easy for them > > > to > > > > > use > > > > > > > > Spark > > > > > > > > > > > > as a service and focus on business data of their own, > > not > > > > > where > > > > > > > it > > > > > > > > > > > > stores, how it computes. NetEase has open-sourced > > Kyuubi > > > > > since > > > > > > > > 2018, > > > > > > > > > > > > https://github.com/NetEase/kyuubi. > > > > > > > > > > > > > > > > > > > > > > > > Many thanks to Willem Jiang (Champion), Jeff Zhang, Duo > > > > > Zhang, > > > > > > > > Akira > > > > > > > > > > > > Ajisaka for being the mentors of the Kyuubi project and > > > > > helping > > > > > > > us > > > > > > > > set > > > > > > > > > > > > up this proposal. The proposal can be found at > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal > > > > > . > > > > > > > > > > > > > > > > > > > > > > > > Looking forward to your feedback and thank you very > > much. > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Kent Yao > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > To unsubscribe, e-mail: > > > > > > general-unsubscr...@incubator.apache.org > > > > > > > > > > > > For additional commands, e-mail: > > > > > > > general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > Kent Yao > > > > > > > > > > @ Data Science Center, Hangzhou Research Institute, NetEase > > > > Corp. > > > > > > > > > > a spark enthusiast > > > > > > > > > > kyuubiis a unified multi-tenant JDBC interface for > > > large-scale > > > > > data > > > > > > > > > > processing and analytics, built on top of Apache Spark. > > > > > > > > > > > > > > > > > > > > spark-authorizerA Spark SQL extension which provides SQL > > > > Standard > > > > > > > > > > Authorization for Apache Spark. > > > > > > > > > > spark-postgres A library for reading data from and > > > transferring > > > > > > data > > > > > > > > > > to Postgres / Greenplum with Spark SQL and DataFrames, > > > 10~100x > > > > > > > faster. > > > > > > > > > > itatchiA library that brings useful functions from various > > > > modern > > > > > > > > > > database management systems to Apache Spark. > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: > > > > general-unsubscr...@incubator.apache.org > > > > > > > > > > For additional commands, e-mail: > > > > > general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: > > general-unsubscr...@incubator.apache.org > > > > > > > > For additional commands, e-mail: > > > general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > *Kent Yao * > > > > > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. > > > > > a spark enthusiast > > > > > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant > > > > JDBC > > > > > interface for large-scale data processing and analytics, built on top > > > of > > > > > Apache > > > > > Spark <http://spark.apache.org/>. > > > > > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A > > Spark > > > > SQL > > > > > extension which provides SQL Standard Authorization for Apache Spark > > > > > <http://spark.apache.org/>. > > > > > spark-postgres <https://github.com/yaooqinn/spark-postgres> A > > library > > > > for > > > > > reading data from and transferring data to Postgres / Greenplum with > > > > Spark > > > > > SQL and DataFrames, 10~100x faster. > > > > > itatchi <https://github.com/yaooqinn/spark-func-extras>A library > > that > > > > > brings useful functions from various modern database management > > > > > systems to Apache > > > > > Spark <http://spark.apache.org/>. > > > > > > > > > > > > > > > > > > -- > > > > > > *Kent Yao * > > > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. > > > a spark enthusiast > > > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant > > JDBC > > > interface for large-scale data processing and analytics, built on top of > > > Apache > > > Spark <http://spark.apache.org/>. > > > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark > > SQL > > > extension which provides SQL Standard Authorization for Apache Spark > > > <http://spark.apache.org/>. > > > spark-postgres <https://github.com/yaooqinn/spark-postgres> A library > > for > > > reading data from and transferring data to Postgres / Greenplum with > > Spark > > > SQL and DataFrames, 10~100x faster. > > > itatchi <https://github.com/yaooqinn/spark-func-extras>A library that > > > brings useful functions from various modern database management > > > systems to Apache > > > Spark <http://spark.apache.org/>. > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
