Hi Sheng, Thanks for your detailed check and valuable comments. They are accommodating. I have addressed some of them directly in the proposal. Please also check the table below for some Q&As. QA You listed only 5 core maintainers, but clearly only one has dominated contributions. The distortion and skewness of the contributions are mainly caused by the way how we merge PRs changed(mentioned in another email on the list). I agree with Atri, orphaned is measurable. It is better to adjust this statement a little. Address the comment to notify the risk in the Kyuubi proposal. Also, with this fact, the core maintainers should notice whether this small group of people to make sure your features iteration, bug fix, and release process working by following ASF's requirements.This would not be an easy job. Currently, we learn the release process from some Apache projects, using major.minor.patch for versioning. As we can see, since Kyuubi 1.0, it takes us 2 ~ 3 weeks for a patch release that contains 5-10 bug fixes until EOL. And it takes us about two months to finish a minor release with 3 ~ 5 big improvements and various small features/bug fixes. We also will start with some release candidates for preview and test before the final. This is how we develop and release Kyuubi for now, but we do have a long way to go and learn to match the ASF requirement. And your committer list is very strange... 1. Why some of them are PPMC and others are committers? Where this hierarchy comes from? The current PPMC candidates help us a lot build the Kyuubi project and community, not directly to the codebase but as mentors(before Apache). Most of them are from Apache mature TLPs, which have close relationships with Kyuubi. They have sufficient Apache Way experience to guide us to become a more Apache-like project before submitting this proposal in many ways. We will continue to promote more contributors to the PPMC/committer based on their code or non-code contributions during incubation. 2. Your core maintainers have 5, which only 4 PPMC? The current #.2 - 5 top contributors will stay as committers, and they will join the PPMC during incubation. 3. I think only the top 4 have over 400 LOC, why you declared you have 5 core developers? I didn't notice that 400 LOC is a bar for core developer, will change 5 to 4 core developers accordingly I think you don't have enough no-salaried developers, am I missing anything? Yes. Changed in the proposal, most of the contributors are salaried, and some of them are willing to help in the spare time. Thanks This project belongs to NetEase Inc. clearly, why NO SGA? Added SGA in Proposal. All work here is approved by NetEase and can be forward. At last, you are asking about github.com/apache/incubator-kyuubi-shaded repo, what does this represent? I can only see you have only one source repo and a doc repo. Removed this requirement in the proposal, and we will apply it from IPMC if we do need this in the future. ------------------------------
Bests Kent Yao Sheng Wu <wu.sheng.841...@gmail.com> 于2021年6月2日周三 下午5:45写道： > Hi > > I read the contributor list and the proposal > > The risk of the Kyuubi project being abandoned is minimal. Many > organizations are using Kyuubi to build critical big data pipelines and > willing to help develop Kyuubi's community if it becomes an ASF project. > > I think the team should evaluate the status again. You listed only 5 core > maintainers, but clearly only one has dominated contributions. I agree with > Atri, orphaned is measurable. > It is better to adjust this statement a little. Also, with this fact, the > core maintainers should notice whether this small group of people to make > sure your features iteration, bug fix, and release process working by > following ASF's requirements. > This would not be an easy job. > > And your committer list is very strange > ``` > Kent Yao (y...@apache.org) (PPMC) > Wenchen Fan (wenc...@apache.org) (PPMC) > Wanqiang Ji (j...@apache.org) (PPMC) > Xun Liu (liu...@apache.org) (PPMC) > Fei Wang (cn.feiw...@gmail.com) > Ulysses You (ulyssesyo...@gmail.com) > Cheng Pan (pan3...@gmail.com) > Hongxiang Jiang (jhx1...@gmail.com) > Zen Jin(xinjingziranc...@gmail.com) > ``` > 1. Why some of them are PPMC and others are committers? Where this > hierarchy comes from? > 2. Your core maintainers have 5, which only 4 PPMC? (Even hierarchy makes > sense, most likely not) > 3. I think only the top 4 have over 400 LOC, why you declared you have 5 > core developers? > > Another re-evaluation request is about `Reliance on Salaried Developers:` > > They are all passionate about the project, and we are confident that the > project will continue even if no salaried developers contribute to the > project. > > I think you don't have enough no-salaried developers, am I missing > anything? > > About IP clearance, `Initial Source and Intellectual Property Submission > Plan` > > As soon as Kyuubi is approved to join Apache Incubator, our initial > committers will submit iCLA(s) and CCLA(s). The codebase is already > licensed under the Apache License 2.0. > > This project belongs to NetEase Inc. clearly, why NO SGA? > > At last, you are asking about `github.com/apache/incubator-kyuubi-shaded` > <http://github.com/apache/incubator-kyuubi-shaded> > repo, what does this represent? I can only see you have only one source > repo and a doc repo. > > Sheng Wu 吴晟 > Twitter, wusheng1108 > > > Atri Sharma <a...@apache.org> 于2021年6月2日周三 下午4:49写道： > > > Thanks. > > > > I looked at the link you advised on and if I am reading it correctly, top > > committer has 630ish commits and the next committer has 73 commits? > > > > IMO this is an indication of the community not being active enough and > > hence the risk of project getting orphaned is measurable. > > > > Atri > > > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, <willem.ji...@gmail.com> wrote: > > > > > Hi Atri Sharma, > > > > > > It's easier to check out the recent contribution of kyuubi from this > > > link[1]. > > > We already identified the one person contribution issue of kyuubi when > > > Kent came to us and we kept reminding him to put more effort into > > > building the community. > > > Last week I had a long F2F conversation with Kent, I think he already > > > knows about it and there have been some improvements in recent months > > > but there is still a long way to go if Kyuubi is accepted as a podling > > > in the Apache Incubator. > > > > > > [1]https://github.com/NetEase/kyuubi/graphs/contributors > > > > > > Willem Jiang > > > > > > Twitter: willemjiang > > > Weibo: 姜宁willem > > > > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma <a...@apache.org> wrote: > > > > > > > > Hi, > > > > > > > > I took a look at commit log and seems like all the commits for last 3 > > > > months have been done by a single person. > > > > > > > > How active are the remaining committers? > > > > > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <y...@apache.org> wrote: > > > > > > > > > Hi Alexander Alten, > > > > > > > > > > Thanks for your interest and +1 for Kyuubi. > > > > > > > > > > Not being very familiar with Dremel, I just took a quick look at > the > > > > > profile of a very similar and exciting project, Apache Drill. If I > > > > > understand correctly, Kyuubi is built on top of the compute layer > > > > > (Apache Spark), while Apache Drill is built on top of the storage > > > > > layer. > > > > > > > > > > Apache Spark has a large user base and usage scenarios all over the > > > > > world. When users have some basic Spark programming and tuning > > skills, > > > > > things are easy for them. But many times, most users do not have > all > > > > > of these abilities when they use Spark directly or indirectly. We > > want > > > > > to build a more easy-to-use platform on top of it to help these > > users. > > > > > On the one hand, we will work on the interface layer to make it > easy > > > > > to access (of course, this effort will make our project look very > > > > > similar to other projects, such as HiveServer2). On the other hand, > > we > > > > > will work under Kyuubi's multi-tenant architecture to extend our > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine) to make it more > stable > > > > > and efficient in our scenario. > > > > > > > > > > What's more, we are going to add more on-prem Kyuubi engine types, > > e.g. > > > > > Flink. > > > > > > > > > > Alexander Alten <a...@scalytics.io> 于2021年5月31日周一 下午2:35写道： > > > > > > > > > > > > > > > > > Hey, > > > > > > > > > > > > Looks like an interesting project, competition to Starburst. How > > > does it > > > > > > compare to Apache Dremel? > > > > > > > > > > > > +1 (non-binding) for incubation. > > > > > > > > > > > > Thanks and stay safe, > > > > > > --alex > > > > > > > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao <y...@apache.org> wrote: > > > > > > > > > > > > > Dear all, > > > > > > > > > > > > > > We want to propose Kyuubi as a new Apache Incubator project. > > > > > > > > > > > > > > Kyuubi is a distributed multi-tenant Thrift JDBC/ODBC server > for > > > > > > > large-scale data management, processing, and analytics, built > on > > > top > > > > > > > of Apache Spark and designed to support more engines (i.e., > > Apache > > > > > > > Flink). We are aiming to make Kyuubi an "out-of-the-box" tool > for > > > data > > > > > > > warehouses and data lakes. Kyuubi concentrates on improving the > > > > > > > experience for non-Spark users, making it easy for them to use > > > Spark > > > > > > > as a service and focus on business data of their own, not where > > it > > > > > > > stores, how it computes. NetEase has open-sourced Kyuubi since > > > 2018, > > > > > > > https://github.com/NetEase/kyuubi. > > > > > > > > > > > > > > Many thanks to Willem Jiang (Champion), Jeff Zhang, Duo Zhang, > > > Akira > > > > > > > Ajisaka for being the mentors of the Kyuubi project and helping > > us > > > set > > > > > > > up this proposal. The proposal can be found at > > > > > > > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal. > > > > > > > > > > > > > > Looking forward to your feedback and thank you very much. > > > > > > > > > > > > > > Best regards, > > > > > > > Kent Yao > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: > general-unsubscr...@incubator.apache.org > > > > > > > For additional commands, e-mail: > > general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Kent Yao > > > > > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. > > > > > a spark enthusiast > > > > > kyuubiis a unified multi-tenant JDBC interface for large-scale data > > > > > processing and analytics, built on top of Apache Spark. > > > > > > > > > > spark-authorizerA Spark SQL extension which provides SQL Standard > > > > > Authorization for Apache Spark. > > > > > spark-postgres A library for reading data from and transferring > data > > > > > to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x > > faster. > > > > > itatchiA library that brings useful functions from various modern > > > > > database management systems to Apache Spark. > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > -- *Kent Yao * @ Data Science Center, Hangzhou Research Institute, NetEase Corp. a spark enthusiast kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark <http://spark.apache.org/>. spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark SQL extension which provides SQL Standard Authorization for Apache Spark <http://spark.apache.org/>. spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster. itatchi <https://github.com/yaooqinn/spark-func-extras>A library that brings useful functions from various modern database management systems to Apache Spark <http://spark.apache.org/>.
