@Justin Mclean <justinmcl...@me.com> - any insights on next steps here?
On Tue, Oct 18, 2022 at 5:44 PM larry mccay <lmc...@apache.org> wrote: > Very good, here is the latest revision with updated Mentors. > Sunil and I have been added to the IPMC as well. > Welcome Madhawa and thanks for stepping up as a Mentor for Livy! > > Abstract > > Livy is a web service that exposes a REST interface for managing long > running Apache Spark contexts in your cluster. With Livy, new applications > can be built on top of Apache Spark that require fine grained interaction > with many Spark contexts [1]. > > While this project has been well regarded and used in many contexts as the > defacto standard API to Spark environments, it has been incubating for over > 5 years without graduation to a TLP and it has become difficult to > impossible for fixes and improvements to be contributed as the current > community seems to have moved on. > > There has been discussion regarding retirement of this podling where there > seems to be some increasing interest in joining and reviving the community > [2]. > > The intent of this proposal is to avoid retiring a well regarded, actively > used and rather mature project by reviving the PPMC and community with new > folks that have a vested interest in the project and health of the > community. > Proposal > > We propose to revive the PPMC with a set of contributors and maintainers > as mentors, PPMC members and committers. > > The retirement DISCUSS thread [2] has shown a growing interest in > providing new committers and bringing improvements and fixes from > organization’s internally maintained forks back to a revived community. > > General Approach to Revival: > > - > > Add new Mentors > - > > Larry McCay, lmc...@apache.org , Cloudera > - > > Sunil Govindan, sun...@apache.org, Cloudera > - > > Jean-Baptiste Onofré, jbono...@apache.org, Talend > - > > Madhawa Gunasekara, madhaw...@gmail.com, Independent > > > > - > > Add new Committers/PPMC > - > > Larry McCay, lmc...@apache.org, Cloudera > - > > Vinod Kumar Vavilapalli, vino...@cloudera.com, Cloudera > - > > Imran Rashid - iras...@apache.org, Cloudera > - > > Gyorgy Gal, ggal ,gal.gyo...@gmail.com, Cloudera > - > > Wing Yew Poon, wyp...@cloudera.com, Cloudera > - > > Xilang Yan, xilang....@gmail.com, Shopee > - > > Jianzhen Wu, myjianz...@gmail.com, Shopee > - > > Nagella Jagadeewara Rao, jnage...@visa.com, Visa > - > > Pralab Kumar, pralk...@visa.com, Visa > - > > Prasad Shrikant, shrikant....@gmail.com, Visa > - > > Brahma Reddy Battula, bra...@apache.org, Visa > > > > - > > Invite existing PPMC members to opt-in or otherwise go emeritus > - > > Jean-Baptiste Onofré, jbono...@apache.org, Talend (opted-in via > Retirement DISCUSS thread [2]) > > > > - > > Invite existing Committers to opt-out or otherwise continue > > > > - > > Establish Roadmap via follow up DISCUSS thread > - > > Known Improvements from Forks which will need proposals and > discussion: > - > > Adding HA for Livy > - > > Updating security capabilities (eg. kerberos for jdbc, fixing > bugs in encryption) > - > > Expanding the support for kubernetes > - > > Responding to CVEs in dependencies (eg. log4j, thrift) > - > > Livy rest cluster - IS THIS SAME AS HA for Livy ABOVE? > - > > Support multi Spark versions > - > > Implemented a metrics system for Livy > - > > Support customize batch/interactive session lifecycle event > handler, default log event with log4j, very helpful for > troubleshooting > - > > Optimize log to track which session id the log message came > from, also very helpful for troubleshooting > - > > Support customize Spark config optimization rules, can be used > to optimize config for users’ job > - > > A set of command line tool which can be used to replace Spark’s > spark-submit, pyspark, spark-sql but actually submit application in > Livy > - > > We are planning to implement a JDBC state store, and allow multi > Livy Thrift sessions to share one backend Spark application in the > next few > months. > - > > These items and others that are brought to community may need > consolidation or multiple configurable options and will need to be part > of > the discussion > - > > One-pager Livy Improvement Proposals (LIP) may make sense to > drive these discussions and convergence > - > > Feature Branch Strategy for large changes > - > > Large features are hard to review we will need to define a > strategy > - > > Determine the Improvements to be delivered across first 3 Releases > with Target Release Dates > > > > - > > Ensure CVE and Dependency management hygiene is in place > > > The above approach will usher the community back to an active status with > a Roadmap of 3 or more release plans and security hygiene in place. > Development Practices > > The Livy project follows a review before commit philosophy. Every commit > automatically runs through the unit tests and generates coverage reports > presented as a pull request comment. Our experience with this process leads > us to believe that it helps ease new contributors into the project. They > get feedback quickly on common mistakes, lowering the burden on reviewers. > Those same reviewers get to lead by example, showing the new contributors > that we value feedback within our community even when changes are done by > more experienced folks. Taken from the original Apache Livy Proposal [1], > this should continue to be true. As mentioned, Livy is a mature project and > as such RTC is the most appropriate for continued quality and awareness. > > > > 1. > > Original Apache Livy Proposal > https://cwiki.apache.org/confluence/display/incubator/LivyProposal > 2. > > Retirement DISCUSS thread > https://lists.apache.org/thread/gcstsrhbp91c5mm55htqn1l3djv8m7o0 > > > On Tue, Oct 18, 2022 at 5:05 PM Madhawa Gunasekara <madhaw...@gmail.com> > wrote: > >> Hi Larry, >> >> I'm an IPMC Member. madhawa30 at gmail dot com is my preferred email >> address. >> apache id: madhawa >> >> Thanks, >> Madhawa >> >> >> On Tue, Oct 18, 2022 at 10:05 PM larry mccay <lmc...@apache.org> wrote: >> >> > Hi Madhawa - >> > >> > That's awesome! >> > Are you already a member of IPMC? >> > If not, are you an ASF member? >> > If you are an ASF member you can request that you be added as an IPMC >> > member. >> > >> > Can you provide your company affiliation for the proposal and preferred >> > email? >> > >> > thanks! >> > >> > --larry >> > >> > On Tue, Oct 18, 2022 at 2:18 PM Madhawa Gunasekara <madhaw...@gmail.com >> > >> > wrote: >> > >> > > Hi Larry, >> > > >> > > I'm interested in working with Livy and would like to join as a >> Mentor. >> > > >> > > Thanks, >> > > Madhawa >> > > >> > > >> > > On Tue, Oct 18, 2022 at 6:57 PM larry mccay <lmc...@apache.org> >> wrote: >> > > >> > > > Sorry, I missed commenting on this: >> > > > >> > > > "There is also no concept as an emeritus PPMC member at the ASF." >> > > > >> > > > I assume that we can remove PPMC members that do not opt-in >> explicitly >> > at >> > > > this point. >> > > > They will have every opportunity to rejoin. >> > > > >> > > > On Tue, Oct 18, 2022 at 12:48 PM larry mccay <lmc...@apache.org> >> > wrote: >> > > > >> > > > > I will ask in a separate thread, @Justin Mclean < >> justinmcl...@me.com >> > > >> > > - >> > > > > thanks. >> > > > > Adding JB adds another company and we are certainly open to anyone >> > else >> > > > > that would like to join as a mentor. >> > > > > At the end of the day, the mentors are for instilling the Apache >> Way >> > > > > knowledge and steering toward graduation. >> > > > > I feel that this diversity, while nice to have, is less important >> > than >> > > > > that of the PPMC and committers for the long term health of the >> > > > community. >> > > > > >> > > > > We need to push this podling to graduation as quickly as possible >> > since >> > > > it >> > > > > is rather mature and needs to get to the next level. >> > > > > >> > > > > Again, any potential Mentors that would like to join are more than >> > > > welcome. >> > > > > >> > > > > On Tue, Oct 18, 2022 at 12:38 PM Justin Mclean < >> > > jus...@classsoftware.com >> > > > > >> > > > > wrote: >> > > > > >> > > > >> Hi, >> > > > >> >> > > > >> I’m sorry, but Imran Rashid can’t be a mentor for the project as >> > they >> > > > are >> > > > >> not an IPMC member. Currently, both Sunil and Larry (as they are >> ASF >> > > > >> members) need to ask to join the IPMC and NOTICE sent to the ASF >> > > board. >> > > > I >> > > > >> would also prefer that mentors come from different companies. >> > > > >> >> > > > >> There is also no concept as an emeritus PPMC member at the ASF. >> > > > >> >> > > > >> Kind Regards, >> > > > >> Justin >> > > > >> >> > > > >> >> > > > >> >> > --------------------------------------------------------------------- >> > > > >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> > > > >> For additional commands, e-mail: >> general-h...@incubator.apache.org >> > > > >> >> > > > >> >> > > > >> > > >> > >> >
