RE: [DISCUSS] Time for the Apache Livy 0.8.0 Release
I apologize I don’t have the bandwidth to help manage the release, but I can pass on what I remember from the last couple releases I did years ago. Livy has a release script that we used for our releases: https://github.com/apache/incubator-livy/blob/master/dev/release-build.sh The script has a usage message when run without parameters but given it hasn’t been run in a couple years you will most likely want to read through the script and double check that all the hard-coded values are still correct. IIRC the script can be dry run, but if you’re being extra careful you can just run it without your Apache credentials saved in the correct ENV vars. Also, for historical reference, we copied the release script from Apache Spark. So, you can look up their release docs from around the time when we first committed the release script and those would generally apply to our release process (I’m assuming Spark has changed its release process since then and its current doc would not help) Alex Bozarth IBM Open Quantum From: larry mccay Date: Monday, June 19, 2023 at 6:37 PM To: ksu...@zohomail.com.invalid Cc: dev , jbonofre , dev , ajboza...@apache.org Subject: [EXTERNAL] Re: [DISCUSS] Time for the Apache Livy 0.8.0 Release I gave a ghost +1 based @dacort's review and I will use testing while testing the RC. @Jean-Baptiste Onofré - I think we are ready to start talking about getting an RC ready. Note that I just got a wiki space created with a simple overview page with some article TODOs at the bottom. One of those is an article on the release process for contributors. We should target getting that done as well. Do you have experience being a release manager for Livy? If not, would @ajbozarth be willing to help move it forward? On Mon, Jun 12, 2023 at 8:37 PM Sumit Kumar wrote: > Just thought to update on the progress. > https://github.com/apache/incubator-livy/pull/407 is ready to be > reviewed, I've already received one approval from @dacort, waiting > for mailto:lmc...@apache.org to have a look too. > > > > > > > > On Fri, 09 Jun 2023 21:50:54 -0700 Jean-Baptiste Onofré < > jbono...@apache.org> wrote --- > > > > +1, it makes sense to have a milestone target. > > I would move forward with a release "as it is" as it would be a good > message to community. > > Regards > JB > > On Wed, May 31, 2023 at 7:09 PM larry mccay <mailto:lmc...@apache.org> > wrote: > > > > I can see this being very helpful in the testing of the RC, if we can > get > > it in. > > Even if it is only a partial usecase to test that we can document as > part > > of the RC announcement that would give new testers something to > immediately > > standup and test. > > That said, I would timebox this so that we can turn the crank and get > the > > release machinery working again within the next week or so. > > > > Thoughts? > > > > On Tue, May 30, 2023 at 7:37 PM Sumit Kumar <mailto:ksu...@apache.org> > wrote: > > > > > https://issues.apache.org/jira/browse/LIVY-969 aims to make developer > > > productivity better by making it easier to develop and test locally in > an > > > integrated environment using docker and docker-compose. May be not > everyone > > > would benefit from this immediately if they are not comfortable with > docker > > > ecosystem. I'm in favor of making progress, so may be we shouldn't > block > > > just for this? How about we cut an RC and see how things go? If I'm > able to > > > catch up the train, great; if not, we will march ahead without it? > > > > > > > > > > > > > > > > > > On Tue, 30 May 2023 16:25:35 -0700 larry mccay larry.mc...@gmail.com> > > > wrote --- > > > > > > > > > > > > Is this a release blocker for 0.8.0, @Sumit Kumar > > mailto:ksu...@apache.org><mailto:ksu...@apache.org%3e> ? > > > > > > On Tue, May 30, 2023 at 12:34 PM Sumit Kumar <mailto:mailto: > ksu...@apache.org> > > > wrote: > > > > > > > I was able to complete the dependency task LIVY-593 but I'm yet to > make > > > > progress on LIVY-969, this may take another week or two. > > > > > > > > > > > > > > > > On Tue, 30 May 2023 08:07:00 -0700 *larry mccay > > > > <mailto:mailto:larry.mc...@gmail.com <mailto:mailto: > larry.mc...@gmail.com>>* wrote > > > --- > > > > > > > > Hi @Jean-Baptiste Onofré <mailto:mailto:jbono...@apache.org> - > > > > > > > > Thanks for the response! &g
Re: [DISCUSS] First Steps for Revival and Release Proposal
On the release process, iirc you just need to use the build-release.sh script in the dev directory of the repo. I do remember have some issues with it the last time I did a release, but I don’t remember what they were or how I addressed them after so many years. That release script was copied from the Spark release script as it was at the time Livy was started, so their community may be able to help. As for dev documentation, there never really was any other than the README. Afaik all our docs are targeted at users, not devs. Alex Bozarth Jupyter Architect, IBM CODAIT GitHub: ajbozarth On 10/31/22, 9:45 AM, "larry mccay" wrote: All - I think we should discuss first steps to reviving this project in the context of a release. There are numerous forks with features that we are looking to get into the revived project but I would suggest that we target an initial release of what is already there to ensure that we have the process down and can address any security issues and document the changes in 0.8.0. There are a couple things that I think we could address in this first step: 1. I can't seem to find any Process docs on the site for doing an actual release. This needs to be documented, if not for doing the release then as an artifact of doing this next release. While we are at it, I believe that the site is also missing instructions for getting started as a developer on the project. Adding such docs may help get new contributors engaged. I had to make a minor change (after hours of googling py-test build problems) to the python-api/setup.py script in order for it to build. Likely just a me problem. 2. CVE and dependency hygiene related tasks to make sure there is a clean version available to start from. This may require some github or other magic for determining problem dependencies that should be put in place and/or documented. 3. Delta between 0.7.0 and 0.8.0 release in terms of provided features, bugs and improvements. In parallel we can discuss the various changes and how to roll them into future releases rather than trying to boil the ocean all at once. A separate DISCUSS thread can be started to do an inventory of proposed features and improvements that will require one-pager wikis (LIPs) to describe the problem statement, usecases, approach. We will undoubtedly need to reconcile multiple implementations of some things by either convergence or optional pluggable implementations. Does anyone have enough context for the release process in order to be Release Manager for 0.8.0? Any other thoughts? Thanks! --larry
RE: Proposal to Revive Apache Livy Community
I would love to stay on as a PPMC, but if my lack of availability concerns the new PPMC then I'm willing to be emeritus. Alex Bozarth Jupyter Architect, IBM CODAIT GitHub: ajbozarth On 10/24/22, 2:46 PM, "larry mccay" wrote: Oh, very good, Alex! Be sure to let us know if you want to remain on the project in any capacity. On Mon, Oct 24, 2022 at 3:18 PM Alex Bozarth wrote: > As the only Livy PPMC still responding to the Mentors on the private list, > I have updated the Livy Podling report for November with the status of the > project and a request to the IPMC to review this proposal since there are > not enough active Livy PPMC members to reach a quorum to pass the proposal. > > As a current Livy PPMC I strongly support this proposal for revitalization > as I do not have enough bandwidth to dedicate to Livy. > > > Alex Bozarth > Jupyter Architect, IBM CODAIT > GitHub: ajbozarth > > On 10/24/22, 1:41 PM, "larry mccay" wrote: > > Gentle reminder that we need to determine next steps here. > We have an updated proposal on this thread. > Do we need a VOTE or can we move forward directly to adjusting the > members, > etc? > > Thanks! > > --larry > > On Thu, Oct 20, 2022 at 3:26 PM larry mccay wrote: > > > @Justin Mclean - any insights on next steps > here? > > > > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay > wrote: > > > >> Very good, here is the latest revision with updated Mentors. > >> Sunil and I have been added to the IPMC as well. > >> Welcome Madhawa and thanks for stepping up as a Mentor for Livy! > >> > >> Abstract > >> > >> Livy is a web service that exposes a REST interface for managing > long > >> running Apache Spark contexts in your cluster. With Livy, new > applications > >> can be built on top of Apache Spark that require fine grained > interaction > >> with many Spark contexts [1]. > >> > >> While this project has been well regarded and used in many contexts > as > >> the defacto standard API to Spark environments, it has been > incubating for > >> over 5 years without graduation to a TLP and it has become > difficult to > >> impossible for fixes and improvements to be contributed as the > current > >> community seems to have moved on. > >> > >> There has been discussion regarding retirement of this podling where > >> there seems to be some increasing interest in joining and reviving > the > >> community [2]. > >> > >> The intent of this proposal is to avoid retiring a well regarded, > >> actively used and rather mature project by reviving the PPMC and > community > >> with new folks that have a vested interest in the project and > health of the > >> community. > >> Proposal > >> > >> We propose to revive the PPMC with a set of contributors and > maintainers > >> as mentors, PPMC members and committers. > >> > >> The retirement DISCUSS thread [2] has shown a growing interest in > >> providing new committers and bringing improvements and fixes from > >> organization’s internally maintained forks back to a revived > community. > >> > >> General Approach to Revival: > >> > >>- > >> > >>Add new Mentors > >>- > >> > >> Larry McCay, lmc...@apache.org , Cloudera > >> - > >> > >> Sunil Govindan, sun...@apache.org, Cloudera > >> - > >> > >> Jean-Baptiste Onofré, jbono...@apache.org, Talend > >> - > >> > >> Madhawa Gunasekara, madhaw...@gmail.com, Independent > >> > >> > >> > >>- > >> > >>Add new Committers/PPMC > >>- > >> > >> Larry McCay, lmc...
RE: Proposal to Revive Apache Livy Community
As the only Livy PPMC still responding to the Mentors on the private list, I have updated the Livy Podling report for November with the status of the project and a request to the IPMC to review this proposal since there are not enough active Livy PPMC members to reach a quorum to pass the proposal. As a current Livy PPMC I strongly support this proposal for revitalization as I do not have enough bandwidth to dedicate to Livy. Alex Bozarth Jupyter Architect, IBM CODAIT GitHub: ajbozarth On 10/24/22, 1:41 PM, "larry mccay" wrote: Gentle reminder that we need to determine next steps here. We have an updated proposal on this thread. Do we need a VOTE or can we move forward directly to adjusting the members, etc? Thanks! --larry On Thu, Oct 20, 2022 at 3:26 PM larry mccay wrote: > @Justin Mclean - any insights on next steps here? > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay wrote: > >> Very good, here is the latest revision with updated Mentors. >> Sunil and I have been added to the IPMC as well. >> Welcome Madhawa and thanks for stepping up as a Mentor for Livy! >> >> Abstract >> >> Livy is a web service that exposes a REST interface for managing long >> running Apache Spark contexts in your cluster. With Livy, new applications >> can be built on top of Apache Spark that require fine grained interaction >> with many Spark contexts [1]. >> >> While this project has been well regarded and used in many contexts as >> the defacto standard API to Spark environments, it has been incubating for >> over 5 years without graduation to a TLP and it has become difficult to >> impossible for fixes and improvements to be contributed as the current >> community seems to have moved on. >> >> There has been discussion regarding retirement of this podling where >> there seems to be some increasing interest in joining and reviving the >> community [2]. >> >> The intent of this proposal is to avoid retiring a well regarded, >> actively used and rather mature project by reviving the PPMC and community >> with new folks that have a vested interest in the project and health of the >> community. >> Proposal >> >> We propose to revive the PPMC with a set of contributors and maintainers >> as mentors, PPMC members and committers. >> >> The retirement DISCUSS thread [2] has shown a growing interest in >> providing new committers and bringing improvements and fixes from >> organization’s internally maintained forks back to a revived community. >> >> General Approach to Revival: >> >>- >> >>Add new Mentors >>- >> >> Larry McCay, lmc...@apache.org , Cloudera >> - >> >> Sunil Govindan, sun...@apache.org, Cloudera >> - >> >> Jean-Baptiste Onofré, jbono...@apache.org, Talend >> - >> >> Madhawa Gunasekara, madhaw...@gmail.com, Independent >> >> >> >>- >> >>Add new Committers/PPMC >>- >> >> Larry McCay, lmc...@apache.org, Cloudera >> - >> >> Vinod Kumar Vavilapalli, vino...@cloudera.com, Cloudera >> - >> >> Imran Rashid - iras...@apache.org, Cloudera >> - >> >> Gyorgy Gal, ggal ,gal.gyo...@gmail.com, Cloudera >> - >> >> Wing Yew Poon, wyp...@cloudera.com, Cloudera >> - >> >> Xilang Yan, xilang@gmail.com, Shopee >> - >> >> Jianzhen Wu, myjianz...@gmail.com, Shopee >> - >> >> Nagella Jagadeewara Rao, jnage...@visa.com, Visa >> - >> >> Pralab Kumar, pralk...@visa.com, Visa >> - >> >> Prasad Shrikant, shrikant@gmail.com, Visa >> - >> >> Brahma Reddy Battula, bra...@apache.org, Visa >> >> >> >>- >> >>Invite existing PPMC members to opt-in or otherwise go emeritus >>- >> >> Jean-Baptiste Onofré, jbono...@apache.org, Talend (opted-in via >> Retirement DISCUSS thread [2]) >> >> >> >>- >&g
Re: [VOTE] Release Livy 0.7.1 based on RC1
+1 thanks for the release work Alex Bozarth Software Engineer Center for Open-Source Data & AI Technologies E-mail: ajboz...@us.ibm.com GitHub: github.com/ajbozarth 505 Howard Street San Francisco, CA 94105 United States From: Saisai Shao To: d...@livy.incubator.apache.org Date: 02/03/2021 09:44 PM Subject:[EXTERNAL] [VOTE] Release Livy 0.7.1 based on RC1 This vote is for releasing Livy 0.7.1 based on RC1. The vote will be open until Feb 10 23:59 UTC and will pass with a minimum of 3 +1 binding votes and a majority of positive votes. [+1] This release is ready to send to the Incubator PMC for approval [-1] This release is not ready because... This vote is held according to Apache Incubator release policy: https://incubator.apache.org/policy/incubation.html#releases The RC is based on tag v0.7.1-incubating-rc1: https://github.com/apache/incubator-livy/commit/7c3d341926db69fb57a4978b15d4e96f06312267 The release files can be found here: https://dist.apache.org/repos/dist/dev/incubator/livy/0.7.1-incubating-rc1/ The staged maven artifacts can be found here: https://repository.apache.org/content/repositories/orgapachelivy-1014 This release is a hot-fix release, only added one commit compared to 0.7.0-incubating https://github.com/apache/incubator-livy/commit/9f1ba47a2f0d8accc435b133b42c3a76aa9ac846 Thanks Saisai
Re: Podling Livy Report Reminder - September 2020
Is there a reason we're being asked to submit two months in a row? I though it was every 3 months, or did that policy change? Alex Bozarth Software Engineer Center for Open-Source Data & AI Technologies E-mail: ajboz...@us.ibm.com GitHub: github.com/ajbozarth 505 Howard Street San Francisco, CA 94105 United States From: jmcl...@apache.org To: d...@livy.incubator.apache.org Date: 08/26/2020 06:55 PM Subject:[EXTERNAL] Podling Livy Report Reminder - September 2020 Dear podling, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Wed, 16 September 2020. The report for your podling will form a part of the Incubator PMC report. The Incubator PMC requires your report to be submitted 2 weeks before the board meeting, to allow sufficient time for review and submission (Wed, September 02). Please submit your report with sufficient time to allow the Incubator PMC, and subsequently board members to review and digest. Again, the very latest you should submit your report is 2 weeks prior to the board meeting. Candidate names should not be made public before people are actually elected, so please do not include the names of potential committers or PPMC members in your report. Thanks, The Apache Incubator PMC Submitting your Report -- Your report should contain the following: * Your project name * A brief description of your project, which assumes no knowledge of the project or necessarily of its field * A list of the three most important issues to address in the move towards graduation. * Any issues that the Incubator PMC or ASF Board might wish/need to be aware of * How has the community developed since the last report * How has the project developed since the last report. * How does the podling rate their own maturity. This should be appended to the Incubator Wiki page at: https://cwiki.apache.org/confluence/display/INCUBATOR/September2020 Note: This is manually populated. You may need to wait a little before this page is created from a template. Note: The format of the report has changed to use markdown. Mentors --- Mentors should review reports for their project(s) and sign them off on the Incubator wiki page. Signing off reports shows that you are following the project - projects that are not signed may raise alarms for the Incubator PMC. Incubator PMC
