Re: [ANNOUNCE] New Celeborn PMC Member: Mridul Muralidharan
Congrats and welcome, Mridul! Thanks, Cheng Pan > On Sep 20, 2024, at 09:30, rexxiong wrote: > > Hi Celeborn Community, > > The Project Management Committee (PMC) for Apache Celeborn > has invited Mridul Muralidharan to become a PMC member and we are pleased > to announce that he has accepted. > > A PMC member helps manage and guide the direction of the project. > We are glad to see his more interactions with the community in the future. > > Please join me in congratulating Mridul! > > Best, > Jiashu Xiong
Re: [VOTE] Release Apache Celeborn 0.5.1-rc0
+1 - links are valid - signatured and checksums are good - can build from source code - Ubuntu 22.04, x86, Java 17 - build/make-distribution.sh --sbt-enabled --release Thanks, Cheng Pan On Thu, Jul 25, 2024 at 4:38 PM Nicholas Jiang wrote: > > +1 (binding) > > I have checked > - Download links are valid. > - Checksums and signatures are valid. > - Git commit hash is correct > - No binary files in the source release > - Built binary from source code with command successfully: > ./build/make-distribution.sh -Pspark-3.5 > > I have upgraded version to 0.5.1-snapshot in internal test environment a week > ago. 0.5.1 works well in test environment. > > Regards, > Nicholas Jiang > > On 2024/07/23 03:38:57 Ethan Feng wrote: > > Hello, Celeborn community, > > > > This is a call for a vote to release Apache Celeborn > > 0.5.1-rc0 > > > > The git tag to be voted upon: > > https://github.com/apache/celeborn/releases/tag/v0.5.1-rc0 > > > > Source and binary artifacts can be found at: > > https://dist.apache.org/repos/dist/dev/celeborn/v0.5.1-rc0 > > > > The git commit hash: > > 85297ca64c0973f19b15831031859c3104f0db5b > > > > The staging repo: > > https://repository.apache.org/content/repositories/orgapacheceleborn-1081 > > > > The fingerprint of the PGP key release artifacts is signed with: > > FCF20BB29C7BEFDF58F998F76392F71F37356FA0 > > > > My public key to verify signatures can be found in: > > https://dist.apache.org/repos/dist/release/celeborn/KEYS > > > > The vote will be open for at least 72 hours or until the necessary > > number of votes are reached. > > > > Please vote accordingly: > > > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove (and the reason) > > > > Steps to validate the release: > > https://www.apache.org/info/verification.html > > > > * Download links, checksums, and PGP signatures are valid. > > * Source code distributions have correct names matching the current release. > > * LICENSE and NOTICE files are correct. > > * All files have license headers if necessary. > > * No unlicensed compiled archives bundled in the source archive. > > * The source tarball matches the git tag. > > * Build from source is successful. > > > > Regards, > > Ethan Feng > >
Re: [VOTE] Release Apache Celeborn 0.4.2-rc1
+1 - links are valid - signatured and checksums are good - can build from source code - Ubuntu 22.04, x86, Java 17 - build/make-distribution.sh --sbt-enabled --release Thanks, Cheng Pan On Mon, Jul 22, 2024 at 6:14 PM Fu Chen wrote: > > Hi Celeborn community, > > This is a call for a vote to release Apache Celeborn 0.4.2-rc1 > > > The git tag to be voted upon: > https://github.com/apache/celeborn/releases/v0.4.2-rc1 > > > The git commit hash: > d3639bb3c3d4cb2d224f1d6542e2d20d3047c76e > > Source and binary artifacts can be found at: > https://dist.apache.org/repos/dist/dev/celeborn/v0.4.2-rc1/ > > > The staging repo: > https://repository.apache.org/content/repositories/orgapacheceleborn-1079 > > > Fingerprint of the PGP key release artifacts are signed with: > 92AF4750DAFCB5E25B5B83EA76F54B977EB5C09B > > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > > Please vote accordingly: > > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Thanks, > Fu Chen
Re: [ANNOUNCE] New Celeborn PMC Member: Nicholas Jiang
Congrats! Thanks, Cheng Pan On Tue, Jul 23, 2024 at 5:34 PM rexxiong wrote: > > Hi Celeborn Community, > > The Project Management Committee (PMC) for Apache Celeborn > has invited Nicholas Jiang to become a PMC member and we are pleased > to announce that he has accepted. > > A PMC member helps manage and guide the direction of the project. > We are glad to see his more interactions with the community in the future. > > Please join me in congratulating Nicholas! > > > Thanks, > Jiashu Xiong
[ANNOUNCE] New Celeborn Committer: Fei Wang
Hi Celeborn Community, The Project Management Committee (PMC) for Apache Celeborn has invited Fei Wang to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. A PMC member helps manage and guide the direction of the project. Please join me in congratulating Fei! Thanks, Cheng Pan
Re: [VOTE] CIP-9: Celeborn RESTful API Refine
+1, thanks Thanks, Cheng Pan > On Jul 3, 2024, at 02:07, Fei Wang wrote: > > Hi all, > > Thanks for all the feedback about the CIP-9 Celeborn RESTful API Refine [1]. > The discussion thread is here [2]. > > I'd like to start a vote for it. The vote will be open for at least 72 > hours unless there is an objection or insufficient votes. > > [1] > https://docs.google.com/document/d/1LV2vV-w3XtlbJj2Vi4J77mt4IYCr40-8A_JncZLsHqs/edit?usp=sharing > [2] https://lists.apache.org/thread/mng0pxst0z4gc9gs7mc1frz4pzpk70jb > > Best Regards, > Fei Wang
Re: [VOTE] Release Apache Celeborn 0.5.0-rc3
+1 (binding) - All links are valid. - LICENSE/NOTICE looks fine. - CI result looks great. - No known blockers. Thanks, Cheng Pan > On Jun 19, 2024, at 12:45, Ethan Feng wrote: > > Hello, Celeborn community, > > This is a call for a vote to release Apache Celeborn > 0.5.0-rc3 > > The git tag to be voted upon: > https://github.com/apache/celeborn/releases/tag/v0.5.0-rc3 > > Source and binary artifacts can be found at: > https://dist.apache.org/repos/dist/dev/celeborn/v0.5.0-rc3 > > The git commit hash: > 048ef207359113247bff05dcc203c70021ccfa10 > > The staging repo: > https://repository.apache.org/content/repositories/orgapacheceleborn-1076/ > > The fingerprint of the PGP key release artifacts is signed with: > FCF20BB29C7BEFDF58F998F76392F71F37356FA0 > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/celeborn/KEYS > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Steps to validate the release: > https://www.apache.org/info/verification.html > > * Download links, checksums, and PGP signatures are valid. > * Source code distributions have correct names matching the current release. > * LICENSE and NOTICE files are correct. > * All files have license headers if necessary. > * No unlicensed compiled archives bundled in the source archive. > * The source tarball matches the git tag. > * Build from source is successful. > > There are additional tests: > * Performance test no regression > 1 TB TPC-DS, 0.5.0 VS 0.4.1 : 2042(s) VS 2050(s) > 1.1 TB pure shuffle, 0.5.0 VS 0.4.1 : 11.8min vs 11.8min > > * Result correctness test passed > 1TB TPC-DS runs concurrently, the results are identical. > > * Usability test passed > Rolling upgrade from version 0.4.1 to 0.5.0 succeed. > The metrics system works as expected. > > * Stability test passed > Random worker failures, Celeborn works as expected. > Random master failures, Celeborn works as expected. > Master meta corrupted, Celeborn works as expected. > > * Compatibility test passed > The Celeborn server version of 0.5.0 works fine with the Celeborn client > 0.4.1. > > * Grafana dashboard layout checked > > > Regards, > Ethan Feng
Re: [VOTE] Contrinute Apache Celeborn CLI
+1. Brainstorm: as REST API is mentioned in this proposal, both Iceberg REST Catalog API[1] and Unity Catalog[2] open-sourced by Databricks use tool to auto-generate Server/Client/Model code, well currently all REST APIs in Celeborn are written by hand, which might be something we can learn from in the future. [1] https://github.com/apache/iceberg/tree/main/open-api [2] https://github.com/unitycatalog/unitycatalog Thanks, Cheng Pan > On Jun 13, 2024, at 05:27, Aravind Patnam wrote: > > Hi Ethan, > > Thanks for your comments! > > Regarding using Java/Scala for the CLI, I am fine with this. I had believed > that using Python would be an easier/simpler implementation given that many > CLI's are implemented in Python, but the points you make are fair. Most of > the Celeborn community uses Java/Scala, so this would be more beneficial > for the development and evolution of the CLI. > > Yes, I think the CLI should contain capabilities beyond the HTTP endpoints > Celeborn exposes. The Celeborn HTTP endpoints work great for application > specific use cases, such as finding the applications or shuffles on a > particular worker, however it would not work for situations in which we > would need information on the cluster itself. For example, we use K8s and > these are use cases internally I can foresee that require communication > with an external cluster manager: > > - Retrieve all pods running masters/workers and their statuses > - Manually evict an Celeborn unhealthy pod > - SSH into various different Celeborn pods > - Manage ACLs of the cluster > - Manually restart pods > - Wipe Ratis storage if state is messed up > - Wipe shuffle directories if state is messed up > - Adding/removing new nodes into our node pool > - Perform any other manual arbitrary function on a Celeborn pod > > > These are just a few of the use cases I can think of, but I am sure more > will arise as more users adopt Celeborn :) > > Given that users will have various different cluster managers, I think as I > mentioned before there should be an abstraction layer present that exposes > different operations. Based on the cluster manager the user is using, the > user can implement their specific logic. We can have a few default ones > included (e.g. Kubernetes). > > Hope this answers your questions, let me know if you have any more > questions! > > Thanks, > Aravind > > On Tue, Jun 11, 2024 at 11:57 PM Ethan Feng wrote: > >> Hi Aravind, >> >> I hope this message finds you well. I wanted to express my >> appreciation for the energy and creativity you've invested in the >> Celeborn project; the proposal you submitted is intriguing. >> >> I apologize for the delayed feedback on your proposal — it took me a >> bit longer to get to it than anticipated. After reviewing it, I have a >> couple of inquiries that I'd like to discuss in order to gain a >> clearer understanding: >> >> I observed that you're planning to implement the CLI in Python. Could >> you elaborate on the choice behind not leveraging the Java stack for >> this purpose? The Java ecosystem already includes mature tools such as >> "commons-cli" or "Scala CLI," which are capable of facilitating CLI >> tool development. Given the prevalent familiarity with the Java stack >> within our community, I believe leveraging it could accelerate the >> CLI's development and evolution through wider collaboration. >> >> From email discussions, you've indicated an interest in offering a >> generic interface API for Celeborn, which is certainly exciting. >> However, I'm concerned that basing a CLI on HTTP API might not fully >> align with this vision. Could you provide additional insights into how >> you envision the CLI advancing beyond the capabilities of the current >> HTTP REST API? >> >> Based on previous exchanges, the CLI is expected to communicate with >> an external cluster manager. Is there an abstraction layer in place to >> interface uniformly with various external cluster managers, or is this >> something under consideration? >> >> I'm looking forward to learning more about your perspectives and the >> pathway you foresee for the CLI's development. >> >> regards, >> Ethan >> >> Mridul Muralidharan 于2024年6月12日周三 14:36写道： >>> >>> +1 >>> >>> Regards, >>> Mridul >>> >>> >>> On Wed, Jun 12, 2024 at 1:08 AM Shaoyun Chen wrote: >>> >>>> +1 >>>> >>>> Keyong Zhou 于2024年6月
Re: [Discussion] Proposal Management in Celeborn Community
+1 for archiving proposals on confluence. Does Confluence support inline comments like Google Docs does? I think it’s a convincing functionality for the discussion period. Thanks, Cheng Pan > On May 29, 2024, at 11:19, rexxiong wrote: > > Hello, Celeborn community, > > In the past, when Celeborn introduced new major features or significant > changes, we typically used Google Docs to launch proposals. However, a major > issue with Google Docs is the difficulty in centrally managing these > proposals. Therefore, after referring to other communities and based on > discussions with several PMCs offline, it appears that Apache Confluence > could be a viable alternative for our needs. With that in mind, I would like > to invite all of you to share your thoughts, experiences, and preferences > regarding the use of Apache Confluence versus Google Docs for our proposal > management. Your feedback will be invaluable in helping us make an informed > decision that best meets the needs of our community. > > Meanwhile, I have archived previous proposals and written the Celeborn > Improvement Proposal (CIP) process on Confluence. > > What do you think? Looking forward to your thoughts on this proposal. > > > Thanks, > Jiashu Xiong
Re: [VOTE] Release Apache Celeborn 0.4.1-rc1
+1 I have rolled out this version to a small cluster for several days, everything goes well so far. I checked the org.apache.celeborn:celeborn-client-spark-3-shaded_2.12:0.4.1, it does not pull transitive deps now. While there might be some issues with the relocation of Guava classes, specifically, package `com.google.thirdparty.publicsuffix` still be there. Thanks, Cheng Pan > On May 14, 2024, at 12:13, Nicholas Jiang wrote: > > Hi Celeborn community, > > This is a call for a vote to release Apache Celeborn > > 0.4.1-rc1 > > > The git tag to be voted upon: > > https://github.com/apache/celeborn/releases/tag/v0.4.1-rc1 > > The git commit hash: > 641180142c5ef36430a6afcd702c9487a6007458 source and binary artifacts can be > found at: > > https://dist.apache.org/repos/dist/dev/celeborn/v0.4.1-rc1 > > The staging repo: > > https://repository.apache.org/content/repositories/orgapacheceleborn-1055 > > > Fingerprint of the PGP key release artifacts are signed with: > D73CADC1DAB63BD3C770BB6D9476842D24B7C885 > > My public key to verify signatures can be found in: > > https://dist.apache.org/repos/dist/release/celeborn/KEYS > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Steps to validate the release: > > https://www.apache.org/info/verification.html > > * Download links, checksums and PGP signatures are valid. > * Source code distributions have correct names matching the current release. > * LICENSE and NOTICE files are correct. > * All files have license headers if necessary. > * No unlicensed compiled archives bundled in source archive. > * The source tarball matches the git tag. > * Build from source is successful. > > Regards, > Nicholas Jiang
Re: [ANNOUNCE] Add Mridul Muralidharan as new committer
Congrats Mridul, your expertise in Spark kernel, and the Security area are impressive. Thanks, Cheng Pan > On Apr 29, 2024, at 09:21, Keyong Zhou wrote: > > Hi Celeborn Community, > > The Project Management Committee (PMC) for Apache Celeborn > has invited Mridul Muralidharan to become a committer and we are pleased > to announce that he has accepted. > > Being a committer enables easier contribution to the > project since there is no need to go via the patch > submission process. This should enable better productivity. > A PMC member helps manage and guide the direction of the project. > > Please join me in congratulating Mridul Muralidharan! > > Regards, > Keyong Zhou
Re: [DISCUSS] Time for 0.4.1
+1, we do need a patch release for 0.4 Thanks, Cheng Pan > On Apr 12, 2024, at 19:59, Nicholas Jiang wrote: > > Hey, Celeborn community, > > > It has been a while since the 0.4.0 release, and there are some critical > fixes land branch-0.4, for example, [CELEBORN-1252][FOLLOWUP] Fix > Worker#computeResourceConsumption NullPointerException for > userResourceConsumption that does not contain given userIdentifier. From my > perspective, it’s time to prepare for releasing 0.4.1. > > > WDYT? And I’m volunteering to be the release manager if no one has applied. > > Regards, > Nicholas Jiang
Re: Re: [ANNOUNCE] Add Chandni Singh as new committer
Welcome, and congrats! Thanks, Cheng Pan On Thu, Mar 21, 2024 at 7:46 PM Yihe Li wrote: > > Congratulations! > > Regards, > Yihe Li > > On 2024/03/21 11:21:52 Fu Chen wrote: > > Congratulations and thank you to Chandni for the contributions to the > > community! > > > > Regards, > > Fu Chen > > > > Nicholas 于2024年3月21日周四 19:20写道： > > > > > > Congratulations Chandni. Well deserved! > > > > > > > > > > > > > > > Regards, > > > Nicholas Jiang > > > > > > > > > > > > > > > At 2024-03-21 19:16:25, "Shaoyun Chen" wrote: > > > >Congratulations! > > > > > > > >Mridul Muralidharan 于2024年3月21日周四 16:54写道： > > > >> > > > >> Congratulations Chandni ! Great job :-) > > > >> > > > >> Regards, > > > >> Mridul > > > >> > > > >> > > > >> On Thu, Mar 21, 2024 at 3:30 AM Keyong Zhou wrote: > > > >> > > > >> > Hi Celeborn Community, > > > >> > > > > >> > The Podling Project Management Committee (PPMC) for Apache Celeborn > > > >> > has invited Chandni Singh to become a committer and we are pleased > > > >> > to announce that she has accepted. > > > >> > > > > >> > Being a committer enables easier contribution to the > > > >> > project since there is no need to go via the patch > > > >> > submission process. This should enable better productivity. > > > >> > A (P)PMC member helps manage and guide the direction of the project. > > > >> > > > > >> > Please join me in congratulating Chandni Singh! > > > >> > > > > >> > Thanks, > > > >> > Keyong Zhou > > > >> > > >
Re: [VOTE] Graduate Apache Celeborn (incubating) as a TLP - Community
+1 Thanks, Cheng Pan > On Mar 1, 2024, at 16:52, Yu Li wrote: > > Hi All, > > After a thorough discussion [1], I'd like to call a formal vote to > graduate Apache Celeborn (incubating) as a TLP. Below are some facts > and project highlights carried from [1] as well as the draft > resolution: > > - Currently, our community consists of 19 committers (including > mentors) from more than 10 companies, with 12 serving as PPMC members. > - So far, we have boasted 81 contributors. > - Throughout the incubation period, we've made 6 releases in 16 > months, at a stable pace. > - We've had 6 different release managers to date. > - Our software is used in production by 10+ well known entities. > - As yet, we have opened 1,286 issues with 1,176 successfully resolved. > - We have submitted a total of 1,816 PRs, out of which 1,805 have been > merged or closed. > - Through self-assessment [2], we have met all maturity criteria as > outlined in [3]. > > We've resolved all branding issues which include Logo, GitHub repo, > document, website, and others [4] [5]. > > -- > Establish the Apache Celeborn Project > > WHEREAS, the Board of Directors deems it to be in the best interests of > the Foundation and consistent with the Foundation's purpose to establish > a Project Management Committee charged with the creation and maintenance > of open-source software, for distribution at no charge to the public, > related to an intermediate data service for big data computing engines > to boost performance, stability, and flexibility. > > NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee > (PMC), to be known as the "Apache Celeborn Project", be and hereby is > established pursuant to Bylaws of the Foundation; and be it further > > RESOLVED, that the Apache Celeborn Project be and hereby is responsible > for the creation and maintenance of software related to an intermediate > data service for big data computing engines to boost performance, > stability, and flexibility; and be it further > > RESOLVED, that the office of "Vice President, Apache Celeborn" be and > hereby is created, the person holding such office to serve at the > direction of the Board of Directors as the chair of the Apache Celeborn > Project, and to have primary responsibility for management of the > projects within the scope of responsibility of the Apache Celeborn > Project; and be it further > > RESOLVED, that the persons listed immediately below be and hereby are > appointed to serve as the initial members of the Apache Celeborn > Project: > > * Becket Qin > * Cheng Pan > * Duo Zhang > * Ethan Feng > * Fu Chen > * Jiashu Xiong > * Kerwin Zhang > * Keyong Zhou > * Lidong Dai > * Willem Ning Jiang > * Wu Wei > * Yi Zhu > * Yu Li > > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Keyong Zhou be appointed to > the office of Vice President, Apache Celeborn, to serve in accordance > with and subject to the direction of the Board of Directors and the > Bylaws of the Foundation until death, resignation, retirement, removal > or disqualification, or until a successor is appointed; and be it > further > > RESOLVED, that the Apache Celeborn Project be and hereby is tasked with > the migration and rationalization of the Apache Incubator Celeborn > podling; and be it further > > RESOLVED, that all responsibilities pertaining to the Apache Incubator > Celeborn podling encumbered upon the Apache Incubator PMC are hereafter > discharged. > -- > > Best Regards, > Yu > > [1] https://lists.apache.org/thread/z17rs0mw4nyv0s112dklmv7s3j053mby > [2] > https://cwiki.apache.org/confluence/display/CELEBORN/Apache+Maturity+Model+Assessment+for+Celeborn > [3] https://community.apache.org/apache-way/apache-project-maturity-model.html > [4] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-206 > [5] https://whimsy.apache.org/pods/project/celeborn
Re: [DISCUSS] Graduate Celeborn as TLP
The Apache Podling Website Checks[1] issue has been fixed now. [1] https://whimsy.apache.org/pods/project/celeborn Thanks, Cheng Pan > On Feb 29, 2024, at 13:41, Mridul Muralidharan wrote: > > +1 > Looking forward to Celeborn as a TLP ! > > Best wishes to the community :-) > > Regards, > Mridul > > > On Tue, Feb 27, 2024 at 5:23 AM Willem Jiang wrote: > >> Thanks for the clarification. Now we are good to go. >> >> Willem Jiang >> >> >> >> On Tue, Feb 27, 2024 at 7:15 PM Keyong Zhou wrote: >>> >>> Thanks Willian for the information, as Cheng said, we didn't start the >>> registration process before :) >>> >>> Best, >>> Keyong Zhou >>> >>> Willem Jiang 于2024年2月27日周二 18:48写道： >>> >>>> It‘s OK if we don't register any trademark of Celeborn. >>>> If we already registered the trademark of Celeborn, we need to have >>>> the approval of the trademark VP. >>>> >>>> Willem Jiang >>>> >>>> >>>> On Tue, Feb 27, 2024 at 6:19 PM Cheng Pan wrote: >>>>> >>>>> Hi Willem, >>>>> >>>>> For trademark concerns, the "Apache Celeborn” gets approval by >> ASF[1], >>>> do we need any additional work? >>>>> >>>>> [1] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-206 >>>>> >>>>> Thanks, >>>>> Cheng Pan >>>>> >>>>> >>>>>> On Feb 27, 2024, at 17:45, Willem Jiang >>>> wrote: >>>>>> >>>>>> +1， it's good to see Celeborn is ready for graduation. >>>>>> >>>>>> I have a quick question about Celeborn's trademark. Did we start >> the >>>>>> registration process before? >>>>>> >>>>>> BTW the podling name search is approved by trademark VP [1] >>>>>> >>>>>> [1] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-206 >>>>>> >>>>>> Willem Jiang >>>>>> >>>>>> On Tue, Feb 27, 2024 at 9:40 AM Yu Li wrote: >>>>>>> >>>>>>> Dear Celeborn Devs, >>>>>>> >>>>>>> We, the Celeborn community, began our incubation journey on >> October >>>>>>> 18, 2022. Since then, with the continuous efforts of you all, our >>>>>>> community has steadily developed and gradually matured, >> approaching >>>>>>> the graduation criteria [1]. Therefore, I'd like to call a >> discussion >>>>>>> to graduate Celeborn as TLP. Below are some statistics I >> collected, >>>>>>> please check it and let me know your thoughts. >>>>>>> >>>>>>> - Currently, our community consists of 19 committers (including >>>>>>> mentors) from more than 10 companies, with 12 serving as PPMC >> members >>>>>>> [2]. >>>>>>> - So far, we have boasted 81 contributors. >>>>>>> - Throughout the incubation period, we've made 6 releases [3] in >> 16 >>>>>>> months, at a stable pace. >>>>>>> - We've had 6 different release managers to date. >>>>>>> - Our software is used in production by 10+ well known entities >> [4]. >>>>>>> - As yet, we have opened 1,286 issues with 1,176 successfully >>>> resolved [5]. >>>>>>> - We have submitted a total of 1,816 PRs, out of which 1,805 have >> been >>>>>>> merged or closed [6]. >>>>>>> - Through self-assessment [7], we have met all maturity criteria >> as >>>>>>> outlined in [1]. >>>>>>> >>>>>>> And below is the drafted graduation resolution, JFYI: >>>>>>> -- >>>>>>> Establish the Apache Celeborn Project >>>>>>> >>>>>>> WHEREAS, the Board of Directors deems it to be in the best >> interests >>>> of >>>>>>> the Foundation and consistent with the Foundation's purpose to >>>> establish >>>>>>> a Project Management Committee charged with the creation and >>>> maintenance >>>>>>> of open-source softwa
Re: [DISCUSS] Graduate Celeborn as TLP
Hi Willem, For trademark concerns, the "Apache Celeborn” gets approval by ASF[1], do we need any additional work? [1] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-206 Thanks, Cheng Pan > On Feb 27, 2024, at 17:45, Willem Jiang wrote: > > +1， it's good to see Celeborn is ready for graduation. > > I have a quick question about Celeborn's trademark. Did we start the > registration process before? > > BTW the podling name search is approved by trademark VP [1] > > [1] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-206 > > Willem Jiang > > On Tue, Feb 27, 2024 at 9:40 AM Yu Li wrote: >> >> Dear Celeborn Devs, >> >> We, the Celeborn community, began our incubation journey on October >> 18, 2022. Since then, with the continuous efforts of you all, our >> community has steadily developed and gradually matured, approaching >> the graduation criteria [1]. Therefore, I'd like to call a discussion >> to graduate Celeborn as TLP. Below are some statistics I collected, >> please check it and let me know your thoughts. >> >> - Currently, our community consists of 19 committers (including >> mentors) from more than 10 companies, with 12 serving as PPMC members >> [2]. >> - So far, we have boasted 81 contributors. >> - Throughout the incubation period, we've made 6 releases [3] in 16 >> months, at a stable pace. >> - We've had 6 different release managers to date. >> - Our software is used in production by 10+ well known entities [4]. >> - As yet, we have opened 1,286 issues with 1,176 successfully resolved [5]. >> - We have submitted a total of 1,816 PRs, out of which 1,805 have been >> merged or closed [6]. >> - Through self-assessment [7], we have met all maturity criteria as >> outlined in [1]. >> >> And below is the drafted graduation resolution, JFYI: >> -- >> Establish the Apache Celeborn Project >> >> WHEREAS, the Board of Directors deems it to be in the best interests of >> the Foundation and consistent with the Foundation's purpose to establish >> a Project Management Committee charged with the creation and maintenance >> of open-source software, for distribution at no charge to the public, >> related to an intermediate data service for big data computing engines >> to boost performance, stability, and flexibility. >> >> NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee >> (PMC), to be known as the "Apache Celeborn Project", be and hereby is >> established pursuant to Bylaws of the Foundation; and be it further >> >> RESOLVED, that the Apache Celeborn Project be and hereby is responsible >> for the creation and maintenance of software related to an intermediate >> data service for big data computing engines to boost performance, >> stability, and flexibility; and be it further >> >> RESOLVED, that the office of "Vice President, Apache Celeborn" be and >> hereby is created, the person holding such office to serve at the >> direction of the Board of Directors as the chair of the Apache Celeborn >> Project, and to have primary responsibility for management of the >> projects within the scope of responsibility of the Apache Celeborn >> Project; and be it further >> >> RESOLVED, that the persons listed immediately below be and hereby are >> appointed to serve as the initial members of the Apache Celeborn >> Project: >> >> * Becket Qin >> * Cheng Pan >> * Duo Zhang >> * Ethan Feng >> * Fu Chen >> * Jiashu Xiong >> * Kerwin Zhang >> * Keyong Zhou >> * Lidong Dai >> * Willem Ning Jiang >> * Wu Wei >> * Yi Zhu >> * Yu Li >> >> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Keyong Zhou be appointed to >> the office of Vice President, Apache Celeborn, to serve in accordance >> with and subject to the direction of the Board of Directors and the >> Bylaws of the Foundation until death, resignation, retirement, removal >> or disqualification, or until a successor is appointed; and be it >> further >> >> RESOLVED, that the Apache Celeborn Project be and hereby is tasked with >> the migration and rationalization of the Apache Incubator Celeborn >> podling; and be it further >> >> RESOLVED, that all responsibilities pertaining to the Apache Incubator >> Celeborn podling encumbered upon the Apache Incubator PMC are hereafter >> discharged. >> -- >> >> Best Regards, >> Yu >> >> [1] https://incubator.apache.org/guides/graduation.html >> [2] https://celeborn.apache.org/community/project_management_committee >> [3] >> https://issues.apache.org/jira/projects/CELEBORN?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page&status=released >> [4] https://github.com/apache/incubator-celeborn/issues/2140 >> [5] https://s.apache.org/celeborn_jira_issues >> [6] https://github.com/apache/incubator-celeborn/pulls >> [7] >> https://cwiki.apache.org/confluence/display/CELEBORN/Apache+Maturity+Model+Assessment+for+Celeborn
Re: [DISCUSS] Graduate Celeborn as TLP
Thanks, Yu, for driving this, overall I agree we can graduate and move forward. I just found there are minor issues on the Podling Website Checks[1], the PPMC is actively working on this. [1] https://whimsy.apache.org/pods/project/celeborn Thanks, Cheng Pan > On Feb 27, 2024, at 10:58, Nicholas Jiang wrote: > > Hi Yu, > > > > > +1. Celeborn has active community with much contribution of developers and > many company production practice including my company bilibili. It's time to > start the graduation procedure. Forward to the graduation of Apache Celeborn. > > > > > Regards, > > Nicholas Jiang > > > At 2024-02-27 09:40:04, "Yu Li" wrote: >> Dear Celeborn Devs, >> >> We, the Celeborn community, began our incubation journey on October >> 18, 2022. Since then, with the continuous efforts of you all, our >> community has steadily developed and gradually matured, approaching >> the graduation criteria [1]. Therefore, I'd like to call a discussion >> to graduate Celeborn as TLP. Below are some statistics I collected, >> please check it and let me know your thoughts. >> >> - Currently, our community consists of 19 committers (including >> mentors) from more than 10 companies, with 12 serving as PPMC members >> [2]. >> - So far, we have boasted 81 contributors. >> - Throughout the incubation period, we've made 6 releases [3] in 16 >> months, at a stable pace. >> - We've had 6 different release managers to date. >> - Our software is used in production by 10+ well known entities [4]. >> - As yet, we have opened 1,286 issues with 1,176 successfully resolved [5]. >> - We have submitted a total of 1,816 PRs, out of which 1,805 have been >> merged or closed [6]. >> - Through self-assessment [7], we have met all maturity criteria as >> outlined in [1]. >> >> And below is the drafted graduation resolution, JFYI: >> -- >> Establish the Apache Celeborn Project >> >> WHEREAS, the Board of Directors deems it to be in the best interests of >> the Foundation and consistent with the Foundation's purpose to establish >> a Project Management Committee charged with the creation and maintenance >> of open-source software, for distribution at no charge to the public, >> related to an intermediate data service for big data computing engines >> to boost performance, stability, and flexibility. >> >> NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee >> (PMC), to be known as the "Apache Celeborn Project", be and hereby is >> established pursuant to Bylaws of the Foundation; and be it further >> >> RESOLVED, that the Apache Celeborn Project be and hereby is responsible >> for the creation and maintenance of software related to an intermediate >> data service for big data computing engines to boost performance, >> stability, and flexibility; and be it further >> >> RESOLVED, that the office of "Vice President, Apache Celeborn" be and >> hereby is created, the person holding such office to serve at the >> direction of the Board of Directors as the chair of the Apache Celeborn >> Project, and to have primary responsibility for management of the >> projects within the scope of responsibility of the Apache Celeborn >> Project; and be it further >> >> RESOLVED, that the persons listed immediately below be and hereby are >> appointed to serve as the initial members of the Apache Celeborn >> Project: >> >> * Becket Qin >> * Cheng Pan >> * Duo Zhang >> * Ethan Feng >> * Fu Chen >> * Jiashu Xiong >> * Kerwin Zhang >> * Keyong Zhou >> * Lidong Dai >> * Willem Ning Jiang >> * Wu Wei >> * Yi Zhu >> * Yu Li >> >> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Keyong Zhou be appointed to >> the office of Vice President, Apache Celeborn, to serve in accordance >> with and subject to the direction of the Board of Directors and the >> Bylaws of the Foundation until death, resignation, retirement, removal >> or disqualification, or until a successor is appointed; and be it >> further >> >> RESOLVED, that the Apache Celeborn Project be and hereby is tasked with >> the migration and rationalization of the Apache Incubator Celeborn >> podling; and be it further >> >> RESOLVED, that all responsibilities pertaining to the Apache Incubator >> Celeborn podling encumbered upon the Apache Incubator P
Re: [ANNONCE] New PPMC member: Fu Chen
Congrats! Thanks, Cheng Pan > On Feb 20, 2024, at 08:22, Nicholas wrote: > > Congratulations to Fu Chen!Regards, > Nicholas Jiang > > > > > At 2024-02-20 00:23:06, "Shaoyun Chen" wrote: >> Congratulations! >> >> Keyong Zhou 于2024年2月19日周一 21:16写道： >>> >>> Hi Celeborn Community, >>> >>> The Podling Project Management Committee (PPMC) for Apache Celeborn >>> has invited Fu Chen to become our PPMC member and >>> we are pleased to announce that he has accepted. >>> >>> Fu Chen has been actively contributing to Celeborn community for more then >>> one year[1], including SBT build, >>> performance improvement, code refactor, bug fixes, code reviews, design >>> discussion, docs, etc. >>> >>> Please join me in congratulating Fu Chen! >>> >>> Being a committer enables easier contribution to the >>> project since there is no need to go via the patch >>> submission process. This should enable better productivity. >>> A PPMC member helps manage and guide the direction of the project. >>> >>> [1] https://github.com/apache/incubator-celeborn/commits?author=cfmcgrady >>> >>> Thanks, >>> On behalf of the Apache Celeborn PPMC
Re: [VOTE] Release Apache Celeborn(Incubating) 0.4.0-incubating-rc4
+1 binding I checked - links are valid - build success from source (CentOS 7, x86, OpenJDK 8) - LICENSE and NOTICE look good - DISCLAIMER exist - incubating present in artifacts name Thanks, Cheng Pan > On Jan 19, 2024, at 15:29, Ethan Feng wrote: > > +1(binding) > > I checked: > √ release files in the correct location > √ release files have the word incubating > √ digital signature and hashes correct > √ DISCLAIMER file exist > √ LICENSE and NOTICE files exist and are correct > √ the contents of the release match the tag in VCS > √ can build the release from the source > √ maven artifacts look correct > > Thanks, > Ethan Feng > > Yihe Li 于2024年1月19日周五 14:28写道： >> >> +1 (non-binding) >> I checked the following things: >> - git commit hash is correct. >> - download links are valid. >> - release files are in correct location. >> - release files have the word incubating in their name. >> - signatures and checksums are good. >> - DISCLAIMER, LICENSE and NOTICE files exist. >> - build success from source code(ubuntu 16.04). >> ``` >> ./build/make-distribution.sh --release >> real30m46.326s >> user154m25.760s >> sys 29m14.300s >> ./build/make-distribution.sh --sbt-enabled --release >> real12m57.246s >> user68m48.212s >> sys 10m8.060s >> ``` >> >> Thanks, >> Yihe Li >> >> On 2024/01/19 02:24:53 Keyong Zhou wrote: >>> +1 (binding) >>> >>> I checked >>> - git commit hash is correct. >>> - links are valid. >>> - "incubating" is in the name. >>> - signatures are good. >>> ``` >>> gpg --import KEYS >>> gpg --verify apache-celeborn-0.4.0-incubating-source.tgz.asc >>> gpg --verify apache-celeborn-0.4.0-incubating-bin.tgz.asc >>> ``` >>> - checksums are good. >>> ``` >>> sha512sum --check apache-celeborn-0.4.0-incubating-source.tgz.sha512 >>> sha512sum --check apache-celeborn-0.4.0-incubating-bin.tgz.sha512 >>> ``` >>> - LICENSE looks good. >>> - NOTICE looks good. >>> - DISCLAIMER exists. >>> - build success from source code (macOS). >>> ``` >>> ./build/make-distribution.sh --sbt-enabled --release >>> ``` >>> >>> Thanks, >>> Keyong Zhou >>> >>> Fu Chen 于2024年1月18日周四 21:40写道： >>> >>>> Hi Celeborn community, >>>> >>>> This is a call for a vote to release Apache Celeborn (Incubating) >>>> 0.4.0-incubating-rc4 >>>> >>>> >>>> The git tag to be voted upon: >>>> >>>> https://github.com/apache/incubator-celeborn/releases/tag/v0.4.0-incubating-rc4 >>>> >>>> >>>> The git commit hash: >>>> 8bc07466dd85a90216820617015e329fb806c7dd source and binary artifacts can be >>>> found at: >>>> >>>> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.4.0-incubating-rc4 >>>> >>>> >>>> The staging repo: >>>> https://repository.apache.org/content/repositories/orgapacheceleborn-1051 >>>> >>>> >>>> Fingerprint of the PGP key release artifacts are signed with: >>>> 92AF4750DAFCB5E25B5B83EA76F54B977EB5C09B >>>> >>>> >>>> My public key to verify signatures can be found in: >>>> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >>>> >>>> >>>> The vote will be open for at least 72 hours or until the necessary >>>> number of votes are reached. >>>> >>>> >>>> Please vote accordingly: >>>> >>>> >>>> [ ] +1 approve >>>> [ ] +0 no opinion >>>> [ ] -1 disapprove (and the reason) >>>> >>>> >>>> Checklist for release: >>>> >>>> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist >>>> Steps to validate the release: >>>> https://www.apache.org/info/verification.html >>>> >>>> >>>> * Download links, checksums and PGP signatures are valid. >>>> * Source code distributions have correct names matching the current >>>> release. >>>> * Release files have the word incubating in their name. >>>> * DISCLAIMER, LICENSE and NOTICE files are correct. >>>> * All files have license headers if necessary. >>>> * No unlicensed compiled archives bundled in source archive. >>>> * The source tarball matches the git tag. >>>> * Build from source is successful. >>>> >>>> Please be aware that there has been a transition in the Celeborn project's >>>> build tool, shifting from Maven to SBT. The SBT build documentation is >>>> available >>>> at https://celeborn.apache.org/docs/latest/developers/sbt/. >>>> >>>> For illustrative purposes: >>>> >>>> Packaging the project >>>> ``` >>>> ./build/sbt clean package >>>> ``` >>>> >>>> Creating the distribution >>>> ``` >>>> ./build/make-distribution.sh --sbt-enabled --release >>>> ``` >>>> >>>> Thanks, >>>> Fu Chen >>>> >>>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.4.0-incubating-rc1
-1 because of the license issue. The non-fat jar (e.g. celeborn-worker_2.12-0.4.0-incubating.jar) should also ship the NOTICE and LICENSE file, usually under the root folder or META-INF. Thanks, Cheng Pan On Mon, Dec 25, 2023 at 1:10 PM Shaoyun Chen wrote: > > +1 (non-binding) > > I checked the following things: > > - signatures are good. > ``` > gpg --import KEYS > gpg --verify apache-celeborn-0.4.0-incubating-source.tgz.asc > gpg --verify apache-celeborn-0.4.0-incubating-bin.tgz.asc > ``` > - checksums are good. > ``` > sha512sum --check apache-celeborn-0.4.0-incubating-source.tgz.sha512 > sha512sum --check apache-celeborn-0.4.0-incubating-bin.tgz.sha512 > ``` > - build success from source code. > ``` > ./build/make-distribution.sh --release > ./build/make-distribution.sh --sbt-enabled --release > ``` > time > ``` > ./build/make-distribution.sh --release 1779.60s user 89.17s system > 311% cpu 10:00.60 total > ``` > ``` > ./build/make-distribution.sh --sbt-enabled --release 893.63s user > 40.47s system 361% cpu 4:18.59 total > ``` > > Yihe Li 于2023年12月25日周一 11:48写道： > > > > +1 (non-binding) > > I checked the following things: > > - git commit hash is correct. > > - download links are valid. > > - release files are in correct location. > > - release files have the word incubating in their name. > > - signatures and checksums are good. > > - DISCLAIMER, LICENSE and NOTICE files exist. > > - build success from source code(ubuntu 16.04). > > ``` > > ./build/make-distribution.sh --release > > ./build/make-distribution.sh -Pspark-3.3 > > ``` > > > > Thanks, > > Yihe Li > > > > On 2023/12/25 02:15:00 Ethan Feng wrote: > > > +1(binding) > > > > > > I checked: > > > √ release files in the correct location > > > √ release files have the word incubating > > > √ digital signature and hashes correct > > > √ DISCLAIMER file exist > > > √ LICENSE and NOTICE files exist and are correct > > > √ the contents of the release match the tag in VCS > > > √ can build the release from the source > > > √ maven artifacts look correct > > > > > > > > > Thanks, > > > Ethan Feng > > > > > > Fu Chen 于2023年12月23日周六 00:05写道： > > > > > > > > Hi Celeborn community, > > > > > > > > This is a call for a vote to release Apache Celeborn (Incubating) > > > > 0.4.0-incubating-rc1 > > > > > > > > > > > > The git tag to be voted upon: > > > > https://github.com/apache/incubator-celeborn/releases/tag/v0.4.0-incubating-rc1 > > > > > > > > > > > > The git commit hash: > > > > 6efa068c6715e80e4ddee6bc37d6aaf5bf5251fa source and binary artifacts > > > > can be > > > > found at: > > > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.4.0-incubating-rc1 > > > > > > > > > > > > The staging repo: > > > > https://repository.apache.org/content/repositories/orgapacheceleborn-1047 > > > > > > > > > > > > Fingerprint of the PGP key release artifacts are signed with: > > > > 92AF4750DAFCB5E25B5B83EA76F54B977EB5C09B > > > > > > > > > > > > My public key to verify signatures can be found in: > > > > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > > > > > > > > > > The vote will be open for at least 72 hours or until the necessary > > > > number of votes are reached. > > > > > > > > > > > > Please vote accordingly: > > > > > > > > > > > > [ ] +1 approve > > > > [ ] +0 no opinion > > > > [ ] -1 disapprove (and the reason) > > > > > > > > > > > > Checklist for release: > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > > > Steps to validate the release: > > > > https://www.apache.org/info/verification.html > > > > > > > > > > > > * Download links, checksums and PGP signatures are valid. > > > > * Source code distributions have correct names matching the current > > > > release. > > > > * Release files have the word incubating in their name. > > > > * DISCLAIMER, LICENSE and NOTICE files are correct. > > > > * All files have license headers if necessary. > > > > * No unlicensed compiled archives bundled in source archive. > > > > * The source tarball matches the git tag. > > > > * Build from source is successful. > > > > > > > > Please be aware that there has been a transition in the Celeborn > > > > project's > > > > build tool, shifting from Maven to SBT. The SBT build documentation is > > > > available > > > > at https://celeborn.apache.org/docs/latest/developers/sbt/. > > > > > > > > For illustrative purposes: > > > > > > > > Packaging the project > > > > ``` > > > > ./build/sbt clean package > > > > ``` > > > > > > > > Creating the distribution > > > > ``` > > > > ./build/make-distribution.sh --sbt-enabled --release > > > > ``` > > > > > > > > Thanks, > > > > Fu Chen > > >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.2-incubating-rc1
+1 - all links are valid - checksums and signatures are good - artifacts name contain “incubating” - build success on Ubuntu 22.04 JDK 17 x86 with the command: `build/make-distribution.sh -Pspark-3.3` Cheng Pan > On Dec 22, 2023, at 12:16, Shaoyun Chen wrote: > > +1 (non-binding) > > I checked the following things: > > - signatures are good. > ``` > gpg --import KEYS > gpg --verify apache-celeborn-0.3.2-incubating-source.tgz.asc > gpg --verify apache-celeborn-0.3.2-incubating-bin.tgz.asc > ``` > - checksums are good. > ``` > sha512sum --check apache-celeborn-0.3.2-incubating-source.tgz.sha512 > sha512sum --check apache-celeborn-0.3.2-incubating-bin.tgz.sha512 > ``` > - build success from source code. > ``` > ./build/make-distribution.sh -Pspark-3.2 > ./build/make-distribution.sh --release > ``` > > rexxiong 于2023年12月22日周五 11:09写道： >> >> +1 (binding) >> I checked >> - Download links are valid. >> - git commit hash is correct >> - Checksums and signatures are valid. >> - No binary files in the source release >> - Files have the word incubating in their name. >> - DISCLAIMER,LICENSE and NOTICE files exist. >> >> Thanks, >> Jiashu Xiong >> >> Fu Chen 于2023年12月21日周四 21:57写道： >> >>> +1 >>> >>> I checked >>> - download links are valid. >>> - git commit hash is correct. >>> - no binary files in the source release. >>> - signatures are good. >>> ``` >>> gpg --import KEYS >>> gpg --verify apache-celeborn-0.3.2-incubating-source.tgz.asc >>> gpg --verify apache-celeborn-0.3.2-incubating-bin.tgz.asc >>> ``` >>> - checksums are good. >>> ``` >>> sha512sum --check apache-celeborn-0.3.2-incubating-source.tgz.sha512 >>> sha512sum --check apache-celeborn-0.3.2-incubating-bin.tgz.sha512 >>> ``` >>> - build success from source code (Pop!_OS 22.04 LTS). >>> ``` >>> ./build/mvn clean package -DskipTests -Pspark-3.4 >>> ``` >>> >>> Keyong Zhou 于2023年12月21日周四 16:55写道： >>> >>>> +1 (binding) >>>> >>>> I checked >>>> - git commit hash is correct. >>>> - links are valid. >>>> - "incubating" is in the name. >>>> - PGP keys are good. >>>> - hashes are correct. >>>> - LICENSE looks good. >>>> - NOTICE looks good. >>>> - DISCLAIMER exists. >>>> - build success from source code (macOS). ``` >>> ./build/make-distribution.sh >>>> --release ``` >>>> >>>> Thanks, >>>> Keyong Zhou >>>> >>>> Nicholas Jiang 于2023年12月21日周四 14:06写道： >>>> >>>>> Hi Celeborn community, >>>>> >>>>> >>>>> This is a call for a vote to release Apache Celeborn (Incubating) >>>>> 0.3.2-incubating-rc1 >>>>> >>>>> >>>>> The git tag to be voted upon: >>>>> >>>>> >>>> >>> https://github.com/apache/incubator-celeborn/releases/tag/v0.3.2-incubating-rc1 >>>>> >>>>> >>>>> The git commit hash: >>>>> bce190d8a0a53434ef57ef33e53720f5bf4d14d6 source and binary artifacts >>> can >>>> be >>>>> found at: >>>>> >>>>> >>>> >>> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.2-incubating-rc1 >>>>> >>>>> >>>>> The staging repo: >>>>> >>>> >>> https://repository.apache.org/content/repositories/orgapacheceleborn-1045 >>>>> >>>>> >>>>> Fingerprint of the PGP key release artifacts are signed with: >>>>> D73CADC1DAB63BD3C770BB6D9476842D24B7C885 >>>>> >>>>> >>>>> My public key to verify signatures can be found in: >>>>> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >>>>> >>>>> >>>>> The vote will be open for at least 72 hours or until the necessary >>>>> number of votes are reached. >>>>> >>>>> >>>>> Please vote accordingly: >>>>> >>>>> >>>>> [ ] +1 approve >>>>> [ ] +0 no opinion >>>>> [ ] -1 disapprove (and the reason) >>>>> >>>>> >>>>> Checklist for release: >>>>> >>>>> >>>> >>> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist >>>>> Steps to validate the release: >>>>> https://www.apache.org/info/verification.html >>>>> >>>>> >>>>> * Download links, checksums and PGP signatures are valid. >>>>> * Source code distributions have correct names matching the current >>>>> release. >>>>> * Release files have the word incubating in their name. >>>>> * DISCLAIMER, LICENSE and NOTICE files are correct. >>>>> * All files have license headers if necessary. >>>>> * No unlicensed compiled archives bundled in source archive. >>>>> * The source tarball matches the git tag. >>>>> * Build from source is successful. >>>>> >>>>> >>>>> Regards, >>>>> Nicholas Jiang >>>> >>>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.4.0-incubating-rc0
Kindly reminder, this release is packaged via SBT, better to use `./build/make-distribution.sh --sbt-enabled —release` for building check. > On Dec 22, 2023, at 12:48, Ethan Feng wrote: > > -1(binding) > > There is a bug that needs to be fixed. > > Thanks, > Ethan Feng. > > rexxiong 于2023年12月22日周五 11:35写道： >> >> +1 (binding) >> I checked >> - Download links are valid. >> - git commit hash is correct >> - Checksums and signatures are valid. >> - No binary files in the source release >> - Files have the word incubating in their name. >> - DISCLAIMER,LICENSE and NOTICE files exist. >> - Successfully built the binary from the source on MacOs with Command: >> ./build/make-distribution.sh -Pspark-3.3 >> >> Thanks, >> Jiashu Xiong >> >> Nicholas Jiang 于2023年12月22日周五 11:05写道： >> >>> +1 (non-binding) >>> >>> I checked >>> - Download links are valid. >>> - Checksums and signatures are valid. >>> - Git commit hash is correct >>> - No binary files in the source release >>> - Files have the word incubating in names. >>> - DISCLAIMER,LICENSE and NOTICE files exist. >>> - Successfully built the binary from the source via command: >>> ./build/make-distribution.sh --release >>> >>> Regards, >>> Nicholas Jiang >>> >>> On 2023/12/21 13:41:36 Fu Chen wrote: Hi Celeborn community, This is a call for a vote to release Apache Celeborn (Incubating) 0.4.0-incubating-rc0 The git tag to be voted upon: >>> https://github.com/apache/incubator-celeborn/releases/tag/v0.4.0-incubating-rc0 The git commit hash: de6d8d69af3381ee899ba8d92c5d63b332cbdfbf source and binary artifacts can >>> be found at: >>> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.4.0-incubating-rc0 The staging repo: >>> https://repository.apache.org/content/repositories/orgapacheceleborn-1046 Fingerprint of the PGP key release artifacts are signed with: 92AF4750DAFCB5E25B5B83EA76F54B977EB5C09B My public key to verify signatures can be found in: https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS The vote will be open for at least 72 hours or until the necessary number of votes are reached. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason) Checklist for release: >>> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist Steps to validate the release: https://www.apache.org/info/verification.html * Download links, checksums and PGP signatures are valid. * Source code distributions have correct names matching the current >>> release. * Release files have the word incubating in their name. * DISCLAIMER, LICENSE and NOTICE files are correct. * All files have license headers if necessary. * No unlicensed compiled archives bundled in source archive. * The source tarball matches the git tag. * Build from source is successful. Please be aware that there has been a transition in the Celeborn >>> project's build tool, shifting from Maven to SBT. The SBT build documentation is available at https://celeborn.apache.org/docs/latest/developers/sbt/. For illustrative purposes: Packaging the project ``` ./build/sbt clean package ``` Creating the distribution ``` ./build/make-distribution.sh --sbt-enabled --release ``` Thanks, Fu Chen >>>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.2-incubating-rc0
+1 - Checked all links are valid - GitHub CI result is good Thanks, Cheng Pan > On Dec 20, 2023, at 10:38, Mridul Muralidharan wrote: > > +1 > > Signatures, digests, license, etc check out fine. > Checked out tag and build/tested with -Pspark3.1 and -Pflink-1.17 > > Regards, > Mridul > > > On Tue, Dec 19, 2023 at 8:06 PM rexxiong wrote: > >> +1 (binding) >> I checked >> - Download links are valid. >> - git commit hash is correct >> - Checksums and signatures are valid. >> - No binary files in the source release >> - Files have the word incubating in their name. >> - DISCLAIMER,LICENSE and NOTICE files exist. >> - Successfully built the binary from the source on MacOs with Command: >> ./build/make-distribution.sh --release >> >> Thanks, >> Jiashu Xiong >> >> Fu Chen 于2023年12月20日周三 00:18写道： >> >>> +1 >>> >>> I checked >>> - download links are valid. >>> - git commit hash is correct. >>> - no binary files in the source release. >>> - signatures are good. >>> ``` >>> gpg --import KEYS >>> gpg --verify apache-celeborn-0.3.2-incubating-source.tgz.asc >>> gpg --verify apache-celeborn-0.3.2-incubating-bin.tgz.asc >>> ``` >>> - checksums are good. >>> ``` >>> sha512sum --check apache-celeborn-0.3.2-incubating-source.tgz.sha512 >>> sha512sum --check apache-celeborn-0.3.2-incubating-bin.tgz.sha512 >>> ``` >>> - build success from source code (Pop!_OS 22.04 LTS). >>> ``` >>> ./build/mvn clean package -DskipTests -Pspark-3.4 >>> ``` >>> >>> Shaoyun Chen 于2023年12月19日周二 22:45写道： >>> >>>> +1 (non-binding) >>>> >>>> I checked the following things: >>>> >>>> - signatures are good. >>>> ``` >>>> gpg --import KEYS >>>> gpg --verify apache-celeborn-0.3.2-incubating-source.tgz.asc >>>> gpg --verify apache-celeborn-0.3.2-incubating-bin.tgz.asc >>>> ``` >>>> - checksums are good. >>>> ``` >>>> sha512sum --check apache-celeborn-0.3.2-incubating-source.tgz.sha512 >>>> sha512sum --check apache-celeborn-0.3.2-incubating-bin.tgz.sha512 >>>> ``` >>>> - build success from source code. >>>> ``` >>>> ./build/make-distribution.sh -Pspark-3.2 >>>> ./build/make-distribution.sh --release >>>> ``` >>>> >>>> Yihe Li 于2023年12月19日周二 20:56写道： >>>>> >>>>> +1 (non-binding) >>>>> I checked the following things: >>>>> - git commit hash is correct. >>>>> - download links are valid. >>>>> - release files are in correct location. >>>>> - release files have the word incubating in their name. >>>>> - signatures and checksums are good. >>>>> - DISCLAIMER, LICENSE and NOTICE files exist. >>>>> - build success from source code(ubuntu 16.04). >>>>> ``` >>>>> ./build/make-distribution.sh --release >>>>> ./build/make-distribution.sh --Pspark-3.3 >>>>> ``` >>>>> >>>>> Thanks, >>>>> Yihe Li >>>>> >>>>> On 2023/12/19 06:37:40 Ethan Feng wrote: >>>>>> +1(binding) >>>>>> >>>>>> I checked: >>>>>> √ release files in the correct location >>>>>> √ release files have the word incubating >>>>>> √ digital signature and hashes correct >>>>>> √ DISCLAIMER file exist >>>>>> √ LICENSE and NOTICE files exist and are correct >>>>>> √ the contents of the release match the tag in VCS >>>>>> √ can build the release from the source >>>>>> √ maven artifacts looks correct >>>>>> >>>>>> Thanks, >>>>>> Ethan Feng >>>>>> >>>>>> Nicholas Jiang 于2023年12月19日周二 12:32写道： >>>>>>> >>>>>>> Hi Celeborn community, >>>>>>> >>>>>>> >>>>>>> This is a call for a vote to release Apache Celeborn (Incubating) >>>>>>> 0.3.2-incubating-rc0 >>>>>>> >>>>>>> >>>>>>> The git tag to be voted upon: >>>>>>> >>>> >>> >> https://github.com/apache/incubator-celeborn/releases/tag
Re: [DISCUSS] Time for 0.4.0
+1, thanks Fu for volunteering. With CELEBORN-989 and CELEBORN-1156, Celeborn is ready for release using SBT, it would be great to try in 0.4.0. If everything goes well, we can drop the Maven building system in the next version as we planned. Thanks, Cheng Pan > On Dec 14, 2023, at 10:27, keyong zhou wrote: > > +1, thanks Fu! > > Fu Chen 于2023年12月14日周四 10:26写道： > >> Hi, Celeborn community, >> >> It has been a while since the 0.3.0 release, and I think it’s time to >> prepare >> for the next feature release 0.4.0. And I’m volunteering to be the release >> manager if no others were applied. >> >> If no objections, I plan to cut branch-0.4 next week. >> >> Thanks, >> Fu Chen >>
Re: [DISCUSS] Time for 0.3.2
+1, thanks for volunteering. Feel free to ping me if you encounter permission issues during the release phase. Thanks, Cheng Pan > On Dec 7, 2023, at 14:31, Nicholas wrote: > > Hey, Celeborn community, > > It has been a while since the 0.3.1 release, and there are some critical > fixes land branch-0.3, for example, [CELEBORN-1037] Incorrect output for > metrics of Prometheus. From my perspective, it’s time to prepare for > releasing 0.3.2. > > WDYT? And I’m volunteering to be the release manager if no one has applied. > > Regards, > Nicholas Jiang
Re: Add k8s operator for celeborn
Thanks for bringing this operator! I have a quick look at the JIRA and README, could you please elaborate more about details? > the operator can help us to manage the celeborn clusters. Additionally, please consider providing minikube/k3s-based integration tests(or setup steps), so that other developers can play with it quickly without additional testing K8s cluster requirements. Thanks, Cheng Pan > On Nov 8, 2023, at 16:53, leo65535 wrote: > > > > Hello everyone, > > > As we known, kubernetes is designed for automation, Kubernetes' operator > pattern concept > lets you extend the cluster's behaviour without modifying the code of > Kubernetes itself > by linking controllers to one or more custom resources. Operators are clients > of the > Kubernetes API that act as controllers for a Custom Resource. > > > So we can use k8s operator sdk to write our own operator for celeborn. the > operator can > help us to manage the celeborn clusters. > > > Please check the jira[1] for more detail. > > > Looking forward to your feedback and suggestions, thanks. > > > Bests, > xleoken > > > [1] > https://issues.apache.org/jira/browse/CELEBORN-1117 >
[ANNOUNCE] New Committer: Shaoyun Chen
Hi Celeborn Community, The Podling Project Management Committee (PPMC) for Apache Celeborn has invited Shaoyun Chen to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. A (P)PMC member helps manage and guide the direction of the project. Please join me in congratulating Shaoyun Chen! Thanks, Cheng Pan
[ANNOUNCE] Apache Celeborn(incubating) 0.3.1 available
Hi all, Apache Celeborn(Incubating) community is glad to announce the new release of Apache Celeborn(Incubating) 0.3.1. Celeborn is dedicated to improving the efficiency and elasticity of different map-reduce engines and provides an elastic, high-efficient service for intermediate data including shuffle data, spilled data, result data, etc. Download Link: https://celeborn.apache.org/download/ GitHub Release Tag: - https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating Release Notes: - https://celeborn.apache.org/community/release_notes/release_note_0.3.1 Home Page: https://celeborn.apache.org/ Celeborn Resources: - Issue Management: https://issues.apache.org/jira/projects/CELEBORN - Mailing List: dev@celeborn.apache.org Thanks, Cheng Pan On behalf of the Apache Celeborn(incubating) community
Re: [RESULT][VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc3
Sorry, with additional non-binding +1 from Kent Yao Thanks, Cheng Pan > On Oct 13, 2023, at 20:17, Cheng Pan wrote: > > Hi IMPC, > > The vote to release Apache Celeborn(Incubating) 0.3.1-incubating-rc3 > PASSES with +3 (binding), +1 (non-binding), and no +0 or -1. > > +1 (binding): > Willem Jiang > Duo Zhang > Gang Li > > +1 (non-binding): > Cheng Pan > > The vote thread: > https://lists.apache.org/thread/g9ctor045zv0v9o0y2vocdj44k75rfst > > Many thanks to our mentors and IPMC and all who helped with the review > and vote for the Apache Celeborn (Incubating) release. > > I will publish the releases and make an announcement. > > Thanks, > Cheng Pan > >
[RESULT][VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc3
Hi IMPC, The vote to release Apache Celeborn(Incubating) 0.3.1-incubating-rc3 PASSES with +3 (binding), +1 (non-binding), and no +0 or -1. +1 (binding): Willem Jiang Duo Zhang Gang Li +1 (non-binding): Cheng Pan The vote thread: https://lists.apache.org/thread/g9ctor045zv0v9o0y2vocdj44k75rfst Many thanks to our mentors and IPMC and all who helped with the review and vote for the Apache Celeborn (Incubating) release. I will publish the releases and make an announcement. Thanks, Cheng Pan
[RESULT][VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc3
Hello Apache Celeborn PPMC and Community, The vote closes now as 72hr have passed. The vote PASSES with (* = binding) +1: - Jiashu Xiong * - Keyong Zhou * - Shaoyun Chen - Fu Chen - Cheng Pan * +0: None -1: None The vote thread: https://lists.apache.org/thread/wt69x5h3jxrmozkjgh2v544sgpdqhnvq I will now bring the vote to gene...@incubator.apache.org to get approval by the IPMC. If this vote passes too, the release is accepted and will be published. Thanks to all who helped with the release! Thanks, On behalf of Apache Celeborn(Incubating) community
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc3
Also +1 from me. Thanks all, the vote passed, I will send the result soon. Thanks, Cheng Pan > On Oct 7, 2023, at 12:12, rexxiong wrote: > > +1 (binding) > I checked > - Download links are valid. > - git commit hash is correct > - Checksums and signatures are valid. > - No binary files in the source release > - Files have the word incubating in their name. > - DISCLAIMER,LICENSE and NOTICE files exist. > - Successfully built the binary from the source on MacOs with Command: > ./build/make-distribution.sh --release > > Thanks, > Jiashu Xiong > > Keyong Zhou 于2023年10月7日周六 11:42写道： > >> +1 (binding) >> >> I checked >> - git commit hash is correct. >> - links are valid. >> - "incubating" is in the name. >> - PGP keys are good. >> - hashes are correct. >> - LICENSE looks good. >> - NOTICE looks good. >> - DISCLAIMER exists. >> - build success from source code (macOS). ``` ./build/make-distribution.sh >> --release ``` >> >> Thanks, >> Keyong Zhou >> >> Fu Chen 于2023年10月7日周六 10:27写道： >> >>> +1 >>> >>> I checked >>> - download links are valid. >>> - git commit hash is correct. >>> - no binary files in the source release. >>> - signatures are good. >>> ``` >>> gpg --import KEYS >>> gpg --verify apache-celeborn-0.3.1-incubating-source.tgz.asc >>> gpg --verify apache-celeborn-0.3.1-incubating-bin.tgz.asc >>> ``` >>> - checksums are good. >>> ``` >>> sha512sum --check apache-celeborn-0.3.1-incubating-source.tgz.sha512 >>> sha512sum --check apache-celeborn-0.3.1-incubating-bin.tgz.sha512 >>> ``` >>> - build success from source code (ubuntu 18.04). >>> ``` >>> ./build/mvn clean package -DskipTests -Pspark-3.3 >>> ``` >>> >>> Shaoyun Chen 于2023年10月1日周日 16:34写道： >>> >>>> +1 (non-binding) >>>> >>>> I checked the following things: >>>> >>>> - signatures are good. >>>> ``` >>>> gpg --import KEYS >>>> gpg --verify apache-celeborn-0.3.1-incubating-source.tgz.asc >>>> gpg --verify apache-celeborn-0.3.1-incubating-bin.tgz.asc >>>> ``` >>>> - checksums are good. >>>> ``` >>>> sha512sum --check apache-celeborn-0.3.1-incubating-source.tgz.sha512 >>>> sha512sum --check apache-celeborn-0.3.1-incubating-bin.tgz.sha512 >>>> ``` >>>> - build success from source code. >>>> ``` >>>> ./build/make-distribution.sh -Pspark-3.2 >>>> ``` >>>> >>>> Cheng Pan 于2023年9月28日周四 21:55写道： >>>> >>>>> +CC PJ Fanning and Duo Zhang, since you found license issue in >> previous >>>>> RC vote[1] >>>>> >>>>> lists.apache.org >>>>> <https://lists.apache.org/thread/8v4wy5o132rpsjync6465zztgjlf6h5p> >>>>> [image: favicon.ico] >>>>> <https://lists.apache.org/thread/8v4wy5o132rpsjync6465zztgjlf6h5p> >>>>> <https://lists.apache.org/thread/8v4wy5o132rpsjync6465zztgjlf6h5p> >>>>> >>>>> >>>>> On Sep 28, 2023, at 21:52, Cheng Pan wrote: >>>>> >>>>> Hi Celeborn community, >>>>> >>>>> This is a call for a vote to release Apache Celeborn (Incubating) >>>>> 0.3.1-incubating-rc3 >>>>> >>>>> The git tag to be voted upon: >>>>> >>>>> >>> >> https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc3 >>>>> >>>>> The git commit hash: >>>>> 861788810642ef19769b097b82bee70b92e0ace0 >>>>> >>>>> The source and binary artifacts can be found at: >>>>> >>>>> >>> >> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc3 >>>>> >>>>> The staging repo: >>>>> >>> >> https://repository.apache.org/content/repositories/orgapacheceleborn-1040 >>>>> >>>>> Fingerprint of the PGP key release artifacts are signed with: >>>>> 8FC8075E1FDC303276C676EE8001952629BCC75D >>>>> >>>>> My public key to verify signatures can be found in: >>>>> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >>>>> >>>>> The vote will be open for at least 72 hours or until the necessary >>>>> number of votes are reached. >>>>> >>>>> Please vote accordingly: >>>>> >>>>> [ ] +1 approve >>>>> [ ] +0 no opinion >>>>> [ ] -1 disapprove (and the reason) >>>>> >>>>> Checklist for release: >>>>> >>>>> >>> >> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist >>>>> >>>>> Steps to validate the release: >>>>> https://www.apache.org/info/verification.html >>>>> >>>>> Instructions for making binary artifacts from source: >>>>> build/make-distribution.sh --release >>>>> >>>>> Thanks, >>>>> Cheng Pan >>>>> >>>>> >>>>> >>> >>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc3
+CC PJ Fanning and Duo Zhang, since you found license issue in previous RC vote[1] https://lists.apache.org/thread/8v4wy5o132rpsjync6465zztgjlf6h5p > On Sep 28, 2023, at 21:52, Cheng Pan wrote: > > Hi Celeborn community, > > This is a call for a vote to release Apache Celeborn (Incubating) > 0.3.1-incubating-rc3 > > The git tag to be voted upon: > https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc3 > > The git commit hash: > 861788810642ef19769b097b82bee70b92e0ace0 > > The source and binary artifacts can be found at: > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc3 > > The staging repo: > https://repository.apache.org/content/repositories/orgapacheceleborn-1040 > > Fingerprint of the PGP key release artifacts are signed with: > 8FC8075E1FDC303276C676EE8001952629BCC75D > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Checklist for release: > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > Steps to validate the release: > https://www.apache.org/info/verification.html > > Instructions for making binary artifacts from source: > build/make-distribution.sh --release > > Thanks, > Cheng Pan
[VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc3
Hi Celeborn community, This is a call for a vote to release Apache Celeborn (Incubating) 0.3.1-incubating-rc3 The git tag to be voted upon: https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc3 The git commit hash: 861788810642ef19769b097b82bee70b92e0ace0 The source and binary artifacts can be found at: https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc3 The staging repo: https://repository.apache.org/content/repositories/orgapacheceleborn-1040 Fingerprint of the PGP key release artifacts are signed with: 8FC8075E1FDC303276C676EE8001952629BCC75D My public key to verify signatures can be found in: https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS The vote will be open for at least 72 hours or until the necessary number of votes are reached. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason) Checklist for release: https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist Steps to validate the release: https://www.apache.org/info/verification.html Instructions for making binary artifacts from source: build/make-distribution.sh --release Thanks, Cheng Pan
Re: [DISCUSS] Support authentication in Celeborn
Thanks to Chandni for the summary, and all who joined the meeting for the discussion. Thanks, Cheng Pan > On Sep 22, 2023, at 07:36, Chandni Singh wrote: > > Chandni
Re: [RESULT][VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc2
Thanks for heading up, I noticed it too, but considering most user is going to modify the helm chart for production usage, I also think it’s not a blocking issue. Thanks, Cheng Pan > On Sep 15, 2023, at 11:49, Zhongqiang Chen wrote: > > There is still a small issue [CELEBORN-972] on helm chart files at > 0.3.1-incubating-rc2, but this issue will not block this release. > After fixed the issue, I have tested 1T TPCS on flink + celeborn on k8s and > the test results for 99 queries are ok. > > Thanks, > Zhongqiang Chen > > At 2023-09-15 11:21:49, "Cheng Pan" wrote: >> Hello Apache Celeborn PPMC and Community, >> >> The vote closes now as 72hr have passed. The vote PASSES with >> (* = binding) >> +1: >> - Jiashu Xiong * >> - Keyong Zhou * >> - Shaoyun Chen >> - Fu Chen >> - Binjie Yang >> - Ethan Feng * >> - Cheng Pan * >> >> +0: None >> >> -1: None >> >> The vote thread: >> https://lists.apache.org/thread/8bbnpxy6kyn7wxvjwmz2c8vn7bcw67cv >> >> I will now bring the vote to gene...@incubator.apache.org to get >> approval by the IPMC. >> >> If this vote passes too, the release is accepted and will be published. >> >> Thanks to all who helped with the release! >> >> >> Thanks, >> On behalf of Apache Celeborn(Incubating) community >>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc2
Thanks, as 72 hours passed, I’m going to close this vote with my own +1. Thanks, Cheng Pan > On Sep 13, 2023, at 16:45, Ethan Feng wrote: > > +1 (binding) > > [√] incubating in name > [√] signature and hash fine > [√] DISCLAIMER is fine > [√] LICENSE and NOTICE are fine > [√] No unexpected binary files > [√] All source files have ASF headers > [√] Can be built from source > [√] Results are correct on 1TB TPC-DS > > Thanks, > Ethan > > Binjie Yang 于2023年9月13日周三 16:18写道： >> >> +1 >> >> Checked: >> - build binary package with ./build/make-distribution.sh -Pspark-3.3 >> - check git commit in RELEASE file >> - Build Docker Image with docker build -f docker/Dockerfile . >> >> Thanks, >> Binjie Yang >> >> On 2023/09/12 07:12:44 Fu Chen wrote: >>> +1 >>> >>> I checked >>> - download links are valid. >>> - git commit hash is correct. >>> - no binary files in the source release. >>> - signatures are good. >>> ``` >>> gpg --import KEYS >>> gpg --verify apache-celeborn-0.3.1-incubating-source.tgz.asc >>> gpg --verify apache-celeborn-0.3.1-incubating-bin.tgz.asc >>> ``` >>> - checksums are good. >>> ``` >>> sha512sum --check apache-celeborn-0.3.1-incubating-source.tgz.sha512 >>> sha512sum --check apache-celeborn-0.3.1-incubating-bin.tgz.sha512 >>> ``` >>> - build success from source code (ubuntu 18.04). >>> ``` >>> ./build/mvn clean package -DskipTests -Pspark-3.3 >>> ``` >>> >>> >>> Shaoyun Chen 于2023年9月12日周二 11:01写道： >>> >>>> +1 (non-binding) >>>> >>>> I checked the following things: >>>> >>>> - signatures are good. >>>> ``` >>>> gpg --import KEYS >>>> gpg --verify apache-celeborn-0.3.1-incubating-source.tgz.asc >>>> gpg --verify apache-celeborn-0.3.1-incubating-bin.tgz.asc >>>> ``` >>>> - checksums are good. >>>> ``` >>>> sha512sum --check apache-celeborn-0.3.1-incubating-source.tgz.sha512 >>>> sha512sum --check apache-celeborn-0.3.1-incubating-bin.tgz.sha512 >>>> ``` >>>> - build success from source code. >>>> ``` >>>> ./build/make-distribution.sh -Pspark-3.2 >>>> ``` >>>> >>>> Keyong Zhou 于2023年9月12日周二 09:57写道： >>>>> >>>>> +1 (binding) >>>>> >>>>> I checked >>>>> - git commit hash is correct. >>>>> - links are valid. >>>>> - "incubating" is in the name. >>>>> - PGP keys are good. >>>>> - hashes are correct. >>>>> - LICENSE looks good. >>>>> - NOTICE looks good. >>>>> - DISCLAIMER exists. >>>>> - build success from source code (macOS). ``` >>>> ./build/make-distribution.sh >>>>> --release ``` >>>>> >>>>> Thanks, >>>>> Keyong Zhou >>>>> >>>>> rexxiong 于2023年9月11日周一 22:08写道： >>>>> >>>>>> +1 (binding) >>>>>> I checked >>>>>> - Download links are valid. >>>>>> - git commit hash is correct >>>>>> - Checksums and signatures are valid. >>>>>> - No binary files in the source release >>>>>> - Files have the word incubating in their name. >>>>>> - DISCLAIMER,LICENSE and NOTICE files exist. >>>>>> - Successfully built the binary from the source on MacOs with Command: >>>>>> ./build/make-distribution.sh --release >>>>>> >>>>>> Thanks, >>>>>> Jiashu Xiong >>>>>> >>>>>> Cheng Pan 于2023年9月11日周一 16:27写道： >>>>>> >>>>>>> Hi Celeborn community, >>>>>>> >>>>>>> This is a call for a vote to release Apache Celeborn (Incubating) >>>>>>> 0.3.1-incubating-rc2 >>>>>>> >>>>>>> The git tag to be voted upon: >>>>>>> >>>>>>> >>>>>> >>>> https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc2 >>>>>>> >>>>>>> The git commit hash: >>>>>>> 7ec5596748af49ef9cb429d08550e89d94d5cc74 >>>>>>> >>>>>>> The source and binary artifacts can be found at: >>>>>>> >>>>>>> >>>>>> >>>> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc2 >>>>>>> >>>>>>> The staging repo: >>>>>>> >>>>>> >>>> https://repository.apache.org/content/repositories/orgapacheceleborn-1039 >>>>>>> >>>>>>> Fingerprint of the PGP key release artifacts are signed with: >>>>>>> 8FC8075E1FDC303276C676EE8001952629BCC75D >>>>>>> >>>>>>> My public key to verify signatures can be found in: >>>>>>> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >>>>>>> >>>>>>> The vote will be open for at least 72 hours or until the necessary >>>>>>> number of votes are reached. >>>>>>> >>>>>>> Please vote accordingly: >>>>>>> >>>>>>> [ ] +1 approve >>>>>>> [ ] +0 no opinion >>>>>>> [ ] -1 disapprove (and the reason) >>>>>>> >>>>>>> Checklist for release: >>>>>>> >>>>>>> >>>>>> >>>> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist >>>>>>> >>>>>>> Steps to validate the release: >>>>>>> https://www.apache.org/info/verification.html >>>>>>> >>>>>>> Instructions for making binary artifacts from source: >>>>>>> build/make-distribution.sh --release >>>>>>> >>>>>>> Thanks, >>>>>>> Cheng Pan >>>>>> >>>> >>>
[RESULT][VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc2
Hello Apache Celeborn PPMC and Community, The vote closes now as 72hr have passed. The vote PASSES with (* = binding) +1: - Jiashu Xiong * - Keyong Zhou * - Shaoyun Chen - Fu Chen - Binjie Yang - Ethan Feng * - Cheng Pan * +0: None -1: None The vote thread: https://lists.apache.org/thread/8bbnpxy6kyn7wxvjwmz2c8vn7bcw67cv I will now bring the vote to gene...@incubator.apache.org to get approval by the IPMC. If this vote passes too, the release is accepted and will be published. Thanks to all who helped with the release! Thanks, On behalf of Apache Celeborn(Incubating) community
[VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc2
Hi Celeborn community, This is a call for a vote to release Apache Celeborn (Incubating) 0.3.1-incubating-rc2 The git tag to be voted upon: https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc2 The git commit hash: 7ec5596748af49ef9cb429d08550e89d94d5cc74 The source and binary artifacts can be found at: https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc2 The staging repo: https://repository.apache.org/content/repositories/orgapacheceleborn-1039 Fingerprint of the PGP key release artifacts are signed with: 8FC8075E1FDC303276C676EE8001952629BCC75D My public key to verify signatures can be found in: https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS The vote will be open for at least 72 hours or until the necessary number of votes are reached. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason) Checklist for release: https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist Steps to validate the release: https://www.apache.org/info/verification.html Instructions for making binary artifacts from source: build/make-distribution.sh --release Thanks, Cheng Pan
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc1
Cancel this vote because of the correctness issue, will prepare the next RC once it gets resolved. Thanks, Cheng Pan > On Sep 6, 2023, at 15:52, Cheng Pan wrote: > > Hi Celeborn community, > > This is a call for a vote to release Apache Celeborn (Incubating) > 0.3.1-incubating-rc1 > > The git tag to be voted upon: > https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc1 > > The git commit hash: > 6814a634f2c21a95d3199c4c7d0bca8f1be55cc2 source and binary artifacts can be > found at: > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc1 > > The staging repo: > https://repository.apache.org/content/repositories/orgapacheceleborn-1038 > > Fingerprint of the PGP key release artifacts are signed with: > 8FC8075E1FDC303276C676EE8001952629BCC75D > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Checklist for release: > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > Steps to validate the release: > https://www.apache.org/info/verification.html > > > Instructions for making binary artifacts from source: > build/make-distribution.sh --release > > Thanks, > Cheng Pan > >
[VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc1
Hi Celeborn community, This is a call for a vote to release Apache Celeborn (Incubating) 0.3.1-incubating-rc1 The git tag to be voted upon: https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc1 The git commit hash: 6814a634f2c21a95d3199c4c7d0bca8f1be55cc2 source and binary artifacts can be found at: https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc1 The staging repo: https://repository.apache.org/content/repositories/orgapacheceleborn-1038 Fingerprint of the PGP key release artifacts are signed with: 8FC8075E1FDC303276C676EE8001952629BCC75D My public key to verify signatures can be found in: https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS The vote will be open for at least 72 hours or until the necessary number of votes are reached. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason) Checklist for release: https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist Steps to validate the release: https://www.apache.org/info/verification.html Instructions for making binary artifacts from source: build/make-distribution.sh --release Thanks, Cheng Pan
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc0
This vote failed because there is one binding -1, will prepare the next RC in a few days. Thanks to all who participated in the voting to help with this release. Thanks, Cheng Pan > On Sep 1, 2023, at 15:38, Cheng Pan wrote: > > Ethan, could you please create the corresponding JIRA ticket(s) and set the > priority to blocker? > > Thanks, > Cheng Pan > > >> On Sep 1, 2023, at 15:33, Ethan Feng wrote: >> >> -1(bingding) >> The deploy doc in this RC needs to be updated. >> >> Regards, >> Ethan >> >> 在 2023年9月1日星期五，Mridul Muralidharan 写道： >> >>> +1 >>> >>> Signatures, digests, license, etc check out fine. >>> Checked out tag and build/tested with -Pspark-3.1 >>> >>> Regards, >>> Mridul >>> >>> >>> On Thu, Aug 31, 2023 at 11:35 AM Cheng Pan wrote: >>> >>>> Hi Celeborn community, >>>> >>>> This is a call for a vote to release Apache Celeborn (Incubating) >>>> 0.3.1-incubating-rc0 >>>> >>>> The git tag to be voted upon: >>>> >>>> https://github.com/apache/incubator-celeborn/releases/tag/ >>> v0.3.1-incubating-rc0 >>>> >>>> The git commit hash: >>>> b3992274e207959125d8784d9b61a6e8043612fc source and binary artifacts >>> can be >>>> found at: >>>> >>>> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0 >>> .3.1-incubating-rc0 >>>> >>>> The staging repo: >>>> https://repository.apache.org/content/repositories/orgapache >>> celeborn-1037 >>>> >>>> Fingerprint of the PGP key release artifacts are signed with: >>>> 8FC8075E1FDC303276C676EE8001952629BCC75D >>>> >>>> My public key to verify signatures can be found in: >>>> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >>>> >>>> The vote will be open for at least 72 hours or until the necessary >>>> number of votes are reached. >>>> >>>> Please vote accordingly: >>>> >>>> [ ] +1 approve >>>> [ ] +0 no opinion >>>> [ ] -1 disapprove (and the reason) >>>> >>>> Checklist for release: >>>> >>>> https://cwiki.apache.org/confluence/display/INCUBATOR/Incuba >>> tor+Release+Checklist >>>> Steps to validate the release: >>>> https://www.apache.org/info/verification.html >>>> >>>> * Download links, checksums and PGP signatures are valid. >>>> * Source code distributions have correct names matching the current >>>> release. >>>> * Release files have the word incubating in their name. >>>> * DISCLAIMER, LICENSE and NOTICE files are correct. >>>> * All files have license headers if necessary. >>>> * No unlicensed compiled archives bundled in source archive. >>>> * The source tarball matches the git tag. >>>> * Build from source is successful. >>>> >>>> Thanks, >>>> Cheng Pan >>>> >>> >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc0
Ethan, could you please create the corresponding JIRA ticket(s) and set the priority to blocker? Thanks, Cheng Pan > On Sep 1, 2023, at 15:33, Ethan Feng wrote: > > -1(bingding) > The deploy doc in this RC needs to be updated. > > Regards, > Ethan > > 在 2023年9月1日星期五，Mridul Muralidharan 写道： > >> +1 >> >> Signatures, digests, license, etc check out fine. >> Checked out tag and build/tested with -Pspark-3.1 >> >> Regards, >> Mridul >> >> >> On Thu, Aug 31, 2023 at 11:35 AM Cheng Pan wrote: >> >>> Hi Celeborn community, >>> >>> This is a call for a vote to release Apache Celeborn (Incubating) >>> 0.3.1-incubating-rc0 >>> >>> The git tag to be voted upon: >>> >>> https://github.com/apache/incubator-celeborn/releases/tag/ >> v0.3.1-incubating-rc0 >>> >>> The git commit hash: >>> b3992274e207959125d8784d9b61a6e8043612fc source and binary artifacts >> can be >>> found at: >>> >>> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0 >> .3.1-incubating-rc0 >>> >>> The staging repo: >>> https://repository.apache.org/content/repositories/orgapache >> celeborn-1037 >>> >>> Fingerprint of the PGP key release artifacts are signed with: >>> 8FC8075E1FDC303276C676EE8001952629BCC75D >>> >>> My public key to verify signatures can be found in: >>> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >>> >>> The vote will be open for at least 72 hours or until the necessary >>> number of votes are reached. >>> >>> Please vote accordingly: >>> >>> [ ] +1 approve >>> [ ] +0 no opinion >>> [ ] -1 disapprove (and the reason) >>> >>> Checklist for release: >>> >>> https://cwiki.apache.org/confluence/display/INCUBATOR/Incuba >> tor+Release+Checklist >>> Steps to validate the release: >>> https://www.apache.org/info/verification.html >>> >>> * Download links, checksums and PGP signatures are valid. >>> * Source code distributions have correct names matching the current >>> release. >>> * Release files have the word incubating in their name. >>> * DISCLAIMER, LICENSE and NOTICE files are correct. >>> * All files have license headers if necessary. >>> * No unlicensed compiled archives bundled in source archive. >>> * The source tarball matches the git tag. >>> * Build from source is successful. >>> >>> Thanks, >>> Cheng Pan >>> >>
[VOTE] Release Apache Celeborn(Incubating) 0.3.1-incubating-rc0
Hi Celeborn community, This is a call for a vote to release Apache Celeborn (Incubating) 0.3.1-incubating-rc0 The git tag to be voted upon: https://github.com/apache/incubator-celeborn/releases/tag/v0.3.1-incubating-rc0 The git commit hash: b3992274e207959125d8784d9b61a6e8043612fc source and binary artifacts can be found at: https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.1-incubating-rc0 The staging repo: https://repository.apache.org/content/repositories/orgapacheceleborn-1037 Fingerprint of the PGP key release artifacts are signed with: 8FC8075E1FDC303276C676EE8001952629BCC75D My public key to verify signatures can be found in: https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS The vote will be open for at least 72 hours or until the necessary number of votes are reached. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason) Checklist for release: https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist Steps to validate the release: https://www.apache.org/info/verification.html * Download links, checksums and PGP signatures are valid. * Source code distributions have correct names matching the current release. * Release files have the word incubating in their name. * DISCLAIMER, LICENSE and NOTICE files are correct. * All files have license headers if necessary. * No unlicensed compiled archives bundled in source archive. * The source tarball matches the git tag. * Build from source is successful. Thanks, Cheng Pan
Re: [DISCUSS] Time for 0.3.1
Thanks, I plan to cut the 0.3.1-rc0 on Fri Aug 30. Please let me know if you have any PRs that want to be shipped. Thanks, Cheng Pan On Mon, Aug 28, 2023 at 2:08 PM Keyong Zhou wrote: > > Thanks Pan for volunteering! I also think it's time to release 0.3.1. > > Thanks, > Keyong Zhou > > Binjie Yang 于2023年8月28日周一 10:53写道： > > > +1, thanks for driving this. > > > > Thanks, > > Binjie Yang > > > > On 2023/08/28 02:48:02 Cheng Pan wrote: > > > Hi, Celeborn community, > > > > > > It has been a while since the 0.3.0 release, and there are some critical > > fixes land branch-0.3, I think it’s time to prepare for releasing 0.3.1. > > > > > > WDYT? And I’m volunteering to be the release manager if no one has > > applied. > > > > > > Thanks, > > > Cheng Pan > > > > > > > > > > >
[DISCUSS] Time for 0.3.1
Hi, Celeborn community, It has been a while since the 0.3.0 release, and there are some critical fixes land branch-0.3, I think it’s time to prepare for releasing 0.3.1. WDYT? And I’m volunteering to be the release manager if no one has applied. Thanks, Cheng Pan
[ANNOUNCEMENT] New Committer: Fu Chen
Hi Celeborn Community, The Podling Project Management Committee (PPMC) for Apache Celeborn has invited Fu Chen to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. A PMC member helps manage and guide the direction of the project. Please join me in congratulating Fu Chen! Yours, Cheng Pan, on behalf of the Apache Celeborn PPMC
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.0-incubating-rc2
+1 (binding) I checked - signature and checksum of source artifacts are good - DISCLAIMER exists - artifacts' name contains "incubating" - no unexpected binary files inside source artifacts - building succeeded from source on Ubuntu 20.04 with OpenJDK 8 command: build/mvn clean package -DskipTests -Pspark-3.3 Additionally, I tested it with 1T TeraSort[1], and everything looks fine. spark-submit \ --class io.github.pan3793.spark.terasort.TeraGen \ --packages io.github.pan3793:spark-terasort_2.12:2.0.0 \ spark-internal \ 1t hdfs:///tmp/terasort_1t_in spark-submit \ --class io.github.pan3793.spark.terasort.TeraSort \ --packages io.github.pan3793:spark-terasort_2.12:2.0.0 \ spark-internal \ hdfs:///tmp/terasort_1t_in hdfs:///tmp/terasort_1t_out spark-submit \ --class io.github.pan3793.spark.terasort.TeraValidate \ --packages io.github.pan3793:spark-terasort_2.12:2.0.0 \ spark-internal \ hdfs:///tmp/terasort_1t_out [1] https://github.com/pan3793/spark-terasort Thanks, Cheng Pan > On Jul 19, 2023, at 15:10, Keyong Zhou wrote: > > +1 (binding) I checked - git commit hash is correct. - links are valid. - > "incubating" is in the name. - PGP keys are good. - hashes are correct. - > LICENSE looks good. - NOTICE looks good. - DISCLAIMER exists. - build > success from source code (macOS). ``` ./build/make-distribution.sh > --release ``` Thanks, Keyong Zhou > > Zhongqiang Chen 于2023年7月18日周二 18:34写道： > >> Hi Celeborn community, >> >> This is a call for a vote to release Apache Celeborn (Incubating) >> 0.3.0-incubating-rc2 >> >> The git tag to be voted upon: >> >> https://github.com/apache/incubator-celeborn/releases/tag/v0.3.0-incubating-rc2 >> >> The git commit hash: >> 6c5e3f8e7021f409b44e6352142d13e7ac3ffe93The source and binary artifacts >> can be found at: >> >> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.0-incubating-rc2 >> >> The staging repo: >> https://repository.apache.org/content/repositories/orgapacheceleborn-1036 >> >> Fingerprint of the PGP key release artifacts are signed with: >> 4A99BC4356A17B2FCFDBF7EE220A9E2898A6A6D4 >> >> My public key to verify signatures can be found in: >> https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >> >> The vote will be open for at least 72 hours or until the necessary >> number of votes are reached. >> >> Please vote accordingly: >> >> [ ] +1 approve >> [ ] +0 no opinion >> [ ] -1 disapprove (and the reason) >> >> Checklist for release: >> >> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist >> Steps to validate the release: >> https://www.apache.org/info/verification.html >> >> * Download links, checksums and PGP signatures are valid. >> * Source code distributions have correct names matching the current >> release. >> * Release files have the word incubating in their name. >> * DISCLAIMER, LICENSE and NOTICE files are correct. >> * All files have license headers if necessary. >> * No unlicensed compiled archives bundled in source archive. >> * The source tarball matches the git tag. >> * Build from source is successful. >> >> Thanks, >> Zhongqiang Chen >> >>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.0-incubating-rc1
+1 (binding) I checked - signature and checksum of source artifacts are good - DISCLAIMER exists - artifacts' name contains "incubating" - no unexpected binary files inside source artifacts - building succeeded from source on Ubuntu 20.04 with OpenJDK 8 command: build/mvn clean package -DskipTests -Pspark-3.3 Additionally, I tested it with 1T TeraSort[1], and everything looks fine. spark-submit \ --class io.github.pan3793.spark.terasort.TeraGen \ --packages io.github.pan3793:spark-terasort_2.12:2.0.0 \ spark-internal \ 1t hdfs:///tmp/terasort_1t_in spark-submit \ --class io.github.pan3793.spark.terasort.TeraSort \ --packages io.github.pan3793:spark-terasort_2.12:2.0.0 \ spark-internal \ hdfs:///tmp/terasort_1t_in hdfs:///tmp/terasort_1t_out spark-submit \ --class io.github.pan3793.spark.terasort.TeraValidate \ --packages io.github.pan3793:spark-terasort_2.12:2.0.0 \ spark-internal \ hdfs:///tmp/terasort_1t_out [1] https://github.com/pan3793/spark-terasort Thanks, Cheng Pan > On Jul 15, 2023, at 07:59, Zhongqiang Chen wrote: > > Hi Celeborn community, > > This is a call for a vote to release Apache Celeborn (Incubating) > 0.3.0-incubating-rc1 > > The git tag to be voted upon: > https://github.com/apache/incubator-celeborn/releases/tag/v0.3.0-incubating-rc1 > > The git commit hash: > 6967a8c2fd61c82cad85019b1d4af95930d083caThe source and binary artifacts can > be found at: > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.0-incubating-rc1 > > The staging repo: > https://repository.apache.org/content/repositories/orgapacheceleborn-1031/ > > Fingerprint of the PGP key release artifacts are signed with: > 4A99BC4356A17B2FCFDBF7EE220A9E2898A6A6D4 > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Checklist for release: > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > Steps to validate the release: > https://www.apache.org/info/verification.html > > * Download links, checksums and PGP signatures are valid. > * Source code distributions have correct names matching the current release. > * Release files have the word incubating in their name. > * DISCLAIMER, LICENSE and NOTICE files are correct. > * All files have license headers if necessary. > * No unlicensed compiled archives bundled in source archive. > * The source tarball matches the git tag. > * Build from source is successful. > > Thanks, > Zhongqiang Chen >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.0-incubating-rc0
-1 (binding) Sorry, two critical issues were found after RC0 tagged: CELEBORN-776[1] recovers compatibility between 0.2 client and 0.3 server, without this patch, the user can not roll upgrade from 0.2 to 0.3; CELEBORN-614[2] fix the worker trim issue, the worker is highly hung without this fix. Except to the issues, I checked the following things are good: - signature and checksum of source artifacts are good - DISCLAIMER exists - artifacts' name contains "incubating" - no unexpected binary files inside source artifacts - building succeeded from source on Ubuntu 22.04 with OpenJDK 8 command: build/mvn clean package -DskipTests -Pspark-3.3 [1] CELEBORN-776 https://github.com/apache/incubator-celeborn/pull/1686 [2] CELEBORN-614 https://github.com/apache/incubator-celeborn/pull/1689 Thanks, Cheng Pan On Thu, Jul 6, 2023 at 4:02 PM Zhongqiang Chen wrote: > > Hi Celeborn community, > > This is a call for a vote to release Apache Celeborn (Incubating) > 0.3.0-incubating-rc0 > > The git tag to be voted upon: > https://github.com/apache/incubator-celeborn/releases/tag/v0.3.0-incubating-rc0 > > The git commit hash: > 1b11809451ec0d82762c5fdb89037956d2410632 > The source and binary artifacts can be found at: > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.3.0-incubating-rc0 > > The staging repo: > https://repository.apache.org/content/repositories/orgapacheceleborn-1027/ > > Fingerprint of the PGP key release artifacts are signed with: > 4A99BC4356A17B2FCFDBF7EE220A9E2898A6A6D4 > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The vote will be open for at least 72 hours or until the necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Checklist for release: > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > Steps to validate the release: > https://www.apache.org/info/verification.html > > * Download links, checksums and PGP signatures are valid. > * Source code distributions have correct names matching the current release. > * Release files have the word incubating in their name. > * DISCLAIMER, LICENSE and NOTICE files are correct. > * All files have license headers if necessary. > * No unlicensed compiled archives bundled in source archive. > * The source tarball matches the git tag. > * Build from source is successful. > > Thanks, > Zhongqiang Chen >
Re: [DISCUSSION] Release Apache Celeborn(Incubating) 0.3.0-incubating-rc0
Thanks Zhongqiang for driving this release. I think it’s good to start the first RC next week, in the meanwhile, we can prepare the release note first, you may want to create a Google Docs/GitHub discussion or other online docs to make it easy for review and collaboration. Thanks, Cheng Pan > On Jun 30, 2023, at 12:28, Zhongqiang Chen wrote: > > > > > Hi Celeborn community, > > > > > I am pleased to announce that we are planning to release version > 0.3.0-incubating-rc0 of Apache Celeborn (Incubating)! > > Since version 0.2.1, 28+ contributors have submitted 440+ commits. The most > essential features are firstly, unified shuffle service supports both spark > and flink, and secondly, significant upgrades in stability and performance > > > > > Before we proceed with the release, I’d like to start a discussion to ensure > that everything is in order and to address any outstanding issues. > > Please take a moment to review the current state of the project and share any > concerns or questions you may have. Here are a few items that we would like > to discuss: > > > > > 1. Are there any outstanding bugs that need to be addressed before the > release? > > > > > 2. Are there any compatibility issues that we should be aware of? > > now we have prepared a migration doc from 0.2.1 to 0.3.0 > https://github.com/apache/incubator-celeborn/blob/main/docs/migration.md > > > > > 3. Are there any new features that should be included in this release? > > > > > If you have any thoughts on these topics, or anything else related to the > release, please reply to this email with your feedback. Your input is greatly > appreciated! > > We plan to freeze code next week and then release version 0.3.0 of Apache > Celeborn (Incubating)!, so please provide your feedback as soon as possible. > Thank you for your contributions to this project, and I look forward to > hearing from you soon. > > > > > Best regards, > > Zhongqiang Chen
Re: [DISCUSS] Allow external contributors to run CI without approval
+1 for "only requires approval first time" Keyong Zhou 于 2023年6月16日周五 下午5:48写道： > +1 > > Thanks, > Keyong Zhou > > Ethan Feng 于2023年6月16日周五 16:27写道： > > > Recent moves by Apache Infra have changed the policy on GitHub Actions > from > > "Only requires approval first time" to "Requires approval every time". > > > > I think this is not friendly for getting folks involved in > > the project and this increased the cost for committers to process the > > pull requests. > > > > Please respond to this thread if you are in support of going back to > > "Only requires approval the first time" or if you don't believe this is a > > good idea please respond as well. > > > > Thanks, > > Ethan Feng > > >
Committers: Please use `dev/merge_pr.py` to merge new PRs
Hi Celeborn Committers, A PR merge tool `dev/merge_pr.py` was added to the Celeborn git repo[1][2], it aims to simplify the PR merge and backport process, and improve the git commit history. AFAIK, this tool is originally from Apache Parquet, then, it was borrowed and modified by Apache Spark, Apache Kyuubi. Compare with the current way we are using, use GitHub squash to merge PR, this tool has the following pros: - squash all commits into one, this is the same as GitHub squash, but also fix the PR title in canonicalize format: [CELEBORN-][LABEL] Title of the pull request - it’s an interactive script, the committer just needs to follow the prompt and react with y/n, then can complete the PR backport, JIRA assignment, JIRA fixed version update - reserve the author and committer information in the git history, the GitHub squash always marks the committer as GitHub , which is really bad. - reserve the PR description as the git commit message body, again, please fill in the PR description seriously :) Some setup procedures are required to use this merge tool: 1. invoking it at the celeborn git project root dir 2. naming the upstream repo as “apache”, you can check it by `git remote -v`, the desired output should be like apache-celeborn git:(main) git remote -v apache g...@github.com:apache/incubator-celeborn.git (fetch) apache g...@github.com:apache/incubator-celeborn.git (push) 3. export environment variables ASF_USERNAME ASF_PASSWORD which is used for JIRA authentication 4. export environment variable GITHUB_OAUTH_KEY which is used for GitHub authentication. You can create an OAuth key at https://github.com/settings/tokens 5. install python JIRA dependencies by `pip install jira` or `pip install -r requirements.txt` As an example, [3] is merged by this tool. Please let me know if you have any questions or concerns w/ this tool. [1] https://issues.apache.org/jira/browse/CELEBORN-623 [2] https://github.com/apache/incubator-celeborn/pull/1539 [3] https://github.com/apache/incubator-celeborn/commit/67762783d0d51acb147f35e9a59e2bfec48ec04b Thanks, Cheng Pan
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.1-incubating-rc0
Sorry, I missed the vote result mail, please ignore my last message Thanks, Cheng Pan On Mar 23, 2023 at 21:47:40, Cheng Pan wrote: > It’s time to close the voting and move forward > > Thanks, > Cheng Pan > > > On Mar 20, 2023 at 18:03:42, kerwin zhang > wrote: > >> +1 (binding) >> >> I checked >> >> - PGP signatures are valid. >> wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >> gpg --import KEYS >> gpg --verify apache-celeborn-0.2.1-incubating-source.tgz.asc >> gpg --verify apache-celeborn-0.2.1-incubating-bin.tgz.asc >> - checksums are correct. >> sha512sum --check apache-celeborn-0.2.1-incubating-bin.tgz.sha512 >> sha512sum --check apache-celeborn-0.2.1-incubating-source.tgz.sha512 >> - LICENSE and NOTICE files exist. >> - DISCLAIMER file exists. >> - release files are in correct location. >> - release files have the word "incubating" in their name. >> - build success from source code (macOS). >> >> ``` >> ./build/make-distribution.sh -Pspark-3.3 >> ``` >> >> Thanks, >> Kerwin Zhang >> >> Keyong Zhou 于2023年3月20日周一 15:46写道： >> >> >> +1 (binding) >> >> >> I checked >> >> >> - git commit hash is correct. >> >> - links are valid. >> >> - "incubating" is in the name. >> >> - PGP keys are good. >> >> - hashes are correct. >> >> - LICENSE looks good. >> >> - NOTICE looks good. >> >> - DISCLAIMER exists. >> >> - build success from source code (macOS). >> >> >> ``` >> >> ./build/make-distribution.sh --release >> >> ``` >> >> >> Thanks, >> >> Keyong Zhou >> >> >> On 2023/03/17 09:17:25 rexxiong wrote: >> >> > Hi Celeborn community, >> >> > >> >> > This is a call for a vote to release Apache Celeborn (Incubating) >> >> > 0.2.1-incubating-rc0 >> >> > >> >> > The git tag to be voted upon: >> >> > >> https://github.com/apache/incubator-celeborn/releases/tag/v0.2.1-incubating-rc0 >> >> > >> >> > The git commit hash: >> >> > 93898d0020899ba7e98ededaad14a6043cec07c9 >> >> > >> >> > The source and binary artifacts can be found at: >> >> > >> https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.1-incubating-rc0 >> >> > >> >> > The staging repo: >> >> > >> https://repository.apache.org/content/repositories/orgapacheceleborn-1010 >> >> > >> >> > Fingerprint of the PGP key release artifacts are signed with: >> >> > B4AF9302A52006F3711C784388BDDBB8C6724EC9 >> >> > >> >> > My public key to verify signatures can be found in: >> >> > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS >> >> > >> >> > The vote will be open for at least 72 hours or until the necessary >> >> > number of votes are reached. >> >> > >> >> > Please vote accordingly: >> >> > >> >> > [ ] +1 approve >> >> > [ ] +0 no opinion >> >> > [ ] -1 disapprove (and the reason) >> >> > >> >> > Checklist for release: >> >> > >> https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist >> >> > Steps to validate the release: >> >> > https://www.apache.org/info/verification.html >> >> > >> >> > * Download links, checksums and PGP signatures are valid. >> >> > * Source code distributions have correct names matching the current >> release. >> >> > * Release files have the word incubating in their name. >> >> > * DISCLAIMER, LICENSE and NOTICE files are correct. >> >> > * All files have license headers if necessary. >> >> > * No unlicensed compiled archives bundled in source archive. >> >> > * The source tarball matches the git tag. >> >> > * Build from source is successful. >> >> > >> >> > Thanks, >> >> > rexxiong >> >> > >> >>
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.1-incubating-rc0
It’s time to close the voting and move forward Thanks, Cheng Pan On Mar 20, 2023 at 18:03:42, kerwin zhang wrote: > +1 (binding) > > I checked > > - PGP signatures are valid. > wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > gpg --import KEYS > gpg --verify apache-celeborn-0.2.1-incubating-source.tgz.asc > gpg --verify apache-celeborn-0.2.1-incubating-bin.tgz.asc > - checksums are correct. > sha512sum --check apache-celeborn-0.2.1-incubating-bin.tgz.sha512 > sha512sum --check apache-celeborn-0.2.1-incubating-source.tgz.sha512 > - LICENSE and NOTICE files exist. > - DISCLAIMER file exists. > - release files are in correct location. > - release files have the word "incubating" in their name. > - build success from source code (macOS). > > ``` > ./build/make-distribution.sh -Pspark-3.3 > ``` > > Thanks, > Kerwin Zhang > > Keyong Zhou 于2023年3月20日周一 15:46写道： > > > +1 (binding) > > > I checked > > > - git commit hash is correct. > > - links are valid. > > - "incubating" is in the name. > > - PGP keys are good. > > - hashes are correct. > > - LICENSE looks good. > > - NOTICE looks good. > > - DISCLAIMER exists. > > - build success from source code (macOS). > > > ``` > > ./build/make-distribution.sh --release > > ``` > > > Thanks, > > Keyong Zhou > > > On 2023/03/17 09:17:25 rexxiong wrote: > > > Hi Celeborn community, > > > > > > This is a call for a vote to release Apache Celeborn (Incubating) > > > 0.2.1-incubating-rc0 > > > > > > The git tag to be voted upon: > > > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.1-incubating-rc0 > > > > > > The git commit hash: > > > 93898d0020899ba7e98ededaad14a6043cec07c9 > > > > > > The source and binary artifacts can be found at: > > > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.1-incubating-rc0 > > > > > > The staging repo: > > > > https://repository.apache.org/content/repositories/orgapacheceleborn-1010 > > > > > > Fingerprint of the PGP key release artifacts are signed with: > > > B4AF9302A52006F3711C784388BDDBB8C6724EC9 > > > > > > My public key to verify signatures can be found in: > > > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > > > > The vote will be open for at least 72 hours or until the necessary > > > number of votes are reached. > > > > > > Please vote accordingly: > > > > > > [ ] +1 approve > > > [ ] +0 no opinion > > > [ ] -1 disapprove (and the reason) > > > > > > Checklist for release: > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > > Steps to validate the release: > > > https://www.apache.org/info/verification.html > > > > > > * Download links, checksums and PGP signatures are valid. > > > * Source code distributions have correct names matching the current > release. > > > * Release files have the word incubating in their name. > > > * DISCLAIMER, LICENSE and NOTICE files are correct. > > > * All files have license headers if necessary. > > > * No unlicensed compiled archives bundled in source archive. > > > * The source tarball matches the git tag. > > > * Build from source is successful. > > > > > > Thanks, > > > rexxiong > > > > >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.1-incubating-rc0
+1 (binding) I have checked the followings: - signatures and checksums are valid. - release artifacts include incubating in their name. - DISCLAIMER exists in release artifacts. - LICENSE and NOTICE look good to me. - build from source succeeded. FYI @Zhongqiang Chen, - During incubating phase, the first round release voting in dev@celeborn channel, PPMC members' (including Mentors) vote is binding, Committer voting is not binding; and the second round voting in general@incubator channel, only IPMC members’ vote is binding. - Better to use your apache mail account for voting. Thanks, Cheng Pan On Mar 19, 2023 at 13:19:44, Ethan Feng wrote: > +1 (binding) > > I checked the following items. > > [√] incubating in name > [√] signature and hash fine > [√] DISCLAIMER is fine > [√] LICENSE and NOTICE are fine > [√] No unexpected binary files > [√] All source files have ASF headers > [√] Can be built from source(profile spark-3.3) > [√] Maven artifacts are fine > > > Thanks, > Ethan Feng > > 陈中强 于2023年3月19日周日 10:12写道： > > > +1 (binding) > > > > > > > > > I have checked the followings: > > > > > > 1. PGP signatures are valid. > > > wget > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > gpg --import KEYS > > > gpg --verify apache-celeborn-0.2.1-incubating-bin.tgz.asc > > > gpg --verify apache-celeborn-0.2.1-incubating-source.tgz.asc > > > > > > 2. checksums are correct. > > > sha512sum --check apache-celeborn-0.2.1-incubating-bin.tgz.sha512 > > > sha512sum --check apache-celeborn-0.2.1-incubating-source.tgz.sha512 > > > > > > 3. Source code distributions have correct names matching the current > release. > > > > > > 4. Release files have the word incubating in their name. > > > > > > 5. DISCLAIMER, LICENSE and NOTICE files are correct. > > > > > > 6. All files have license headers if necessary. > > > > > > 7. No unlicensed compiled archives bundled in source archive. > > > > > > 8. The source tarball matches the git tag. > > > > > > 9. Build from source is successful. > > > > > > > > > > > > > > > > > > > At 2023-03-17 17:17:25, "rexxiong" wrote: > > >Hi Celeborn community, > > > > > >This is a call for a vote to release Apache Celeborn (Incubating) > > >0.2.1-incubating-rc0 > > > > > >The git tag to be voted upon: > > > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.1-incubating-rc0 > > > > > >The git commit hash: > > >93898d0020899ba7e98ededaad14a6043cec07c9 > > > > > >The source and binary artifacts can be found at: > > > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.1-incubating-rc0 > > > > > >The staging repo: > > >https://repository.apache.org/content/repositories/orgapacheceleborn-1010 > > > > > >Fingerprint of the PGP key release artifacts are signed with: > > >B4AF9302A52006F3711C784388BDDBB8C6724EC9 > > > > > >My public key to verify signatures can be found in: > > >https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > > > >The vote will be open for at least 72 hours or until the necessary > > >number of votes are reached. > > > > > >Please vote accordingly: > > > > > >[ ] +1 approve > > >[ ] +0 no opinion > > >[ ] -1 disapprove (and the reason) > > > > > >Checklist for release: > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > >Steps to validate the release: > > >https://www.apache.org/info/verification.html > > > > > >* Download links, checksums and PGP signatures are valid. > > >* Source code distributions have correct names matching the current > release. > > >* Release files have the word incubating in their name. > > >* DISCLAIMER, LICENSE and NOTICE files are correct. > > >* All files have license headers if necessary. > > >* No unlicensed compiled archives bundled in source archive. > > >* The source tarball matches the git tag. > > >* Build from source is successful. > > > > > >Thanks, > > >rexxiong > >
[ANNOUNCE] New committer: Binjie Yang
Hi Celeborn community, I'm very excited to announce that recently we added Binjie Yang[1] as our new committer! Binjie has kept contributing to Celeborn since Jul 2022, mainly on the integration w/ Kubernetes. Looking forward that Binjie will continue contributing to the project, pushing Celeborn to the next level together with all contributors of the community! Also, we are looking forward to add more and more committers to our project [1] https://github.com/zwangsheng Thanks, Cheng Pan
Re: [ANNOUNCE] Apache Celeborn(incubating) 0.2.0 available
Congrats, and thank Ethan and everyone who helps with the first release after Celeborn entered the Apache Incubator, one minor thing is we should create a new tag[2] on the same commit of the passed vote RC version[1] instead of just using it. [1] https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc5 [2] https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating Thanks, Cheng Pan On Mar 1, 2023 at 23:38:45, Ethan Feng wrote: > Hi all, > > Apache Celeborn(Incubating) community is glad to announce the > new release of Apache Celeborn(Incubating) 0.2.0 > > Celeborn is dedicated to improving the efficiency and elasticity of > different map-reduce engines and provides an elastic, high-efficient > service for intermediate data including shuffle data, spilled data, > result data, etc. > Currently Celeborn supports Spark full-featured and improves Spark > job's performance, stability and elasticity. > > Download Link: https://celeborn.apache.org/download/ > > GitHub Release Tag: > - > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc5 > > Release Notes: > - https://celeborn.apache.org/community/release_notes/release_note_0.2.0 > > Website: https://celeborn.apache.org/ > > Celeborn Resources: > - Issue: https://issues.apache.org/jira/projects/CELEBORN > - Mailing list: dev@celeborn.apache.org > > Ethan Feng > On behalf of the Apache Celeborn(incubating) community >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.0-incubating-rc4
+1 (binding) I checked - links are valid. - "incubating" is in the name. - LICENSE looks good. - NOTICE looks good. - DISCLAIMER exists. - signatures are good. ``` wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS gpg --import KEYS gpg --verify apache-celeborn-0.2.0-incubating-source.tgz.asc gpg --verify apache-celeborn-0.2.0-incubating-bin.tgz.asc ``` - checksums are good. ``` sha512sum --check apache-celeborn-0.2.0-incubating-bin.tgz.sha512 sha512sum --check apache-celeborn-0.2.0-incubating-source.tgz.sha512 ``` - build success from source code (openjdk-8, ubuntu 22.04 x86). ``` build/make-distribution.sh --release ``` Thanks, Cheng Pan On Feb 7, 2023 at 11:06:34, Yu Li wrote: > Hi Angers and Kerwin, > > Could you supplement some more details (items checked against this RC > according to the checklist [1]) for your voting? Thanks. > > Best Regards, > Yu > > [1] > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > On Mon, 6 Feb 2023 at 17:34, kerwin zhang > wrote: > > +1 (binding) > > > angers zhu 于2023年2月6日周一 17:33写道： > > > > +1 (binding) > > > > > > Keyong Zhou 于2023年2月6日周一 17:31写道： > > > > > > > +1 (binding) > > > > > > > > I checked > > > > > > > > - git commit hash is correct. > > > > - links are valid. > > > > - "incubating" is in the name. > > > > - PGP keys are good. > > > > - hashes are correct. > > > > - LICENSE looks good. > > > > - NOTICE looks good. > > > > - DISCLAIMER exists. > > > > - build success from source code (macOS). > > > > > > > > ``` > > > > ./build/make-distribution.sh --release > > > > ``` > > > > > > > > Thanks, > > > > Keyong Zhou > > > > > > > > Ethan Feng 于2023年2月4日周六 21:29写道： > > > > > > > > > Hi Celeborn community, > > > > > > > > > > This is a call for a vote to release Apache Celeborn (Incubating) > > > > > 0.2.0-incubating-rc4 > > > > > > > > > > The git tag to be voted upon: > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc4 > > > > > > > > > > The git commit hash: > > > > > 282e0b0bbc76fa9339e931b0b252b6cbb16dddf5 > > > > > > > > > > The source and binary artifacts can be found at: > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.0-incubating-rc4/ > > > > > > > > > > Fingerprint of the PGP key release artifacts are signed with: > > > > > FCF20BB29C7BEFDF58F998F76392F71F37356FA0 > > > > > > > > > > My public key to verify signatures can be found in: > > > > > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > > > > > > > > The vote will be open for at least 72 hours or until necessary > > > > > number of votes are reached. > > > > > > > > > > Please vote accordingly: > > > > > > > > > > [ ] +1 approve > > > > > [ ] +0 no opinion > > > > > [ ] -1 disapprove (and the reason) > > > > > > > > > > Checklist for release: > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > > > > Steps to validate the release: > > > > > https://www.apache.org/info/verification.html > > > > > > > > > > Starting with my +1 (binding): > > > > > > > > > > * Download links, checksums and PGP signatures are valid. > > > > > * Source code distributions have correct names matching the current > > > > > release. > > > > > * Release files have the word incubating in their name. > > > > > * DISCLAIMER, LICENSE and NOTICE files are correct. > > > > > * All files have license headers if necessary. > > > > > * No unlicensed compiled archives bundled in source archive. > > > > > * The source tarball matches the git tag. > > > > > * Build from source is successful. > > > > > > > > > > Thanks, > > > > > Ethan Feng > > > > > > > > > > > > > > >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.0-incubating-rc3
+1 (binding) I checked - links are valid. - "incubating" is in the name. - LICENSE looks good. - NOTICE looks good. - DISCLAIMER exists. - signatures are good. ``` wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS gpg --import KEYS gpg --verify apache-celeborn-0.2.0-incubating-source.tgz.asc gpg --verify apache-celeborn-0.2.0-incubating-bin.tgz.asc ``` - checksums are good. ``` sha512sum --check apache-celeborn-0.2.0-incubating-bin.tgz.sha512 sha512sum --check apache-celeborn-0.2.0-incubating-source.tgz.sha512 ``` - build success from source code (openjdk-8, macOS aarch64). ``` build/make-distribution.sh --release ``` Thanks, Cheng Pan On Jan 19, 2023 at 11:30:37, Keyong Zhou wrote: > +1 (binding) > > I checked > > - git commit hash is correct. > - links are valid. > - "incubating" is in the name. > - PGP keys are good. > - hashes are correct. > - LICENSE looks good. > - NOTICE looks good. > - DISCLAIMER exists. > - build success from source code (macOS). > > ``` > ./build/make-distribution.sh --release > ``` > > Thanks, > Keyong Zhou > > > Ethan Feng 于2023年1月18日周三 22:01写道： > > Hi Celeborn community, > > > This is a call for the vote to release Apache Celeborn (Incubating) > > 0.2.0-incubating-rc3 > > > The git tag to be voted upon: > > > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc3 > > > The git commit hash: > > 98b356f599b9e8960d755cb6add8f6934345d2a8 > > > The source and binary artifacts can be found at: > > > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.0-incubating-rc3/ > > > The fingerprint of the PGP key release artifacts is signed with: > > FCF20BB29C7BEFDF58F998F76392F71F37356FA0 > > > My public key to verify signatures can be found in: > > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > > The vote will be open for at least 72 hours or until necessary > > number of votes are reached. > > > Please vote accordingly: > > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove (and the reason) > > > Checklist for release: > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > Steps to validate the release: > > https://www.apache.org/info/verification.html > > > Starting with my +1 (binding): > > > * Download links, checksums and PGP signatures are valid. > > * Source code distributions have correct names matching the current > > release. > > * Release files have the word incubating in their name. > > * DISCLAIMER, LICENSE and NOTICE files are correct. > > * All files have license headers if necessary. > > * No unlicensed compiled archives bundled in source archive. > > * The source tarball matches the git tag. > > * Build from source is successful. > > > Thanks, > > Ethan Feng > > >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.0-incubating-rc2
+1 (non-binding) I checked - links are valid. - "incubating" is in the name. - LICENSE looks good. - NOTICE looks good. - DISCLAIMER exists. - signatures are good. ``` wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS gpg --import KEYS gpg --verify apache-celeborn-0.2.0-incubating-source.tgz.asc gpg --verify apache-celeborn-0.2.0-incubating-bin.tgz.asc ``` - checksums are good. ``` sha512sum --check apache-celeborn-0.2.0-incubating-bin.tgz.sha512 sha512sum --check apache-celeborn-0.2.0-incubating-source.tgz.sha512 ``` - build success from source code (openjdk-8, macOS aarch64). ``` build/make-distribution.sh --release ``` Thanks, Cheng Pan Thanks, Cheng Pan On Jan 9, 2023 at 12:07:42, Ethan Feng wrote: > Hello Incubator Community, > > This is a call for a vote to release Apache Celeborn(Incubating) version > 0.2.0-incubating-rc2 > > The Apache Celeborn community has voted on and approved a proposal to > release > Apache Celeborn(Incubating) version 0.2.0-incubating-rc2 > > We now kindly request the Incubator PMC members review and vote on this > incubator release. > > Celeborn community vote thread: > • https://lists.apache.org/thread/v4t7r6h8043s0hvhhvlzhb35nr9gvshr > > Vote result thread: > • https://lists.apache.org/thread/7n0gw4dz8d5p852sfok9f61cqods6h76 > > The release candidate: > • > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.0-incubating-rc2 > > Git tag for the release: > • > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc2 > > Public keys file: > • https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The change log is available in: > • > > https://github.com/apache/incubator-celeborn/compare/v0.1.4...v0.2.0-incubating-rc2 > > The vote will be open for at least 72 hours or until the necessary number > of votes are reached. > > Please vote accordingly: > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove with the reason > > More detailed checklist please refer: > • > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > Steps to validate the release， Please refer to: > • https://www.apache.org/info/verification.html > > Thanks, > On behalf of Apache Celeborn(Incubating) community >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.0-incubating-rc2
+1 (non-binding) I checked - links are valid. - "incubating" is in the name. - LICENSE looks good. - NOTICE looks good. - DISCLAIMER exists. - signatures are good. ``` wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS gpg --import KEYS gpg --verify apache-celeborn-0.2.0-incubating-source.tgz.asc gpg --verify apache-celeborn-0.2.0-incubating-bin.tgz.asc ``` - checksums are good. ``` sha512sum --check apache-celeborn-0.2.0-incubating-bin.tgz.sha512 sha512sum --check apache-celeborn-0.2.0-incubating-source.tgz.sha512 ``` - build success from source code (openjdk-8, macOS aarch64). ``` build/make-distribution.sh --release ``` Thanks, Cheng Pan On Jan 9, 2023 at 12:07:42, Ethan Feng wrote: > Hello Incubator Community, > > This is a call for a vote to release Apache Celeborn(Incubating) version > 0.2.0-incubating-rc2 > > The Apache Celeborn community has voted on and approved a proposal to > release > Apache Celeborn(Incubating) version 0.2.0-incubating-rc2 > > We now kindly request the Incubator PMC members review and vote on this > incubator release. > > Celeborn community vote thread: > • https://lists.apache.org/thread/v4t7r6h8043s0hvhhvlzhb35nr9gvshr > > Vote result thread: > • https://lists.apache.org/thread/7n0gw4dz8d5p852sfok9f61cqods6h76 > > The release candidate: > • > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.0-incubating-rc2 > > Git tag for the release: > • > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc2 > > Public keys file: > • https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The change log is available in: > • > > https://github.com/apache/incubator-celeborn/compare/v0.1.4...v0.2.0-incubating-rc2 > > The vote will be open for at least 72 hours or until the necessary number > of votes are reached. > > Please vote accordingly: > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove with the reason > > More detailed checklist please refer: > • > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > Steps to validate the release， Please refer to: > • https://www.apache.org/info/verification.html > > Thanks, > On behalf of Apache Celeborn(Incubating) community >
Re: [VOTE] Release Apache Celeborn(Incubating) 0.2.0-incubating-rc2
+1 (binding) I checked - links are valid. - "incubating" is in the name. - LICENSE looks good. - NOTICE looks good. - DISCLAIMER exists. - signatures are good. ``` wget https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS gpg --import KEYS gpg --verify apache-celeborn-0.2.0-incubating-source.tgz.asc gpg --verify apache-celeborn-0.2.0-incubating-bin.tgz.asc ``` - checksums are good. ``` sha512sum --check apache-celeborn-0.2.0-incubating-bin.tgz.sha512 sha512sum --check apache-celeborn-0.2.0-incubating-source.tgz.sha512 ``` - build success from source code (openjdk-8, macOS aarch64). ``` build/make-distribution.sh --release ``` Thanks, Cheng Pan On Jan 5, 2023 at 16:04:00, Ethan Feng wrote: > Hi Celeborn community, > > This is a call for vote to release Apache Celeborn (Incubating) > 0.2.0-incubating-rc2 > > The git tag to be voted upon: > > https://github.com/apache/incubator-celeborn/releases/tag/v0.2.0-incubating-rc2 > > The git commit hash: > fcd1e8ac8723859648e4597092328fa1ed3b96c2 > > The source and binary artifacts can be found at: > > https://dist.apache.org/repos/dist/dev/incubator/celeborn/v0.2.0-incubating-rc2/ > > Fingerprint of the PGP key release artifacts are signed with: > FCF20BB29C7BEFDF58F998F76392F71F37356FA0 > > My public key to verify signatures can be found in: > https://dist.apache.org/repos/dist/release/incubator/celeborn/KEYS > > The vote will be open for at least 72 hours or until necessary > number of votes are reached. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason) > > Checklist for release: > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > Steps to validate the release: > https://www.apache.org/info/verification.html > > Starting with my +1 (binding): > > * Download links, checksums and PGP signatures are valid. > * Source code distributions have correct names matching the current > release. > * Release files have the word incubating in their name. > * DISCLAIMER, LICENSE and NOTICE files are correct. > * All files have license headers if necessary. > * No unlicensed compiled archives bundled in source archive. > * The source tarball matches the git tag. > * Build from source is successful. > > Thanks, > Ethan Feng >
Re: [DISCUSS] Prepare first release
Hi community, I test branch-0.2 on K8s w/ CELEBORN-151 and some internal modification by running some TPC-DS quires, works as expected. I noticed that there are some configurations still use `rss.` prefix instead of `celeborn.`, we should fix them before releasing 0.2.0. Thanks, Cheng Pan On Dec 19, 2022 at 18:25:44, keyong zhou wrote: > Hi Celeborn (Incubating) community, > > I have tested the core-path and fixed on bug[1]. After fixing this, my test > seems OK. > > The way I tested is as follows: > I run 1T TPCDS with Celeborn on, then I randomly kill worker during the > test, carefully not to trigger datalost. In the end I check whether the > results are correct. > > [1] https://github.com/apache/incubator-celeborn/pull/1101 > > Thanks, > Keyong Zhou > > Keyong Zhou 于2022年12月14日周三 19:41写道： > > Hi celeborn (-incubating) community: > > > Currently we are preparing for the first release (branch-0.2). To ensure > > code quality, I would like to test for core-path correctness and stability, > > could Angerszh and nafiyaix > > help test graceful shutdown and rolling upgrade? > > And could Ethan Feng help test load-aware > > slots allocation? > > > And we would be rather happy if anyone can help test for other modules > > (k8s, HA, etc.). > > > Thanks, > > Keyong > > >
Re: [DISCUSS] Add Celeborn on Kubernetes Integration Test
Hi Binjie, Thanks for bringing up this discussion, I think leverage GitHub Action and Minicube/K3s is the first step we can do, but considering that GitHub Runner has 7G memory limitation(I don’t know about cpu limitation), I’m not sure if we can test Master HA mode on that. And thanks Keyong for providing external K8s cluster, it’s really helpful for the quality of Celeborn. Thanks, Cheng Pan On Dec 7, 2022 at 17:44:04, wangsheng <2213335...@qq.com.invalid> wrote: > Hi Zhou, > > Thanks for your info about the External Kubernetes Cluster. > > I will enrich the solution in case of using External Kubernetes Cluster. > > From my opinion, using an external Kubernetes cluster will improve the > speed of CI by a large margin over using a built-in Kubernetes cluster. > > Anyway, I will enrich both solution, and bring these to Github issue. > > Thanks, > Binjie Yang > > 2022年12月7日 下午3:43，Keyong Zhou 写道： > > > Hi sheng, > > > Thanks for the discussion. For the option of using External Kubernetes > > Cluster, currently we already have an external cluster for test purposes, > > maybe we can reuse the cluster. If necessary, we can also offer a small k8s > > cluster. > > > So if the external cluster is available, which solution do you prefer? > > > Thanks, > > Keyong Zhou > > > wangsheng <2213335...@qq.com.invalid> 于2022年12月6日周二 14:33写道： > > > > Hi Apache Celeborn(Incubating) Community Members: > > > > > > Current Apache Celeborn(Incubating), hereinafter referred to as Celeborn, > > > provides On Kubernetes capabilities and the corresponding Helm Chart, but > > > integration tests for On Kubernetes scenarios are missing. > > > > > > To ensure the availability and stability of the On Kubernetes case, we > > > need to add the above integration tests to the Github workflow. > > > > > > I have added some details in > > > > https://docs.google.com/document/d/1iIRFhXBFFsU6NX_zfi7kuCEYnh6NHUZh1KpOqSSp16s/edit?usp=sharing > > > < > > > > https://docs.google.com/document/d/1iIRFhXBFFsU6NX_zfi7kuCEYnh6NHUZh1KpOqSSp16s/edit?usp=sharing > >, > > > welcome your discussion and comments. > > > > > > Best, > > > Binjie Yang > > > >
Re: [DISCUSS] Prepare first release
+1 for release, the branch name should be branch-0.2, keep consistent w/ existed branch-0.1 Thanks, Cheng Pan On Dec 3, 2022 at 17:44:30, Keyong Zhou wrote: > Hi celeborn (-incubating) community: > > One and a half months has passed since celeborn entered Apache Incubator. > Although it's new to Apache, it's already widely deployed on production > environment. I think it's time to prepare for our first release. > > After discussing with several developers, we have a preliminary agreement > that we can (almost) check out the branch-0.2.0-rc1 release branch from > main. If you have any questions or ideas, feel free to discuss on this > thread. > > Thanks! > Keyong Zhou >
Re: [ANNOUNCE] Add nafiyaix(Xia Yifan) as new committer
Congrats, and thanks for your continuous work on Celeborn project. Thanks, Cheng Pan On Dec 2, 2022 at 14:09:49, angers zhu wrote: > Congratulations！ > > Best Regards > Anegrszh > > Yu Li 于2022年12月2日周五 12:53写道： > > Congratulations and welcome, Yifan! > > > Best Regards, > > Yu > > > On Wed, Nov 30, 2022 at 11:25 PM Keyong Zhou wrote: > > > > Hi celeborn(-incubating) community, > > > > > > I'm very excited to announce that recently we added nafiyaix(Xia Yifan) > > as > > > our new Committer! > > > > > > nafiyaix has kept contributing to celeborn for more than six months, > > > including new features, bugfixes, refactors, and documentation. > > > > > > Looking forward that nafiyaix will continue contributing to the project, > > > pushing celeborn to the next level together with all contributors of the > > > community! > > > > > > Also, we are looking forward to add more and more committers to our > > project > > > :) > > > > > > Thanks! > > > Keyong Zhou > > > > > -- > > Best Regards, > > Yu > > >
