In case it's not clear where `comdev@` is (it wasn't at all to me), here's the thread Julian referenced: https://lists.apache.org/thread.html/rf12538ef0f60f7257e63391e5d4962a6156564020c99d3dfb193f4d7%40%3Cdev.community.apache.org%3E
Neal On Mon, May 3, 2021 at 1:25 PM Julian Hyde <jh...@apache.org> wrote: > If you think that a release every two weeks, following the standard > voting-on-signed-tarball process, is not too onerous, then we're good. > If you read my email thread on comdev@ you will see that SkyWalking > has been following a similar process. > > On Mon, May 3, 2021 at 11:41 AM Andrew Lamb <al...@influxdata.com> wrote: > > > > There have been some great discussions on the document. > > > > I would say the major unresolved questions are > > > > 1. What branching strategy would best balance frequent releases, > > maintainers' time, and contributors' time. > > > > 2. (related) How do we handle breaking API changes between major releases > > that aren't compatible with Rust SemVer expectations? Feature flags? > > Restricted PR merge windows? Separated maintenance branches? > > > > I'll wait a few more days to see how we are doing consensus wise. > > > > Thank you to everyone who has contributed already, > > Andrew > > > > > > On Sun, May 2, 2021 at 8:44 AM Adam Lippai <a...@rigo.sk> wrote: > > > > > Hi, > > > > > > my two cents: at my previous workplace (Tresorit) we created releases > every > > > week and that process was even heavier including cross > > > platform verification and approval, a 8-12 hours long dedicated manual > QA > > > process, discussions with the marketing and support teams. > > > Open source is certainly different as we had 2-8 full time devs per > > > platform and 2 dedicated QA engineers to ensure this pace (including > the > > > actual development), however when we had struggles it was because: > > > > > > 1. People were on vacation and the flow was disrupted > > > 2. We had big, cross platform changes (timespan over 2 weeks) which > we > > > didn't merge back in time, it hurt us in 50%+ of the cases when we > > > tried to > > > "do it in separate feature branch", which might or might not be a > lot > > > > > > The overhead wasn't big, it was almost only the communication with the > > > manual QA and creating human readable release notes. It was never about > > > doing a lot of work, doing only a little, doing bugfix, features or > > > refactoring. It didn't really matter for the overhead or failure rate. > > > A few years later Google devops started promoting DORA: > > > > > > > https://cloud.google.com/blog/products/devops-sre/another-way-to-gauge-your-devops-performance-according-to-dora > > > and their experience is the same, more than one release per month is > needed > > > to make it stable with lower failure rate and being able to act within > a > > > day if anything goes wrong. > > > My team was web and javascript focused (but we provided APIs) and we > > > automated most of the deployment process (well, not the approval and > manual > > > QA of course), however I saw this working pretty well for years, if > > > anything increasing the pace made the process more deterministic. > > > > > > Strictness of the language and the compiler might render this approach > > > impractical (eg. TypeScript and TS based libs struggle more than JS), > but I > > > think management and QA-wise this proposal improves the process > eventually > > > and we'll get a better product *while not working more.* > > > > > > I hope this little anecdotal story reassures some of you that this is > not > > > an over-ambitious change and it enables everyone to continue creating > great > > > things. > > > > > > Best regards, > > > Adam Lippa > > > > > > On Sun, May 2, 2021 at 1:00 PM Andrew Lamb <al...@influxdata.com> > wrote: > > > > > > > Micah and Julian, thank you both for your thoughts. > > > > > > > > I largely agree with Micah; While the Apache process may be heavy > > > weight > > > > in certain aspects, I think we can achieve Rust releases every 2 week > > > > within that framework. > > > > > > > > As I doubt very much we'll get it perfect on the first try, I was > > > > envisioning that we start with 2 week releases, see how it goes for a > > > > while, and then adjust as needed. > > > > > > > > Andrew > > > > > > > > > > > > On Sat, May 1, 2021 at 11:20 PM Micah Kornfield < > emkornfi...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > Apache releases are quite heavyweight, so > > > > > > work best at a frequency of 2 - 3 months, whereas (IIUC) Rust > > > > > > developers expect a more lightweight release on a weekly cadence. > > > > > > > > > > > > > > > Thanks, I think clarifications on requirements are helpful. IIUC > is is > > > > > actually a 2 week cadence which is still fast but seems doable with > > > > > dedicated community members (and some investment in tooling). > > > > > > > > > > What makes the releases heavy weight? It seems like the process is > > > > > slightly more tedious than necessarily onerous. Generating a > signed > > > > > tarball, seems like it should take ~5 minutes or less with the > proper > > > > > tooling? Verification is more heavy weight but again with the > proper > > > > > tooling and a good system for testing out more changes, it does not > > > seem > > > > > like it should take too much developer time if no issues arise. > There > > > > are > > > > > 3 active contributors to Rust on the PMC, so if they are willing to > > > sign > > > > up > > > > > for doing the work of verification and voting on this cadence, what > > > would > > > > > the other requirements around the process be? > > > > > > > > > > Best, > > > > > Micah > > > > > > > > > > On Sat, May 1, 2021 at 3:05 PM Julian Hyde <jh...@apache.org> > wrote: > > > > > > > > > > > The main tension is not in the proposal but the requirements. > It's a > > > > > > classic impedance mismatch. Apache releases are quite > heavyweight, so > > > > > > work best at a frequency of 2 - 3 months, whereas (IIUC) Rust > > > > > > developers expect a more lightweight release on a weekly > cadence. I > > > > > > was trying to find other projects that had had the same problem, > and > > > > > > solved it somehow. And also raise awareness within Apache that > the > > > > > > release process is problematic for some communities in 2021. > > > > > > > > > > > > To correct a couple of misconceptions: > > > > > > * In Apache, the signed source artifacts (tarball) are literally > the > > > > > > release. Not a git hash, not a set of binary artifacts. That is > what > > > > > > people need to vote on. > > > > > > * The release vote does not have to last 72 hours. It can be a > > > shorter > > > > > > period, if the community agrees. > > > > > > > > > > > > Julian > > > > > > > > > > > > > > > > > > On Sat, May 1, 2021 at 1:31 PM Micah Kornfield < > > > emkornfi...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > Hi Julian, > > > > > > > I didn't read this proposal as being in tension with apache > > > releases. > > > > > It > > > > > > > sounds like the intention is to hold a vote every two weeks to > > > > verify a > > > > > > > release artifacts? But maybe I misread or missed something. > Were > > > do > > > > > you > > > > > > > think the tension lies? Is it also producing the signed source > > > > > artifact? > > > > > > > > > > > > > > Since votes last for at least 72 hours this does seem like a > lot of > > > > > > > overhead every two weeks, but it seems that is something for > Rust > > > > > > > maintainers to decide and adjust. > > > > > > > > > > > > > > -Micah > > > > > > > > > > > > > > On Saturday, May 1, 2021, Julian Hyde <jh...@apache.org> > wrote: > > > > > > > > > > > > > > > (Removing user@ from cc. I think this is mainly a dev@ > issue.) > > > > > > > > > > > > > > > > I believe there are some tensions between this process and > the > > > > Apache > > > > > > > > process. In particular, Apache releases tend to be a signed > > > source > > > > > > > > distribution (tarball) that at least three PMC members > download > > > and > > > > > > > > verify. I totally understand why, as Rust developers, you > might > > > > find > > > > > > > > that an onerous process and might want to operate in a > different > > > > way. > > > > > > > > It makes sense, and I believe we can solve it. Perhaps by > using a > > > > > word > > > > > > > > other than "release" for high-frequency snapshots. > > > > > > > > > > > > > > > > It is likely that other projects have already run into this > > > problem > > > > > > > > and have solved it. Therefore I have sent an email to comdev > > > asking > > > > > > > > for advice [1]. Feel free to join the thread. > > > > > > > > > > > > > > > > Julian > > > > > > > > > > > > > > > > [1] > > > > > > > https://lists.apache.org/thread.html/rf12538ef0f60f7257e63391e5d496 > > > > > > > > 2a6156564020c99d3dfb193f4d7%40%3Cdev.community.apache.org%3E > > > > > > > > > > > > > > > > On Sat, May 1, 2021 at 4:01 AM Andrew Lamb < > al...@influxdata.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > I propose regularly releasing, every 2 weeks, minor and > patch > > > > > > releases > > > > > > > > of > > > > > > > > > the arrow-rs crate, following the semver versioning scheme > used > > > > by > > > > > > the > > > > > > > > rest > > > > > > > > > of the Rust ecosystem. I have written a proposal[1] > describing > > > > how > > > > > > this > > > > > > > > > might work. > > > > > > > > > > > > > > > > > > Feedback and comments most welcome. > > > > > > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > https://docs.google.com/document/d/1QTGah5dkRG0Z6Gny_ > > > > > > > > QCHmqMg7L2HmcbEpRISsfNEhSA/edit > > > > > > > > > > > > > > > > > > > > > > > > > > >
