Hi Jihoon, Thanks for pointing that out. Then, I'd update my proposal to include this step as something that always needs to be done manually.
> What I think should always be done manually: > > - [ASF requirement] download all signed source code packages, validate checksums + signatures, compile without tests. start a Druid cluster and ingest wikipedia data onto a micro-quickstart cluster and run a simple query against it. > - [Druid requirement] Vote on whether any features are being released that are incomplete - missing integration tests, un-expected change in behavior, etc. > - [Druid requirement] Verify that all the automated tests that run for the tag have actually passed (maybe this can be automated too?). I imagine this step is to look at travis for the tag to ensure that all the tests actually passed. https://travis-ci.com/github/apache/druid/builds/224182054 - this is the travis build for the 0.21.0 tag (which appears to be failing) If we get agreement on this, I'll be happy to add / update a doc that talks about what specifically is expected from someone voting on an Apache Druid release. While writing out this proposal below, I've realized we have a set of integration tests that do not run as part of the CI pipeline because they require credentials to some third party systems (eg. Kinesis ITs) or we can't get the tests to consistently run in the Travis pipeline without intermittent failures (eg. Hadoop ITs). Does a person voting need to run these integration tests today, since they are not part of the automated pipeline? On Mon, Apr 26, 2021 at 12:30 PM Jihoon Son <jihoon...@apache.org> wrote: > Hi Suneet, thanks for your suggestion. > > However, the Apache release policy [1] explicitly says the below. > > > Before casting +1 binding votes, individuals are REQUIRED to download > all signed source code packages onto their own hardware, verify that they > meet all requirements of ASF policy on releases as described below, > validate all cryptographic signatures, compile as provided, and test the > result on their own platform. > > Because only the source release is our official release, PMCs have to > download at least the source code package and verify it manually per > this requirement. > That being said, we can automate only verifying the binary package and > docker image at this moment. > > Jihoon > > [1] https://www.apache.org/legal/release-policy.html#policy > > On Fri, Apr 23, 2021 at 6:26 PM Suneet Saldanha > <suneet.salda...@imply.io> wrote: > > > > Hi Druids, > > > > I was thinking about the 0.21.0 release vote that was just happening, and > > found it tedious to help with the vote because I needed to build the code > > from source as well as run all the tests on my local machine. This got me > > thinking about what are the key steps that we should check manually in > the > > release process and what steps can we rely on automation that exists in > > Travis today. I don't have full context on what steps are needed, but I > > thought I'd start a thread to hear from everyone about what steps should > be > > done before a release, and whether any of them need to be done manually, > or > > if we could leverage our CI pipelines to replace these steps. > > > > What I think should be done before the release? > > > > - Run all unit tests > > - RAT License checks > > - Run all integration tests > > - Verify checksums > > - Verify signatures > > - Verify release notes contain > > > > What I think should be done manually (because automation doesn't exist > > today): > > > > - Verify docker pull works > > - Verify Druid can start from the pulled docker image > > - Verify no open issues on github with the milestone for the version > > being released > > - Verify release notes contains a note about all issues marked with > > release notes label > > > > > > What I think should always be done manually: > > > > - Vote on whether any features are being released that are incomplete > > - Verify that all the automated tests that run for the tag have > actually > > passed (maybe this can be automated too?) > > > > Look forward to hearing thoughts on this :) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > For additional commands, e-mail: dev-h...@druid.apache.org > >
