On Tue, Oct 24, 2023 at 10:35 AM Kenneth Knowles <k...@apache.org> wrote:
> Tangentially related: > > Long ago, attaching an issue to a release was a mandatory step as part of > closing. Now I think it is not. Is it automatically happening? It looks > like we have 820 with no milestone > https://github.com/apache/beam/issues?q=is%3Aissue+no%3Amilestone+is%3Aclosed > This could (should) be automatically discoverable. A (closed) issues is associated with commits which are associated with a release. > On Tue, Oct 24, 2023 at 1:25 PM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> +1 for going by the commits since this is what matters at the end of the >> day. Also, many issues may not get tagged correctly for a given release due >> to either the contributor not tagging the issue or due to commits for the >> issue spanning multiple Beam releases. >> >> For example, >> >> For all commits in a given release RC: >> * If we find a Github issue for the commit: add a notice to the Github >> issue >> * Else: add the notice to a generic issue for the release including >> tags for the commit ID, PR author, and the committer who merged the PR. >> >> Thanks, >> Cham >> >> >> >> >> On Mon, Oct 23, 2023 at 11:49 AM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> I'd probably vote to include both the issue filer and the contributor. >>> It is pretty equally straightforward - one way to do this would be using >>> all issues related to that release's milestone and extracting the issue >>> author and the issue closer. >>> >>> This does leave out the (unfortunately sizable) set of contributions >>> that don't have an associated issue; if we're worried about that, we could >>> always fall back to anyone with a commit in the last release who doesn't >>> have an associated issue (aka what I thought we were initially proposing >>> and what I think Airflow does today). >>> >>> I'm pretty much +1 on any sort of automation here, and it certainly can >>> come in stages :) >>> >>> On Mon, Oct 23, 2023 at 1:50 PM Johanna Öjeling via dev < >>> dev@beam.apache.org> wrote: >>> >>>> Yes that's a good point to include also those who created the issue. >>>> >>>> On Mon, Oct 23, 2023, 19:18 Robert Bradshaw via dev < >>>> dev@beam.apache.org> wrote: >>>> >>>>> On Mon, Oct 23, 2023 at 7:26 AM Danny McCormick via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> So to summarize, I think there's broad consensus (or at least lazy >>>>>> consensus) around the following: >>>>>> >>>>>> - (1) Updating our release email/guidelines to be more specific about >>>>>> what we mean by release validation/how to be helpful during this process. >>>>>> This includes both encouraging validation within each user's own code >>>>>> base >>>>>> and encouraging people to document/share their process of validation and >>>>>> link it in the release spreadsheet. >>>>>> - (2) Doing something like what Airflow does (#29424 >>>>>> <https://github.com/apache/airflow/issues/29424>) and creating an >>>>>> issue asking people who have contributed to the current release to help >>>>>> validate their changes. >>>>>> >>>>>> I'm also +1 on doing both of these. The first bit (updating our >>>>>> guidelines) is relatively easy - it should just require updating >>>>>> https://github.com/apache/beam/blob/master/contributor-docs/release-guide.md#vote-and-validate-the-release-candidate >>>>>> . >>>>>> >>>>>> I took a look at the second piece (copying what Airflow does) to see >>>>>> if we could just copy their automation, but it looks like it's tied >>>>>> to airflow breeze >>>>>> <https://github.com/apache/airflow/blob/main/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2> >>>>>> (their repo-specific automation tooling), so we'd probably need to build >>>>>> the automation ourselves. It shouldn't be terrible, basically we'd want a >>>>>> GitHub Action that compares the current release tag with the last release >>>>>> tag, grabs all the commits in between, parses them to get the author, and >>>>>> creates an issue with that data, but it does represent more effort than >>>>>> just updating a markdown file. There might even be an existing Action >>>>>> that >>>>>> can help with this, I haven't looked too hard. >>>>>> >>>>> >>>>> I was thinking along the lines of a script that would scrape the >>>>> issues resolved in a given release and add a comment to them noting that >>>>> the change is in release N and encouraging (with clear instructions) how >>>>> this can be validated. Creating a "validate this release" issue with all >>>>> "contributing" participants could be an interesting way to do this as >>>>> well. >>>>> (I think it'd be valuable to get those who filed the issue, not just those >>>>> who fixed it, to validate.) >>>>> >>>>> >>>>>> As our next release manager, I'm happy to review PRs for either of >>>>>> these if anyone wants to volunteer to help out. If not, I'm happy to >>>>>> update >>>>>> the guidelines, but I probably won't have time to add the commit >>>>>> inspection >>>>>> tooling (I'm planning on throwing any extra time towards continuing to >>>>>> automate release candidate creation which is currently a more impactful >>>>>> problem IMO). I would very much like it if both of these things happened >>>>>> though :) >>>>>> >>>>>> Thanks, >>>>>> Danny >>>>>> >>>>>> On Mon, Oct 23, 2023 at 10:05 AM XQ Hu <x...@google.com> wrote: >>>>>> >>>>>>> +1. This is a great idea to try. @Danny McCormick >>>>>>> <dannymccorm...@google.com> FYI as our next release manager. >>>>>>> >>>>>>> On Wed, Oct 18, 2023 at 2:30 PM Johanna Öjeling via dev < >>>>>>> dev@beam.apache.org> wrote: >>>>>>> >>>>>>>> When I have contributed to Apache Airflow, they have tagged all >>>>>>>> contributors concerned in a GitHub issue when the RC is available and >>>>>>>> asked >>>>>>>> us to validate it. Example: #29424 >>>>>>>> <https://github.com/apache/airflow/issues/29424>. >>>>>>>> >>>>>>>> I found that to be an effective way to notify contributors of the >>>>>>>> RC and nudge them to help out. In the issue description there is a >>>>>>>> reference to the guidelines on how to test the RC and a note that >>>>>>>> people >>>>>>>> are encouraged to vote on the mailing list (which could admittedly be >>>>>>>> more >>>>>>>> highlighted because I did not pay attention to it until now and was >>>>>>>> unaware >>>>>>>> that contributors had a vote). >>>>>>>> >>>>>>>> It might be an idea to consider something similar here to increase >>>>>>>> the participation? >>>>>>>> >>>>>>>> On Tue, Oct 17, 2023 at 7:01 PM Jack McCluskey via dev < >>>>>>>> dev@beam.apache.org> wrote: >>>>>>>> >>>>>>>>> I'm +1 on helping explain what we mean by "validate the RC" since >>>>>>>>> we're really just asking users to see if their existing use cases work >>>>>>>>> along with our typical slate of tests. I don't know if offloading >>>>>>>>> that work >>>>>>>>> to our active validators is the right approach though, >>>>>>>>> documentation/screen >>>>>>>>> share of their specific workflow is definitely less useful than >>>>>>>>> having a >>>>>>>>> more general outline of how to install the RC and things to look out >>>>>>>>> for >>>>>>>>> when testing. >>>>>>>>> >>>>>>>>> On Tue, Oct 17, 2023 at 12:55 PM Austin Bennett <aus...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Great effort. I'm also interested in streamlining releases -- so >>>>>>>>>> if there are alot of manual tests that could be automated, would be >>>>>>>>>> great >>>>>>>>>> to discover and then look to address. >>>>>>>>>> >>>>>>>>>> On Tue, Oct 17, 2023 at 8:47 AM Robert Bradshaw via dev < >>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>> >>>>>>>>>>> +1 >>>>>>>>>>> >>>>>>>>>>> I would also strongly suggest that people try out the release >>>>>>>>>>> against their own codebases. This has the benefit of ensuring the >>>>>>>>>>> release >>>>>>>>>>> won't break your own code when they go out, and stress-tests the >>>>>>>>>>> new code >>>>>>>>>>> against real-world pipelines. (Ideally our own tests are all >>>>>>>>>>> passing, and >>>>>>>>>>> this validation is automated as much as possible (though ensuring it >>>>>>>>>>> matches our documentation and works in a clean environment still has >>>>>>>>>>> value), but there's a lot of code and uses out there that we don't >>>>>>>>>>> have >>>>>>>>>>> access to during normal Beam development.) >>>>>>>>>>> >>>>>>>>>>> On Tue, Oct 17, 2023 at 8:21 AM Svetak Sundhar via dev < >>>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I’ve participated in RC testing for a few releases and have >>>>>>>>>>>> observed a bit of a knowledge gap in how releases can be tested. >>>>>>>>>>>> Given that >>>>>>>>>>>> Beam encourages contributors to vote on RC’s regardless of tenure, >>>>>>>>>>>> and that >>>>>>>>>>>> voting on an RC is a relatively low-effort, high leverage way to >>>>>>>>>>>> influence >>>>>>>>>>>> the release of the library, I propose the following: >>>>>>>>>>>> >>>>>>>>>>>> During the vote for the next release, voters can document the >>>>>>>>>>>> process they followed on a separate document, and add the link on >>>>>>>>>>>> column G >>>>>>>>>>>> here >>>>>>>>>>>> <https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=437054928>. >>>>>>>>>>>> One step further, could be a screencast of running the test, and >>>>>>>>>>>> attaching >>>>>>>>>>>> a link of that. >>>>>>>>>>>> >>>>>>>>>>>> We can keep repeating this through releases until we have >>>>>>>>>>>> documentation for many of the different tests. We can then add >>>>>>>>>>>> these docs >>>>>>>>>>>> into the repo. >>>>>>>>>>>> >>>>>>>>>>>> I’m proposing this because I’ve gathered the following feedback >>>>>>>>>>>> from colleagues that are tangentially involved with Beam: They are >>>>>>>>>>>> interested in participating in release validation, but don’t know >>>>>>>>>>>> how to >>>>>>>>>>>> get started. Happy to hear other suggestions too, if there are any >>>>>>>>>>>> to >>>>>>>>>>>> address the above. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Svetak Sundhar >>>>>>>>>>>> >>>>>>>>>>>> Data Engineer >>>>>>>>>>>> s <nellywil...@google.com>vetaksund...@google.com >>>>>>>>>>>> >>>>>>>>>>>>
