Re: [PATCH RFC] contrib: script to generate release notes from commit messages
Augie Fackler writes: > On Thu, Feb 02, 2017 at 11:56:29PM -0800, Gregory Szorc wrote: >> # HG changeset patch >> # User Gregory Szorc >> # Date 1486108530 28800 >> # Thu Feb 02 23:55:30 2017 -0800 >> # Node ID 5fe78521b9cb553b9a7c6bd4d96576a35b8d3517 >> # Parent abf029200e198878a4576a87e095bd8d77d9cea9 >> contrib: script to generate release notes from commit messages >> >> Per discussion on the mailing list, we want better release notes. >> >> This patch introduces a script for producing better release notes. > > Did you look at adapting towncrier from hawkowl on github at all? I'm > sure it's git-only now, but maybe that's easy to adapt? > > (I suspect you did look, but wanted to be sure. That said, this ain't > much code...) > > Overall, I think I'm pretty happy with this approach, though since I'm > the one that brought it up I guess that's unsurprising. How does this > handle the file being copy-edited mid-stream? That is, if I copy-edit > one of the release note lines, will it get clobbered by future runs of > the tool? > > As for who and when, if we don't expect much copyediting before > release time, maybe the thing to do is to just run this as part of the > release process and do the copyediting then? Or if we do, we could > just have a bot do it on mercurial-scm.org as part of the accept > process tooling. We could also do some kind of hybrid approach of committing a file (or files) that apply a patch (or some kind of transformation) to the commit message. Just thinking out loud here. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] contrib: script to generate release notes from commit messages
On Thu, Feb 02, 2017 at 11:56:29PM -0800, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc
> # Date 1486108530 28800
> # Thu Feb 02 23:55:30 2017 -0800
> # Node ID 5fe78521b9cb553b9a7c6bd4d96576a35b8d3517
> # Parent abf029200e198878a4576a87e095bd8d77d9cea9
> contrib: script to generate release notes from commit messages
>
> Per discussion on the mailing list, we want better release notes.
>
> This patch introduces a script for producing better release notes.
Did you look at adapting towncrier from hawkowl on github at all? I'm
sure it's git-only now, but maybe that's easy to adapt?
(I suspect you did look, but wanted to be sure. That said, this ain't
much code...)
Overall, I think I'm pretty happy with this approach, though since I'm
the one that brought it up I guess that's unsurprising. How does this
handle the file being copy-edited mid-stream? That is, if I copy-edit
one of the release note lines, will it get clobbered by future runs of
the tool?
As for who and when, if we don't expect much copyediting before
release time, maybe the thing to do is to just run this as part of the
release process and do the copyediting then? Or if we do, we could
just have a bot do it on mercurial-scm.org as part of the accept
process tooling.
> diff --git a/contrib/generate-release-notes b/contrib/generate-release-notes
> new file mode 100755
> --- /dev/null
> +++ b/contrib/generate-release-notes
> @@ -0,0 +1,270 @@
> +
> +def unpublishedcommitmessages():
> +"""Obtain commit messages for unpublished commits."""
Is it the intent that this only scrapes the log messages of non-public
commits? I feel like maybe the approach should instead be to record a
last-run version in the .rst file, and then we process everything in
the repo that's a descendant of that or something?
> +
> +env = dict(os.environ)
> +env['HGMODULEPOLICY'] = 'py'
> +
> +sep = str(uuid.uuid4())
> +
> +args = [
> +'hg',
> +'log',
> +'-r', '::. and not public()',
> +'-T', '{desc}%s' % sep
> +]
> +
> +p = subprocess.Popen(args, stdout=subprocess.PIPE,
> stderr=subprocess.PIPE,
> + cwd=SRCDIR, env=env)
> +out, err = p.communicate()
> +res = p.wait()
> +if res:
> +raise Exception('error running `hg log`')
> +
> +return out.split(sep)[:-1]
> +
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] contrib: script to generate release notes from commit messages
Gregory Szorc writes: > On Fri, Feb 3, 2017 at 12:40 AM, Denis Laxalde > wrote: > >> Gregory Szorc a écrit : >> >>> # HG changeset patch >>> # User Gregory Szorc >>> # Date 1486108530 28800 >>> # Thu Feb 02 23:55:30 2017 -0800 >>> # Node ID 5fe78521b9cb553b9a7c6bd4d96576a35b8d3517 >>> # Parent abf029200e198878a4576a87e095bd8d77d9cea9 >>> contrib: script to generate release notes from commit messages >>> >> >> [snip] >> >> >> diff --git a/tests/test-generate-release-notes.t >>> b/tests/test-generate-release-notes.t >>> new file mode 100644 >>> --- /dev/null >>> +++ b/tests/test-generate-release-notes.t >>> @@ -0,0 +1,145 @@ >>> +Create a fake repo with a relnotes directory and commits >>> + >>> + $ hg init repo0 >>> + $ export FAKESRCDIR=$TESTTMP/repo0 >>> + $ cd repo0 >>> + $ mkdir relnotes >>> + $ touch relnotes/4.1.rst >>> + $ hg commit -A -m 'add relnotes 4.1' >>> + adding relnotes/4.1.rst >>> + >>> +4.1.rst should be used for relnotes if only available file >>> + >>> + $ $TESTDIR/../contrib/generate-release-notes >>> + updating $TESTTMP/repo0/relnotes/4.1.rst with content from 1 >>> changesets >>> + >>> +4.1.1.rst is used over 4.1.rst >>> + >>> + $ touch relnotes/4.1.1.rst >>> + $ hg commit -A -m 'add relnotes 4.1.1' >>> + adding relnotes/4.1.1.rst >>> + $ $TESTDIR/../contrib/generate-release-notes >>> + updating $TESTTMP/repo0/relnotes/4.1.1.rst with content from 2 >>> changesets >>> + >>> +4.2 is used over 4.1.1 >>> + >>> + $ touch relnotes/4.2.rst >>> + $ hg commit -A -m 'add relnotes 4.2' >>> + adding relnotes/4.2.rst >>> + $ $TESTDIR/../contrib/generate-release-notes >>> + updating $TESTTMP/repo0/relnotes/4.2.rst with content from 3 >>> changesets >>> + >>> +A fix with a single line is documented with .. fix:: >>> + >>> + $ touch fix1 >>> + $ hg commit -A -l - << EOF >>> + > summary line >>> + > >>> + > .. fix:: >>> + > >>> + > this is a simple fix with a single line >>> + > EOF >>> >> >> I looks strange to have an rst block body that is not indented. Typically >> this is not valid rst as the "fix" directive has no content block here. So >> maybe we should consider indentation? That would also make it possible to >> have more text after a directive like: >> >> .. fix:: >> >> this is a simple fix with a single line >> >> more content not to be part of the release notes. >> > > Yes, that is a good suggestion. It also makes the parser simpler since you > can just look for indented lines. > > I'd still like positive feedback from others before I finish this patch: I > don't want to sink more time in it before there is buy-in to this approach. I like the idea of using mailing list (as a review of text) + commit message but as Sid mentioned, we will still need to copy/edit. It's inevitable since we're human. So, as a source of text, the commit message makes sense. But how will we solve copy editing? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] contrib: script to generate release notes from commit messages
On Fri, Feb 3, 2017 at 12:40 AM, Denis Laxalde wrote: > Gregory Szorc a écrit : > >> # HG changeset patch >> # User Gregory Szorc >> # Date 1486108530 28800 >> # Thu Feb 02 23:55:30 2017 -0800 >> # Node ID 5fe78521b9cb553b9a7c6bd4d96576a35b8d3517 >> # Parent abf029200e198878a4576a87e095bd8d77d9cea9 >> contrib: script to generate release notes from commit messages >> > > [snip] > > > diff --git a/tests/test-generate-release-notes.t >> b/tests/test-generate-release-notes.t >> new file mode 100644 >> --- /dev/null >> +++ b/tests/test-generate-release-notes.t >> @@ -0,0 +1,145 @@ >> +Create a fake repo with a relnotes directory and commits >> + >> + $ hg init repo0 >> + $ export FAKESRCDIR=$TESTTMP/repo0 >> + $ cd repo0 >> + $ mkdir relnotes >> + $ touch relnotes/4.1.rst >> + $ hg commit -A -m 'add relnotes 4.1' >> + adding relnotes/4.1.rst >> + >> +4.1.rst should be used for relnotes if only available file >> + >> + $ $TESTDIR/../contrib/generate-release-notes >> + updating $TESTTMP/repo0/relnotes/4.1.rst with content from 1 >> changesets >> + >> +4.1.1.rst is used over 4.1.rst >> + >> + $ touch relnotes/4.1.1.rst >> + $ hg commit -A -m 'add relnotes 4.1.1' >> + adding relnotes/4.1.1.rst >> + $ $TESTDIR/../contrib/generate-release-notes >> + updating $TESTTMP/repo0/relnotes/4.1.1.rst with content from 2 >> changesets >> + >> +4.2 is used over 4.1.1 >> + >> + $ touch relnotes/4.2.rst >> + $ hg commit -A -m 'add relnotes 4.2' >> + adding relnotes/4.2.rst >> + $ $TESTDIR/../contrib/generate-release-notes >> + updating $TESTTMP/repo0/relnotes/4.2.rst with content from 3 >> changesets >> + >> +A fix with a single line is documented with .. fix:: >> + >> + $ touch fix1 >> + $ hg commit -A -l - << EOF >> + > summary line >> + > >> + > .. fix:: >> + > >> + > this is a simple fix with a single line >> + > EOF >> > > I looks strange to have an rst block body that is not indented. Typically > this is not valid rst as the "fix" directive has no content block here. So > maybe we should consider indentation? That would also make it possible to > have more text after a directive like: > > .. fix:: > > this is a simple fix with a single line > > more content not to be part of the release notes. > Yes, that is a good suggestion. It also makes the parser simpler since you can just look for indented lines. I'd still like positive feedback from others before I finish this patch: I don't want to sink more time in it before there is buy-in to this approach. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] contrib: script to generate release notes from commit messages
Gregory Szorc a écrit : # HG changeset patch # User Gregory Szorc # Date 1486108530 28800 # Thu Feb 02 23:55:30 2017 -0800 # Node ID 5fe78521b9cb553b9a7c6bd4d96576a35b8d3517 # Parent abf029200e198878a4576a87e095bd8d77d9cea9 contrib: script to generate release notes from commit messages [snip] diff --git a/tests/test-generate-release-notes.t b/tests/test-generate-release-notes.t new file mode 100644 --- /dev/null +++ b/tests/test-generate-release-notes.t @@ -0,0 +1,145 @@ +Create a fake repo with a relnotes directory and commits + + $ hg init repo0 + $ export FAKESRCDIR=$TESTTMP/repo0 + $ cd repo0 + $ mkdir relnotes + $ touch relnotes/4.1.rst + $ hg commit -A -m 'add relnotes 4.1' + adding relnotes/4.1.rst + +4.1.rst should be used for relnotes if only available file + + $ $TESTDIR/../contrib/generate-release-notes + updating $TESTTMP/repo0/relnotes/4.1.rst with content from 1 changesets + +4.1.1.rst is used over 4.1.rst + + $ touch relnotes/4.1.1.rst + $ hg commit -A -m 'add relnotes 4.1.1' + adding relnotes/4.1.1.rst + $ $TESTDIR/../contrib/generate-release-notes + updating $TESTTMP/repo0/relnotes/4.1.1.rst with content from 2 changesets + +4.2 is used over 4.1.1 + + $ touch relnotes/4.2.rst + $ hg commit -A -m 'add relnotes 4.2' + adding relnotes/4.2.rst + $ $TESTDIR/../contrib/generate-release-notes + updating $TESTTMP/repo0/relnotes/4.2.rst with content from 3 changesets + +A fix with a single line is documented with .. fix:: + + $ touch fix1 + $ hg commit -A -l - << EOF + > summary line + > + > .. fix:: + > + > this is a simple fix with a single line + > EOF I looks strange to have an rst block body that is not indented. Typically this is not valid rst as the "fix" directive has no content block here. So maybe we should consider indentation? That would also make it possible to have more text after a directive like: .. fix:: this is a simple fix with a single line more content not to be part of the release notes. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH RFC] contrib: script to generate release notes from commit messages
# HG changeset patch
# User Gregory Szorc
# Date 1486108530 28800
# Thu Feb 02 23:55:30 2017 -0800
# Node ID 5fe78521b9cb553b9a7c6bd4d96576a35b8d3517
# Parent abf029200e198878a4576a87e095bd8d77d9cea9
contrib: script to generate release notes from commit messages
Per discussion on the mailing list, we want better release notes.
This patch introduces a script for producing better release notes.
The script looks for ".rst" files in a "relnotes" directory.
These files contain formatted release notes which are comprised of
bullet points and sub-sections in various categories.
The script reads commit messages of non-public changesets and
extracts release notes within. These release notes are added to
the latest release notes file found in the "relnotes" directory.
The general idea is that patch authors declare release notes in
commit messages. When a patch is queued (or published), its commit
message will be parsed and the appropriate release notes file will be
updated as appropriate.
This patch is incomplete. Known deficiencies and open issues include:
* Parsing of multiple line notes does not work
* Parsing and formatting of sub-sections is not implemented
* Are sub-sections needed (I think they are because major features
deserve more than a bullet point)
* The syntax for declaring release notes should likely be bikeshedded
* Bikeshed the sections, their titles, and their identifiers.
* Need moar tests
* Figure out how/when the script is executed. Perhaps a merge driver?
diff --git a/contrib/generate-release-notes b/contrib/generate-release-notes
new file mode 100755
--- /dev/null
+++ b/contrib/generate-release-notes
@@ -0,0 +1,270 @@
+#!/usr/bin/env python
+# generate-release-notes -- populate release notes from commit messages
+#
+# Copyright 2017 Gregory Szorc
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+"""
+Release notes files live in ``relnotes/.rst``.
+
+Release notes files have sections defined by ``SECTIONS`` below.
+
+If commit messages have lines beginning with ``.. ::``, the content
+that follows is populated in that section of the release notes.
+
+The ``feature`` section is special in that it can have a title. e.g.
+``.. feature:: this is a new feature``. This will get formatted as its
+own sub-section in the release notes.
+
+"""
+
+from __future__ import absolute_import, print_function
+
+import os
+import subprocess
+import textwrap
+import uuid
+
+# For testing.
+if 'FAKESRCDIR' in os.environ:
+SRCDIR = os.environ['FAKESRCDIR']
+else:
+SRCDIR = os.path.normpath(os.path.join(os.path.dirname(__file__), '..'))
+
+# (header, internal, has sub-sections)
+SECTIONS = [
+('New Features', 'features', True,),
+('Backwards Compatibility Changes', 'bc', False),
+('Fixes', 'fix', False),
+('Performance Improvements', 'perf', False),
+('API Changes', 'api', False),
+]
+
+def relnotespath():
+"""Obtain the path to the current relnotes file.
+
+The current relnotes file is the highest versioned file in the relnotes
+directory.
+"""
+files = os.listdir(os.path.join(SRCDIR, 'relnotes'))
+assert all(f.endswith('.rst') for f in files)
+versions = [map(int, f[:-4].split('.')) for f in files]
+
+latest = '.'.join(map(str, list(sorted(versions))[-1]))
+
+return os.path.join(SRCDIR, 'relnotes', '%s.rst' % latest)
+
+def unpublishedcommitmessages():
+"""Obtain commit messages for unpublished commits."""
+
+env = dict(os.environ)
+env['HGMODULEPOLICY'] = 'py'
+
+sep = str(uuid.uuid4())
+
+args = [
+'hg',
+'log',
+'-r', '::. and not public()',
+'-T', '{desc}%s' % sep
+]
+
+p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+ cwd=SRCDIR, env=env)
+out, err = p.communicate()
+res = p.wait()
+if res:
+raise Exception('error running `hg log`')
+
+return out.split(sep)[:-1]
+
+def parsenotes(path):
+notes = {}
+
+for header, section, hassub in SECTIONS:
+notes[section] = []
+
+with open(path, 'rb') as fh:
+lines = [l.rstrip() for l in fh]
+
+i = 0
+section = None
+subsection = None
+itemlines = []
+while i < len(lines):
+line = lines[i]
+
+if not line:
+# Flush existing entry.
+if itemlines:
+notes[section].append(itemlines)
+itemlines = []
+i += 1
+continue
+
+# Section header.
+if i + 1 < len(lines) and lines[i + 1].startswith('='):
+section = None
+subsection = None
+for header, sec, hassub in SECTIONS:
+if line == header:
+section = sec
+
+if not section:
+raise Exception('Unknown section in %s: %s' % (path, line))
+
+i += 2
+continue
