Hello everyone,
On Sat, Sep 14, 2019 at 04:53:17PM -0400, Jason Merrill wrote:
> At Cauldron this weekend Joel offered to adjust his git hooks
> (https://github.com/brobecke/git-hooks), which are already used by gdb
> and glibc, to meet GCC's needs. Separately, Joseph volunteered to
> deal with converting the gcc-www repository to git and dealing with
> those hooks.
>
> I expect that Joel's hooks should work fine for gcc-www with minimal
> changes, but that the main GCC repository will need more work for
> bugzilla integration.
>
> The GCC SVN hooks initially look like pretty vanilla SVN hooks using
> svnmailer (http://opensource.perlig.de/svnmailer/); the customized
> part of the svnmailer.conf is just
>
> [libstdcxx]
> for_paths = .*(libstdc..-v3)/.*
> to_addr = libstdc++-...@gcc.gnu.org
>
> [java]
> for_paths = .*(boehm-gc|fastjar|gcjx|gcc/java|libffi|libjava|zlib)/.*
> to_addr = java-...@gcc.gnu.org
>
> [gccdefault]
> to_addr = gcc-...@gcc.gnu.org
> bugzilla_to_addr = gcc-bugzi...@gcc.gnu.org
>
> Pretty short...but the last line relies on Daniel's custom
> bugzilla/svnmailer integration (attached below), and it looks like
> Joel's hooks don't have anything comparable. Any thoughts about
> adjusting Daniel's bugzilla.py vs. pulling in something like
> http://www.theoldmonk.net/gitzilla/ ?
Looking at the configuration file, I believe the git-hooks
should have most, if not all, of the features that would be needed for
the GCC repository. In particular, there is already a way to relay
commits to third-party tools via calling of a script; GDB uses that
to interface with their Bugzilla database.
But before I say more I should point to the documentation which,
for historical reasons, is available on the GDB wiki rather than
the git-hooks GitHub repository. I will fix that, but in the meantime:
https://sourceware.org/gdb/wiki/GitHooksUsersGuide
I'm attaching a file called project.config, which shows the current
configuration for the GDB repository, as it is might be a good
starting point for GCC's configuration.
Of interest:
* We can see that "hooks.mailinglist" is pointing to a script.
The purpose of that script is to determine, based on which files
changed, which mailinglists should the commit email be sent to.
* There is a hooks.style_checker line. Currently, GDB points to
a script which does nothing. If GCC has some scripts they want
to be run to validate the contents of a file, this is where
this can be done. There is no mechanism, really, to say "don't
run the style_checker", but it's easy to just pass a null
style_checker as done by GDB.
* For bugzilla integration, GDB uses the hooks.file-commit-cmd
file. I'm attaching the email-to-bugzilla script, although
I don't know how useful it will be for GCC. It predates the git-hooks
and I just re-used it as is when we switched over to the git-hooks.
Given that, it seems like the git-hooks might be ready to support
all the needs of the GCC repository? We would need to:
- write a script that determines the list of recipients based
on the list of files being changed; that should be a trivial
adaptation of the script used on the GDB side;
- Adapt the script filing the commit with bugzilla
- create a refs/meta/config "branch", and add the project.config
file with the git-hooks configuration.
I can definitely help with the configuration setup phase in whatever
capacity you'd like. I would recommend people take a look at the list
of options currently available to see what kind of initial configuration
we might want to start with.
--
Joel
[hooks]
from-domain = sourceware.org
mailinglist = /git/binutils-gdb.git/hooks-bin/email_to.py
# We do not want to force a maximum line length in commit
# revision logs, as they get in the way of copy-pasting
# debugging session, error messages, logs, etc.
max-rh-line-length = 0
# Reject merge commits on a certain number of branches:
# - on master: We request that people rebase their changes
# before pushing instead (merge commits tend to confuse
# git newcommers).
# - on GDB release branches: There is a high risk that a merge
# commit is a merge from master into the branch, which would
# bring a lot more than what the user probably meant to push.
# Request that the user cherry-pick his changes.
reject-merge-commits = refs/heads/master,refs/heads/gdb-.*
# The style checker, applied to the contents of each file being
# modified.
style-checker = /git/binutils-gdb.git/hooks-bin/style_checker
# The URL where we can inspect the commit, inserted in the commit
# notification email, and also copy sent to the file-commit-cmd.
commit-url =
"https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=%(rev)s"
# Do not send emails for the following branches (unofficial
# third-party branches).
no-emails = refs/heads/users/.*
# Send a copy to bugzilla if a commit has a PR number in it.
file-commit-cmd = "/sourceware/infra/bin/email-to-bugzilla -G 'gdb
binutils'"
# The script that calls the irker (IRC notification of new
# commits).
post-receive-hook = /git/binutils-gdb.git/hooks-bin/post-receive
#! /usr/bin/env python
import sys
ML_MAP = {'bfd': 'bfd-...@sourceware.org',
'gdb': 'gdb-...@sourceware.org',
}
OWNER_MAP = (
# BFD file...
('bfd/', 'bfd'),
('binutils/', 'bfd'),
('opcode/', 'bfd'),
('cpu/', 'bfd'),
('elfcpp/', 'bfd'),
('gas/', 'bfd'),
('gold/', 'bfd'),
('gprof/', 'bfd'),
('include/', 'bfd'),
('ld/', 'bfd'),
('opcodes/', 'bfd'),
# GDB files...
('gdb/', 'gdb'),
('readline/', 'gdb'),
('sim/', 'gdb'),
)
EVERYONE = set(ML_MAP[ml_key] for ml_key in ML_MAP)
def ml_from_filename(filename):
for (path, ml_key) in OWNER_MAP:
if filename.startswith(path):
return ML_MAP[ml_key]
# Not found in map, it is a common file.
return EVERYONE
result = set()
for filename in sys.stdin:
ml = ml_from_filename(filename)
if isinstance(ml, basestring):
result.add(ml)
else:
result.update(ml)
if len(result) >= len(EVERYONE):
# We have iterated over enough entries to know already
# that we have selected all possible recipients. So
# stop now.
break
if not result:
# No files given, return EVERYONE
result = EVERYONE
print '\n'.join(sorted(result))
#!/usr/bin/perl
# -*-Perl-*-
#
# Perl filter to handle the log messages from the checkin of files in
# a directory. This script will group the lists of files by log
# message, and mail a single consolidated log message at the end of
# the commit.
#
# This file assumes a pre-commit checking program that leaves the
# names of the first and last commit directories in a temporary file.
#
# Contributed by David Hampton <hamp...@cisco.com>
#
# hacked greatly by Greg A. Woods <wo...@web.net>
#
# Then chopped down just to send bugzilla email, for git.
use POSIX;
use DBI;
#
# Configurable options
#
$TMPDIR = "/sourceware/cvs-tmp";
$BMAILER = "/usr/sbin/sendmail";
#
# Subroutines
#
sub see_if_bugzilla_bug_exists {
local ($dbh, $product, $id) = @_;
# Split $PRODUCT and SQL-ify.
my $sql_product = '';
foreach $i (split (/\s+/, $product)) {
if ($sql_product ne '') {
$sql_product .= ', ';
}
$sql_product .= "'" . $i . "'";
}
my $sth2 = $dbh->prepare ("SELECT COUNT(*) from bugs where bug_id = $id and
product_id = any (select products.id from products where name in
($sql_product))") or return 0;
$sth2->execute() or return 0;
my $count = $sth2->fetchrow_array ();
return $count > 0;
}
sub mail_bug_notification {
local($name, $subject, @text) = @_;
open(MAIL, "| $BMAILER -f\"cvs-commit\@gcc.gnu.org\" $name");
print MAIL "From: cvs-commit\@gcc.gnu.org\n";
print MAIL "Subject: $subject\n";
print MAIL "To: $name\n";
print MAIL "Content-Type: text/plain; charset=UTF-8\n";
print MAIL "\n";
print MAIL join("\n", @text), "\n";
close(MAIL);
}
#
# Main Body
#
# Initialize basic variables
#
$debug = 0;
chop($hostname = `hostname`);
# Parse command line arguments.
while (@ARGV) {
$arg = shift @ARGV;
if ($arg eq '-d') {
$debug = 1;
print STDERR "Debug turned on...\n";
} elsif ($arg eq '-G') {
($bugzillaproduct) && die("Too many '-G' args\n");
$bugzillaproduct = shift @ARGV;
}
}
if ($hostname !~ /\./) {
chop($domainname = `domainname`);
$hostdomain = $hostname . "." . $domainname;
} else {
$hostdomain = $hostname;
}
# Used with sprintf to form name of Gnats notification mailing list.
# %s argument comse from -G option.
$GNATS_MAIL_FORMAT = "%s-bugzilla\@$hostdomain";
# Collect the body of the commit message.
binmode STDIN, ":utf8";
while (<STDIN>) {
chop;
push (@text, $_);
}
$log_txt = join ("\n", @text);
%done_ids = {};
while ($log_txt =~
m/[^Aa](?:bug|PR|BZ)\s+\#?\s*(?:[a-z+-]+\/)?(?:\/)?(\d+)(.*)$/si) {
$bug_id = $1;
$log_txt = $2;
if (!defined $done_ids{$bug_id})
{
$done_ids{$bug_id} = 1;
# Send mail to Bugzilla, if required.
if ($bugzillaproduct ne '') {
my $dbh = undef;
if ($bugzillaproduct eq 'gcc')
{
$dbh = DBI->connect ("dbi:mysql:bugs", "swbugz", "everythingroses");
}
else # elsif ($bugzillaproduct eq 'glibc')
{
$dbh = DBI->connect ("dbi:mysql:sourcesbugs", "swbugz",
"everythingroses");
}
if ($debug)
{
print STDERR "Attempting to see if bug $bug_id exists\n";
}
if (defined $dbh
&& &see_if_bugzilla_bug_exists ($dbh, $bugzillaproduct, $bug_id))
{
if ($debug) { print STDERR "It does\n"; }
if ($bugzillaproduct ne 'gcc') {
&mail_bug_notification( sprintf ($GNATS_MAIL_FORMAT,
"sourceware"), "[Bug $bug_id]", @text);
}
else {
&mail_bug_notification( sprintf ($GNATS_MAIL_FORMAT,
$bugzillaproduct),
"[Bug $bug_id]", @text);
}
}
if (defined $dbh)
{
$dbh->disconnect;
}
}
}
}
exit 0;
