Mark Linimon wrote:
On Fri, Jan 11, 2008 at 07:35:41PM +0100, Peter Schuller wrote:
But understandable or not, the problem becomes particularly
frustrating when it affects PR:s that contain patches. Such PR:s
constitute direct contributions to the project. In cases where such
patches are correct/good enough, the non-application of those patches
have what I believe to be significant effects.
I think you make some excellent points -- especially with how
understandable it is for someone who's submitted a patch which got
ignored, to not do the work to submit the next one.
With respect to patches, the two things that I've tried -- and they have
been insufficient -- are the following:
- weekly email of "PRs containing patches"
- weekly email of "PRs recommended by the bugbuster team"
We can make the following observations:
- the "push" paradigm doesn't work particularly well. Partly this is
due to fatigue if you try to read through all this stuff (see below).
- the "recommended" list has been slightly effective, but it's going
to take some time for it to take off. For one thing, it has been
underpublicized; for another, we don't have the culture of committers
getting "warm fuzzies" from committing PRs. (Since no one working
on that kind of stuff gets paid AFAIK, that's the only positive
feedback they're going to get.)
Last week someone made the observation that if these things were instead
updated daily and posted to a web page, that it would be far more useful.
I've taken that up as a task.
Observation: PR or 'other' - it is presently rather difficult to find
[older | most] patches- which reduces the opportunity for testing and
feedback, and increases the likelihood of having the situation reported
yet-again. And again..
so ..
- Can there be a port category (real or 'virtuaized', added - along the
lines of:
'patches and workarounds'
With caveats, of course.... but hopefully also with provision for
co-located feeback, both positive and negative.
Your other point about "latest PRs" is also a good one. That should be
relatively easy to do. I'll take up that task.
Question: Can an improved 'data mining' project add value?
One wherein the tools to look for commonality - sometimes of the less
obvious nature - would be easier for all-hands?
I include in this the ability to build 'reports' - not just do the
present ad hoc searches.
If such could be done 'better', it might also benefit from crossproject
(i.e all the *BSD's) scope.
* The committer may not be familiar with the code and, even though
the patch looks good generally, it is felt that someone familiar with
that part of the system needs to have a look.
In particular, this hurts for areas of the system that are unmaintained.
What I suggest is a system where a group of non-committers
systematically pre-process PR:s, to provide feedback and additional
quality assurance to the committer that is to process the PR.
This group of people could either be a self-proclaimed group of
volunteers, or perhaps a group of people satisfying the criteria of
"guy we kinda trust to do testing and provide a useful indication of
sanity and correctness, but not with a commit bit".
So far it hasn't happened. We've set up the freebsd-bugbusters@ mailing
list and the #freebsd-bugbusters IRC channel on EFNet (and please join
us!) and the latter is where our last 2 bugathons took place.
It's clear that there are several people who want to help process the
PRs, and we don't have a good answer for them on "how can I contribute?".
The existing tool, and social conventions, don't allow for non-committers
to change PR states. As far as we've done in the past is to grant people
"GNATS access" rights but not "commit rights", on an experimental basis.
We've done this twice, and although it has worked well, just two people
isn't enough. (One has gone on to become a full committer -- which is
great!; the other current does not have as much time for FreeBSD work).
Several hundred PRs were dealt with by these two folks, so I consider the
experiment a success.
What we used as a qualification was "track record of responding to PRs
and
questions on mailing lists", fwiw.
One possible answer to this that I have gotten in the past with
another project, is that noone is stopping me from just grabbing a
bunch of PR:s and posting follow-ups saying I've tested something, or
otherwise giving feedback.
Yes, but that's open-loop as well. It's the same situation as with
submitting
the patch in the first place: it's going to get frustrating if no
committer
picks it up and commits it.
There's not too many people so thick-skinned and persistent as to keep
going forward when no one's using their work.
For example, patches with a high confidence of correctness due to
many people affirming that they have tested it could be automatically
prioritized for committers to deal with, and there will be a clear
and systematic record of what testing was done, and by who.
Right. The "priority" field in GNATS has been so abused as to become
useless. (People assume that setting it higher will cause some kind of
magic to happen.)
My current thinking is that what committers ought to see is a metric of
correctness, and a metric of priority, _as set by someone who's vetted
the PR_. The weekly mailings are too poor an approximation of either
to be useful.
Adding the second metric would cure one problem that you don't mention --
which is that few people have the interest and patience to plow through
N-thousand PRs. It's not humanly possible to look at them all -- even
the new ones as they come in. There's simply too many. So, you create
an expectation "why bother, there's so many anyways". We need to break
that chain of expectation. A good fix is a good fix. The PR count will
never get to zero; I (with bugmaster hat on) would be thrilled if we can
get to the point of just steady-state.
DB Admins, SysAdmins, 'Management' need BSD, too and not all the work
that needs to be done requires 'C' expertise.
Some of us also have spare bandwidth and hardware as well as time, so...
When I talk about priority above, I do not mean to imply that any
committer should work on things they do not want to work on. But I
have to assume that a lot of patches get committed because a
committer decided to go and pick a few PR:s to process, rather than
the committer necessarily has a burning interest in that particular
patch.
I think most get committed because a committer sees a PR come in on the
mailing list and grabs it. Much less often do committers go through the
database looking for things to fix. Again, the lousy "search/browse"
capabilities of the existing tool let us down here.
As such, if said committer could spend those <n> minutes closing five
times as many PR:s because the up-front confidence in the correctness
of the patch is so much higher, perhaps it would help the system to
"scale", moving some of the burden off the committers.
Right. What we need is to start small and create a _culture_ of where
it's fun (or at least intellectually interesting) to work on this stuff.
I think the IRC channel may still be the best bet for this.
Once I finish fiddling around with the sparc64-6 and sparc64-7 release
builds (the first approximation is finished, but I believe we can still
add some more packages), I intend to task-switch over to looking towards
defining a PR workflow and floating some proposals. I hope to have
something concrete to present at BSDCan. Please join us on bugbusters@
to discuss any more ideas, too.
mcl
Coders will always set their own priorities. Even when working for a wage.
;-)
But all-hands benefit from greater stability and fewer 'critical' things
breaking, so rational self-interest might be more effectively applied by
each participant if the 'big picture' - in several flavors - was simply
published in some 'better' manner.
You've ID'ed some of the possibilities - there are bound to be more.
Perhaps some of the committers - or at least the project in general -
could benefit from more 'virtual office staff' - folks who specifically
are NOT coders - to remove obstacles and help leverage their expertise?
Background research, documentation, finding and enlisting testers who
can help, building and maintaining databases - *doing* tests. Some of
that is more amenable to being 'directed' than the coding efforts
themselves. Different personality types...
JMHK$2CW..
