Hi,People can have opinions, however they can not have their own facts. If people say things in public about the past of Quagga which are blatantly (and surely wilfully) misleading I will provide facts. To that end:
- The email based process was _OpenSourceRouting's_, _never_ my preferred way of working. See quoted email below: "Ultimately, I think email is good for discussions, but terrible for holding state. I would prefer we use git to track state." Bugzilla bug or RFE as a canonical place-mark/tracking-point for a contribution (there are a number of tools to integrate bugzilla with git), or GitLab, is what I was advocating. - I tried my damnedest to get certain people to agree to reforms of governance (again, see email below - just one of many), but they would never engage. They just rejected proposals outright, with minimal discussion, (e.g. the below and attached) which surely were reasonable places to /start/ a discussion. (And which later actions of theirs would appear to confirm were not at all unreasonable concerns).I tried, for at least 1.5 years, to work with people who were quibbling about things, to find /some/ kind of reasonable solution. I did my best to listen, to propose reasonable solutions. They _refused_ to engage productively.
Then I was given ultimatums.Including on silly stuff like use of Google Hangouts (Google, whatever one's own preference, is problematic for certain people; some because of principle, others because of location, as GOOG properties are blocked in fairly non-trivial parts of the world, including ones I sometimes spend non-trivial chunks of time in), or real-time conf. calls for final decision making (having experience of working in environments where one must schedule calls with various people across the world, real-time calendar scheduling is a very difficult task, which doesn't scale up - and theory says so too) - not even on these things would people budge.
When pretty much all the public rationales for motivation are counter-factual or at odds with the later actions of people, one must ask what rationales could possibly be left.
Trying to pin the problems of: - Low activity - Unfit for purpose method of tracking contributions - Blockage of changes to governance on Quagga.net is _dishonest_, when these were issues (to the extent they were issues) which _OpenSourceRouting_ - who are _no longer with Quagga.net_ - were responsible for. regards, -- Paul Jakma | [email protected] | @pjakma | Key ID: 0xD86BF79464A2FF6A Fortune: The Law of the Letter: The best way to inspire fresh thoughts is to seal the envelope. ---------- Forwarded message ---------- Date: Tue, 20 Jan 2015 17:54:03 +0000 (GMT) From: Paul Jakma <[email protected]> To: Martin Winter <[email protected]> Subject: processes Hi, So, to pick up the processes thread, interrupted by my fall and break(s).Obviously, the issue is finding the right state machine and accompanying workflow to act as the process for submitting, tracking, reviewing and accepting contributions. On the one hand, that state machine has to fit the software engineering and social goals and needs of the project. On the other hand, that state machine has fit in to a workflow constrained by the tools available, to come up with something workable. Finally, we want that workflow to be as low overhead as possible. We need a goldilocks level of process, not too much, not too little but just right. The little process we have today is probably no longer sufficient for the number of people and interests.
What's the abstract flow for contributions to Quagga?
1. There is the general universe of patches to Quagga
2. Some of these are offered for inclusion, or come to the attention of
the maintainers and thus need review
3. Some have had review but have concerns that need to be addressed.
3a. The most important concerns are often the most abstract: does the
contribution push Quagga in the right direction in terms of
architecture, does it improve or at least maintain the cohesiveness of
design, etc.?
This level of review is about the fundamentals of the contribution. It
can be quite subjective, and it may be hard for a submitter to address
concerns without major reworking. As it is at an abstract level, in
the ideal case the contributor would go through this review /before/
doing the bulk of detailed coding work. Potentially this review could
be sought /before/ doing any coding.
3b. Other concerns can be at the implementation level. Nits in the code or
arrangement of files, lack of documentation, formatting, or
presentation of the contribution, etc. The contributor should address
these.
4. Next, at least some people will want some level of testing of the
contribution. Unit tests, conformance tests, etc.
So there seems to be obvious states and general process here for proposed
changes:
* Pending proposals * Review states: ** Architecture review ** Implementation/change review * Accepted: ** Testing ** MatureI've done a diagramme, see abstract-workflow.png (original source is the .dia). That's pretty much the rough process we go through, but we're not very clear about the states, and we don't give clear feedback about state transitions.
The "Timeout" transition is particularly vague today.It seems to me we could easily be more transparent and explicit about our process. This would help contributors, and it help the maintainers.
E.g. at present it's very possible that a contributor is fuming, waiting for someone to act on his patch, while it's completely off the radar of the maintainers. Being explicit about timeouts would at least clear that up for everyone.
Further, there's an opportunity to involve the community more clearly and explicitly in the architectural review stuff. We can create an explicit "Review Committee" and have it be fairly open. That's an easy way to make it explicitly clear to community members that they're involved.
Next, it would be good to be explicit about the results of reviews. E.g. if one reviewer things something is good and another not. I think we need explicit votes, at least for anything contentious. This would avoid the problem of unclear resolutions, where a proposal hasn't been rejected by everyone and hasn't been accepted by everyone either and it just unclear what the status is. There's been a couple of cases recently, e.g.
So I would propose we start a "Quagga Architecture Review Committee". See next section.
On the testing side, I think Quagga.net master should be "Testing" portion of the above, not the mature portion. I think everyone is agreed that really.
Architecture Review Committee: --------------------I think we need a "Quagga Architecture Review Committee" (QARC) to review things. The rough, high-level flow would be as in the attached architecture-review-detail.png diagramme (with .dia original source).
Firstly, all new proposed changes *must* go through architecture review.For simple stuff, this could be a fast-track "No architecture concerns" approval. This should be a quick & lightweight process, and probably the most commonly used.
For full reviews we might demand the submitter provide some level of discussion of the architectural concerns and why their proposal is best.
Submitters could indicate in changes whether they think they need a full review or are suitable for fast-track. With either the justification for the fast-track, or else the full set of materials to back up why their proposal is the right architecture. Other pending changes could be triaged by Quagga maintainers for this otherwise.
Fast-track proposals can be kicked to full review if needs be.Note that the later implentation review can effectively kick any proposal back to full review by raising major concerns. Including any proposal that had previously had full review waived and had a fast-track architecture review approval.
Full review (or implementation review) can result in major concerns being raised, which would require the submitter to address them to the reviewers satisfaction.
Major concerns kick a proposal back to pending, where it will have to start the process again. A reject would pretty final for the proposal, in that form.
The architecture review need not be done by maintainers. We could have a committee from the wider community - the "ARC". Anyone interested could ask to join this committee, and be accepted by a vote of the ARC, say.
Fast-track approvals could be given by any ARC member, as long as they are not associated with the submitter. Full ARC reviews might require a majority (simple or super), with fewer than X NACKs, say.
This process is loosely based on the system used at Sun, which I thought worked quite well. Having a defined ARC would be an easy way to get wider community involvement and help ensure there is broad consensus on features - rather than having to have the maintainers "sense" the wind. Everyone has an opinion on architecture, and being on the ARC would give some recognition to people. Being on the ARC wouldn't mean having to commit to time-consuming code reviews, or mean that we have to open final commit access to a wide set of people.
A big benefit is that the ARC review does *not* require code. It would just be a text document. This would allow us to encourage people to get ideas reviewed *before* they spend lots of time coding, and potentially save wasted effort. They can have the assurance of a Full approval before they start implementing. It would be *really* good to encourage people to avail of this before starting big projects, to help avoid integration issues, even disappointment later.
I have attached a rough, initial, probably incomplete "charter" for this QARC, see quagga-arc-charter.txt which sets out the rules, and a suggested template that contributors might use to submit proposals for "full reviews", see arc-template.txt.
Fast-tracks wouldn't need such a template.Of course, such documents might need further evolution. Probably best to get a QARC running and let it do that though.
Implementation review: ----------------------This would be concerned with the nit-picky details for a proposed git commit. Code issues, commit messages, etc.
This would basically be the review process we have now, where anyone can raise issues on list, and where final approval comes from the maintainers. So the "Iplmentation review commmittee" here I guess consists of "public" and "core" members. Anyone can give advisory reviews, and the "IRcore" / "maintainers" can make binding decisions.
A changeset could be kicked back to ARC by raising major concerns. This would generally be for changes implementation review uncovers issues that mean a full ARC review is required. Typically this would be for:
* Fast-track approved change with some issue that wasn't realised or disclosed in the initial submission. * Full-review approved change, where the implementation review uncovers major issues that had not been disclosed to or realised by the full ARC reviewIn very rare cases an implementation review might raise a major concern for an issue that had already been explicitly considered and approved by the ARC. This would put the implementation reviewers at odds with the ARC, which would be an unfortunate situation. This could be avoided by having those reviewers sit on the ARC, and the ARC having veto votes.
In the long-run, it would be nice to formalise the implementation review process a bit more so that we could get it down to a checklist. Again, at some point we could open aspects of this to a wider community "committee".
I.e. to start off the "Implementation RC" is basically the maintainers working to ad-hoc-ish processes, as per today, but we should try formalise that too in time (as formal as needs be, but no more!).
Acceptance: -----------When things have both ARC and IRcore approval they can be accepted. Changes then need testing. Testing can be some combination of unit tests, conformance tests, test networks, availability for public testing, etc. After some combination of these (e.g. "No problem reports in X time"), stuff may be viewed as "mature".
I would suggest Quagga.net master should correspond to the "Testing" state. I think the "Mature" state is, largely, not something that Quagga.net can usefully specify, as different people can have very different requirements for this.
Tools: ------In terms of the tools, we have git, email and patchwork. Patchwork can track diffs in some emails and assign/remember states for the diff/contribution, based on special email tags, if people remember. Git can also maintain states for diffs, by using different branches. Patchwork has a web interface and a terminal client, but you can only submit patches or add reviews/comments via the mailing list.
Ultimately, I think email is good for discussions, but terrible for holding state. I would prefer we use git to track state.
E.g. I think we should use git branches to track incoming patches. So the "Pending" state should be a "pending/" namespace in git, not a list email in patchwork. It should require a positive act of triage for a patch to end up there, so that it is explicit to everyone, including the contributor whether or not the patch has been picked up.
We might also need some kind of "rejected/" namespace in git - but it needs a better name. ;)
For proposals and QARC review state, I'd prefer files and directories in a git repo.
But we can discuss tooling further. If that seems sane, the next step is to discuss this more widely. regards, -- Paul Jakma [email protected] @pjakma Key ID: 64A2FF6A Fortune: heavy, adj.: Seduced by the chocolate side of the force.
abstract-workflow.dia
Description: Binary data
architecture-review-detail.dia
Description: Binary data
# Quagga Architecture Review Committee (QARC) Charter
******************************************************
# Overview
The Quagga Architecture Review Committee (QARC) steers the technical,
architectural direction of Quagga.
All changes to the Quagga software must have QARC approval. Many proposed
changes will raise no architectural concerns, and so need no more than a
cursory "fast track" review and approval from a QARC member. The remaining
changes will require a "full" review by QARC, and a vote of approval.
Membership of the QARC is open to all those in the Quagga development
community. Anyone wishing to sit on the QARC is invited to introduce
themselves to a QARC member and ask to be considered.
QARC discussions are conducted in public. All Quagga community members may
participate in these discussions, in addition to QARC members. However,
non-QARC-members should refrain from voting.
# Members of the QARC {#members}
## Active members
* [Paul Jakma](mailto:[email protected])
* [David Lamparter](mailto:[email protected])
## Alumni
# Functions of the QARC
The QARC exists to review and approve all changes to Quagga for
architectural consistency and soundness. The QARC develops the processes
and documentation needed, as required, and assist those contributors seeking
a review.
QARC reviews and approval processes fall into two categories:
* Fast-track
* Full
Pending proposals to the QARC are triaged by a QARC member and directed to
the correct process. To aid this, contributors are encouraged to indicate
which process they think is suitable, ideally using the provided templates.
Votes are conducted as described in
## Fast-track review
The fast-track review is a very light-weight process to allow bug-fixes and
other clearly minimal-architectural impact proposed commits to Quagga to be
quickly approved with the minimum of intervention.
Any QARC member may assign a new proposal fast-track status, if they do not see
any architectural concerns. Any other QARC member may reassign such a
proposal from fast-track to full review.
A new proposal is assigned to fast-track review by editing the
qarc/pending/fastrack-review.txt file, and adding a 1 line description of
the case to the file. Suggested suitable format is the git log --abbrev-commit
--pretty=oneline outfor.
At intervals, the fasttrack-review.txt file should be moved to a
fasttrack-voting.txt file, and a motion put to the QARC to approve the
qarc/pending/fastrack-voting.txt file. The motion should include the
contents of the file, and the voting deadline should be at least 7 days.
The motion is carried if no there are no opposing votes. In which case the
fastrack-voting file should be moved to qarc/cases/Y/fastrack-YMD.txt, where
YMD are the year, month and day of the vote deadline, as per the Unix date
'%Y%m%d' format.
Any QARC member may change the status of a case from fasttrack to full. If
the case is still in fasttrack-review.txt, then the line for the case can
simply be removed from that file. If the case is is in
fasttrack-voting.txt, then the QARC member should vote against the motion,
noting the case(s) objected to.
In either case, the QARC member who causes a case to change from fastrtrack
to full review must ensure the contributors are informed to ensure the
contributors know they are required to prepare a full review.
## Full review
The Full review approval process is more thorough.
Changes to the following areas, in particular, would very likely raise
architectural concerns and so require full-review:
* Quagga Library APIs
* The ZServ protocol
* User visible UI, e.g. configuration or monitoring.
* Interactions between different daemons or sub-systems in Quagga
* The high-level arrangement of functionality within Quagga
The contributor must prepare a review document using the
qarc/templates/full-review-template.txt template. This must be submitted to
the QARC.
A QARC member creates a full review case by creating the file
qarc/pending/full-Y'M'D'.STRING.txt where Y'M'D' is a '%Y%m%d' discussion
timeout, and STRING is an arbitrary short label to identify the case. A
timeout of at least 2 weeks is required for full reviews. The contents of
the file should be the review document. If the review document is not
available, it can be left empty or with a note, and the deadline suitably
adjusted.
The review documents of a new case should be sent to the quagga-dev list
with a "QARC Review full-Y'M'D'.STRING: " prefix in the subject. The QARC (and
any
other interested parties) should discuss the quiz with the contributor.
Once the indicated discussion timeout is reached, a motion should be put to the
QARC
to either approve the case, require modifications to the proposal, extend
the discussion or reject it.
Full review cases may be approved by a 2/3s majority of the full membership if
there are no more than 2 members opposed. Full review cases may be rejected
by the votes of 3 or more members. Requiring modifications may be approved
by a majority of the members. Extending the discussion may be approved by a
majority of the responding members.
Approved cases should be moved from qarc/pending/full-Y'M'D'.STRING.txt to
qarc/cases/Y/full-Y'M'D'.STRING.txt, where YMD is the '%Y%m%d' of the voting
deadline on the motion that approved the case.
Cases where discussion is extended should have the
qarc/pending/full-Y'M'D'.STRING.txt suitably renamed, and be redistributed.
In cases where the contributor is asked to make updates, the contributor
must be informed, and the qarc/pending/full-Y'M'D'.STRING.txt file
removed.
## Housekeeping
The following tasks must be carried out:
* Pending requests to the QARC must be triaged, where those requests do not
already have an interested member shepherding them.
* Votes on motions should be tallied once the voting deadline has been
reached, should the mover fail to.
* The fasttrack files must be sent out and rotated appropriately regularly.
* Any stale orphaned cases in the pending directory should be dealt with or
purged regularly.
The QARC may assign a secretary to carry out these tasks.
# Changes to this charter
## Approval
This charter may be changed by a motion supported by more than half of
the active members listed in [active-members].
Additionally, the [members] sections of this charter may
be changed by a motion supported by more than half of the responding active
members, provided that a months notice of the motion is given to all active
members and that the voting is left open for at least 7 days.
## Motions and voting
Motions to be put to the QARC must be sent to the
[Quagga-dev](https://lists.quagga.net/mailman/listinfo/quagga-dev) email
list, with the subject prefixed with "QARC Motion: ", and must state a
deadline for the vote on the motion in the beginning of the email in GMT.
Movers may withdraw their motion by replying to their original motion email
and prefixing "Withdraw: " to the Subject line. The body may be left empty,
or otherwise must explicitly state the motion is withdrawn at the beginning.
Movers are assumed to support their own motion. Remaining QARC members vote
on motions by replying to the original motion email, retaining the subject
line and prefixing it with "Vote: " and then indicating their view by one of
the following means as the only word on the first line of the email:
Support:
* Yes
* Aye
* +1
* Ack
Oppose:
* No
* Nay
* -1
* Nack
Neutral:
* Neutral
* 0
Motions and ammendments should generally be discussed prior to voting. This
should be done by replying normally to the original "QARC Motion" email or
any email in a thread there-of.
The mover is required to tally the votes on their motion once the voting
deadline has
passed, and announce the result in a reply to their Motion email.
Superflous automated prefixes (e.g. "Re: ", "[quagga-dev]", etc.) may be
stripped at the sender's discretion.
## Quagga Architecture Review Committee proposal # All changes to Quagga must be approved by the Quagga Architecture Review # Committee (QARC). # # Many changes will require only a "fast track" approval. However, if your # proposed change to Quagga could raise architectural concerns, it may need # to go through a "full review". Changes that might raise architectural # concerns include: # # * Changes to the ZServ API between zebra and client daemons # * Changes to the libzebra client API (e.g. adding or removing definitions # to header files) # * Changes to the visible end-user UI or configuration formats # * Significant architectural changes of daemons # # If you need advice on whether a proposed change needs a full review or # could be fast-tracked, seek advice from a QARC member or from the public # list generally. # # A full review requires the proposer to submit a detailed proposal, e.g. # using this template, and ideally be responsive to questions from the QARC # while the QARC discusses and reviews the proposal. Ideally, the proposer # should find a QARC member who is willing to "shepherd" the proposal and # help put it to the QARC. Otherwise, email the proposal to the QARC # directly and ask for it to be considered. # # You do not need to have implemented anything to get a QARC review. Indeed, # you are *strongly encouraged* to get QARC review on any major changes # *before* doing substantial development work! # Introduction Name: Date: Affiliations: # Summary <Executive summary of the rest of the document, e.g. a paragraph or two> # Problem and/or Use-Case(s): <Detailed explanation of the problem, and/or the use-case(s) that is to be addressed> # Proposal <Detailed explanation of the proposed solution> # Impact Visible impact to other Quagga developers or users? * libzebra: * Zserv protocol: * Configuration: * User Interface: * Build system: # Issues <A detailed discussion of the issues, risks and trade-offs of this proposal, with comparison to any other possible approaches - better to cover it here than let reviewers come up with them!> # Testing <How can this be tested?> # Resources & Timeframe <How much has been done, how much is to be done? NB: Again, it is encouraged to submit a review *before* substantial development work> # References
_______________________________________________ Quagga-dev mailing list [email protected] https://lists.quagga.net/mailman/listinfo/quagga-dev
