I have been ignoring bot PRs for daffodil-vscode because I am not interested in
TypeScript/npm/VSCode extension programming, so I can't speak for anyone who
has to review those bot PRs. Speaking only of the bot PRs for the daffodil
Scala/Java codebase and a few other projects I maintain, I prefer a pace of at
most 1 or 2 outstanding bot PRs on any given day, which allows me to knock them
out of the way quickly when I turn my attention to them. If we allowed more
than say, 3 bot PRs to arrive together (which might happen if we triggered bot
PRs only 4 times a year and didn't limit their rate after triggering them), I
probably would lose interest and not want to review any bot PRs at all.
I suggest that we impose a rate limit of 1 bot PR at any given time. You would
have to fix that PR, and wait overnight (some might even prefer one week),
before the bot would issue another PR for review, and so on until all the
dependencies were up to date. If the bot issues only one PR per working day
(or week), I don't think we need to bother turning bot PRs off and turning them
back on once a quarter or when beginning a release. Slow and steady on the
update treadmill seems better than batching bot PRs and trying to catch up on a
large batch of bot PRs, even if each PR takes only a few minutes to review.
I don't think applying bot PRs without reviewing their changes is a good idea
in a project like Apache Daffodil. Doing so increases the risk for anyone
building and running Daffodil from the main branch, and also increases a
release manager's workload since the release manager would have to determine
the last dependency version reviewed and look at a batched diff of multiple
version changes since that last dependency version.
I also don't think much of consciously ignoring bot PRs because that conflicts
with our ideal goal of keeping our list of open PRs as small as possible to
help developers notice PRs that need more review or more work getting them
ready for merge.
John
-----Original Message-----
From: Mike Beckerle <mbecke...@apache.org>
Sent: Monday, July 18, 2022 1:25 PM
To: dev@daffodil.apache.org
Subject: EXT: the update treadmill - bot update implied work considered
excessive
WARNING: This email originated from outside of GE. Please validate the sender's
email address before clicking on links or attachments as they may not be safe.
(I am breaking this discussion out of a bot PR thread to discuss more
broadly.)
Steve Lawrence reminded us recently that updates need to be reviewed relative
to a checklist:
https://cwiki.apache.org/confluence/display/DAFFODIL/Scala+Steward
Given the amount of work this implies, per update, it is 100% clear to me that
the rate of these bot-driven changes that we accept must be massively cut down.
Each one requires significant examination.
What fraction of our dev effort should be spent on this update treadmill. I
would claim < 1%. That's < 2 hours per *month* of work. Frankly I think even
that much is too much.
And by that I mean not just actual work looking at the PRs, but just trying to
keep up with the email load.
I have personally setup email rules to bin all bot update emails into a
sub-folder that I plan to look at only when we do releases, so 2 or 4 times a
year. But alas, every time a dev team member responds to one of those bot
emails, they show up in my inbox again looking for a second reviewer, etc.
So I have not been successful in cutting out this noise.
But it simply must be cut down. The amount of bot-change-related email
dominates all other content on the dev list.
Given that the dev list email is the face of the project as far as
recruiting/retaining more developers, I consider this pollution of the dev list
to be a threat to our project's viability.
I think we are staring at the breaking point for the current state of the art
in what I call "flat linker" software componentry. That is, via these ongoing
attempts for everyone to try to just stay current with the latest version of
every component via update bots. This is what I refer to as the "update
treadmill". By "flat linker" I mean where the whole software system must have
only one version of any given software component. Java modules and/or OGSI or
other namespace systems (non-flat-linker) are the only long-term way out of
this in the long run (note:
https://issues.apache.org/jira/browse/DAFFODIL-2683).
In the meantime, Some ideas:
1. Turning off these update bots, except once a quarter or so, and when we
first start contemplating creating a release. One of the first steps in our
release process is to create a commit for "preparing for release xyzzy".
This would add to that step to re-enable the bots, which would result in a
flurry of update PRs to be dealt with at that time.
2. Accepting these PRs without the scrutiny of the checklist items, so long as
all the CI checks pass, and as part of the pre-release process we reverify all
the transitive dependencies and licenses as a once-a-release activity for any
updated components. Note that there is already release activity associated with
creating release-notes for all updated dependencies, so this would add on to
that work.
3. Make it a basic policy to ignore almost all bot-based update PRs, and just
leave them sitting there. We agree to consciously not respond to them and we
deal with them only if they are security alert related, or we decide
consciously as a project that it is time to work on them.
