Digging up this old thread as any offer to help really deserves a response. I
had travelled to help dear old mom these two weeks and didn't get back to you
after I had a bit more bandwidth.
> On Sep 18, 2021, at 9:31 AM, Jenkins, Rodney J (Rod)
> wrote:
>
> David,
>
> Thank you for the insights and explanation!
>
> I completely understand the technical debt and the challenge of making this
> better during a release. I would like to jump in and see where I can help.
> My problem is I am not a java developer. What I am good it is automating
> tasks, if I can be taught to execute them.
>
> The big ask is: Would anyone want to take the time with me to educate me on
> what has to happen for a release (not during a release)? I am thinking that
> we could set up a dummy repo that has some simple small java code in it to be
> a dummy TomEE release candidate. Create some dummy destinations that mimic
> where the artifacts must be placed. Once I understand the process, I can see
> about making it repeatable.
>
> Personally, I would like to see it done in a way that someone with lesser
> skills (like, but not necessarily, me) does releases. The way I see it now
> is the heavy hitters do the releases. I think their time would be better
> spent on the technical debt, bugs, etc. Maybe we could find a small few that
> would be wiling the be release specialists. I know some where I work that
> MAY be interested. If someone would teach me, I would teach them.
You have the right spirit. In the early days of Geronimo I did the majority of
releases and while I did a good job, wasn't the job I wanted to be doing and
could see how me doing all the releases created a knowledge vacuum that
actually hurts the community.
What I did was suggest we start a system where each release had a pilot and a
copilot. The pilot would be an experienced person who know what they were
doing, the copilot would be someone learning. Next release the copilot would
become the pilot and do the release and a new person would become the copilot.
Each release the documentation got a bit better, the technical debt paid down a
bit. Things improved, releases got more frequent and quality went up.
A significant blocker, however, is that a large number of the steps can't be
done unless you're an Apache committer. Not because of policy, just because
they require access to systems only committers can reach. You won't be able to
run a lot of the commands.
That's not a show stopper, it just means it our creativity in how to leverage
you will be heavily challenged. I'm up for it if you are.
I have been trying to revitalize our release tools in java for automating as
many release tasks as possible. I do think that's the right way to go as the
majority of people who do releases are java developers. In the past I've
written elaborate scripts in bash and inevitably they decay as I'd be the only
one who understood them. That doesn't necessarily mean there's no role for
scripting.
One of the hardest parts of doing releases is remembering all the steps that
have to be done and double checking they were done correctly. This is
absolutely something you could script up and run. I will say in all my
attempts to document release processes, I've noticed even in the best
circumstances they're always a little out of date. Turn your back even a
little and they become just out of date enough no one even looks at the doc
anymore, than it's game over.
If we had a release auditing script that could be pretty amazing. It could be
the checklist people use to "see" the release process as a whole. Again I'm
imagining something like the System V startup output where there's one line per
task. Each line is a step that has to be done a green/red colored status on
the right.
Here are some things I frequently see done wrong and not noticed:
- Forgetting to update the keys file. Topic of the original thread. Because
it's something you only need to do once in a while, it's easy to forget.
There's a release tools command to help people do it, but you have to remember
you rotated your key a few months ago and need to run it. Perfect thing to
audit.
- Missing signatures once published. We've had infrastructure ask us to add
signatures because we've forgotten a few times. There's already a
script/command to do it, but nothing to check if people use them. As
mentioned, people stop using scripts they find confusing, so they decide to
wing it instead. A script to audit this part would be great. Apache infra has
such a script and does run it every few months, but it's not ideal when they're
the ones point it out.
- Leaving SNAPSHOT or old TomEE dependencies in the examples or build. I
spent three days cleaning out references in the examples to older TomEE 7 and 8
snapshots missed by past releases. The way you check is delete your
~/.m2/repository directory and build the release tag
