On Tue, Mar 22, 2022 at 11:51 PM Daniel Shahaf <d...@daniel.shahaf.name> wrote: > > Mark Phippard wrote on Mon, Mar 21, 2022 at 16:46:55 -0400: > > On Mon, Mar 21, 2022 at 4:31 PM Stefan Sperling <s...@elego.de> wrote: > > > On Mon, Mar 21, 2022 at 12:44:44PM -0400, Mark Phippard wrote: > > > > Problem 1: Rolling the tarballs > > > > > > > > The process creates the tarballs but fails near the end. It looks GPG > > > > related? > > > > > > > INFO:root:Building Unix tarballs > > > > INFO:root:Moving artifacts and calculating checksums > > > > Traceback (most recent call last): > > > > File "trunk/tools/dist/release.py", line 1916, in <module> > > > > main() > > > > File "trunk/tools/dist/release.py", line 1912, in main > > > > args.func(args) > > > > File "trunk/tools/dist/release.py", line 983, in roll_tarballs > > > > download_file(KEYS, filepath, None) > > > > File "trunk/tools/dist/release.py", line 289, in download_file > > > > response = urlopen(url) > > > > File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen > > > > return opener.open(url, data, timeout) > > > > File "/usr/lib/python2.7/urllib2.py", line 435, in open > > > > response = meth(req, response) > > > > File "/usr/lib/python2.7/urllib2.py", line 548, in http_response > > > > 'http', request, response, code, msg, hdrs) > > > > File "/usr/lib/python2.7/urllib2.py", line 473, in error > > > > return self._call_chain(*args) > > > > File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain > > > > result = func(*args) > > > > File "/usr/lib/python2.7/urllib2.py", line 556, in > > > > http_error_default > > > > raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) > > > > urllib2.HTTPError: HTTP Error 404: Not Found > > > > > > It seems ASF have removed the KEYS file our script is trying to fetch. > > > See http://people.apache.org/keys/ where it says "Project group files are > > > no longer created." > > > > > > It looks like what the script wants to do here is obtain a copy of > > > the Subversion project's KEYS file and store it along with release > > > artifacts. If we want to keep doing this we will have to maintain > > > our own KEYS file on the website, I suppose. Otherwise, we could > > > decide to no longer provide such a file and remove relevant code > > > from the script. Not sure what is better. > > > > Since I do not know what this was used for, maybe someone can help get > > us to a decision and update the script? Daniel Shahaf, if you see > > this, I think of you as the resident expert on this stuff. Any thoughts? > > The design goals here are two: > > 1. When someone with commit access to N ASF projects updates their PGP > key, they shouldn't have to do O(N) work to update N KEYS files. They > should have to do either O(1) work or, ideally, nothing at all. > > 2. Releases should snapshot the keys that are current at the time they > are generated, in order to remain verifiable in archive.a.o even if the > keys in question are later removed from LDAP (by root@ as part of > a manual password reset, or by the committer via id.a.o). > > The ASF-wide "generate group keys" scripts addressed #1. They used to > generate two files, one with only the full committers' keys and one with > both full and partial committers' keys. We used the former (to reduce > attack surface) until it stopped getting generated. > > Copying the file to subversion-<version>.KEYS addressed #2.
Thanks for responding and please forgive some basic questions. My "knowledge" of this begins and ends with running gpg -ba to sign releases and copying and pasting the output to an email. It sounds like the KEYS file is a list of all of the GPG Keys for ASF committers? What is it used for during the RM process? Is it so that the RM can validate signatures that come in? Does it get included in the release somehow? > > So, what to do? > > - We could talk to some ASF-wide list (comdev, infra, site-dev@) about > resuming generation of group keys files. > > (If you do this, someone will probably ask you to comply with > a markdown document that calls itself policy. I don't believe that > document is Foundation policy, and even if it is, our practice has > been +1ed by an Officer of the Foundation.) > > - We could roll our own automation, relying on the key fingerprints in > id.a.o (LDAP) via <https://people.apache.org/keys/committer/>: > > for availid in $( > perl -anE 'say $F[0] if (/^Blanket/../END ACTIVE > FULL.*SCRIPTS LOOK FOR IT/ and /@/)' \ > < /path/to/COMMITTERS > ) ; do > echo curl -sSfO > https://people.apache.org/keys/committer/${availid}.asc || test $? -eq 22 || > return 1; > done > cat *.asc > subversion-….keys > > - We could roll our own solution without relying even on > <https://people.apache.org/keys/committer/>, since its API has already > been broken twice. Suppose we wanted to roll out a new release soon. What would you recommend? Can we just remove this step and ignore it or do we need someone to code a solution? As for the rest ... I have figured out, or nearly figured out, a solution to all of the build and test issues now. So I am all set with that. Thanks Mark
