Thanks for the clarification. That was a much longer response than I anticipated, so I think I am having trouble communicating over email.
I was referencing "https://www.apache.org/dev/release-distribution.html"; which says <snip> For every artifact distributed to the public through Apache channels, the PMC - MUST supply a valid <https://www.apache.org/dev/release-signing#verifying-signature> OpenPGP-compatible ASCII-armored detached signature <https://www.apache.org/dev/release-signing#openpgp-ascii-detach-sig> file - MUST supply at least one checksum file - SHOULD supply a SHA-256 and/or SHA-512 <https://www.apache.org/dev/release-signing#sha-checksum> checksum file - SHOULD NOT supply a MD5 or SHA-1 checksum file (because these are deprecated) </snip> Our template generates an email with the following <snip> (Append ".sha1", ".md5", or ".asc" to download the signature/hash for a given artifact.) In addition to the tarballs, and their signatures, the following checksum files will be added to the dist/release SVN area after release: accumulo-1.9.3-src.tar.gz.sha512 will contain: SHA512 (accumulo-1.9.3-src.tar.gz) =b366b89295b1835038cb242f8ad46b1d8455753a987333f0e15e3d89749540f2cd59db1bc6cf7100fc9050d3d0bc7340a3b661381549d40f2f0223d4120fd809 accumulo-1.9.3-bin.tar.gz.sha512 will contain: SHA512 (accumulo-1.9.3-bin.tar.gz) =cc909296d9bbd12e08064fccaf21e81b754c183a8264dfa2575762c76705fd0c580b50c2b224c60feaeec120bd618fba4d6176d0f53e96e1ca9da0d9e2556f1f Signing keys are available at https://www.apache.org/dist/accumulo/KEYS (Expected fingerprint: 8CC4F8A2B29C2B040F2B835D6F0CDAE700B6899D) Release notes (in progress) can be found at: https://accumulo.apache.org/release/accumulo-1.9.3/ Release testing instructions: https://accumulo.apache.org/contributor/verifying-release </snip> That last link has the following <snip> OpenPGP is an asymmetric encryption scheme which lends itself well to the globally distributed nature of Apache. Verification of a release artifact can be done using the signature and the release-maker’s public key. Hashes can be verified using the appropriate command (e.g. sha1sum, md5sum) </snip> So putting that all together, I spent time verify the sha1 and md5 when I didn't need to. I was not suggesting we add .sha512 files. Not suggesting we change Maven. The only thing I was suggesting was to make it harder for me (and others maybe) to make the mistake of checking the sha1 and md5 again. I do think it is wording on the template, but the inaccuracies in my suggestion were distracting. Maybe it is a change to https://accumulo.apache.org/contributor/verifying-release#foundation-level-requirements as well. I'll look for the template in code and make some PRs there and on the website after I give it some more thought. Mike On Tue, Apr 2, 2019 at 11:42 AM Christopher <[email protected]> wrote: > On Tue, Apr 2, 2019 at 9:32 AM Michael Wall <[email protected]> wrote: > > > > If I understand this correctly, you are saying the sha1 and md5 are > created > > by nexus? I do recall the discussion about moving to the stronger > hashes, > > so I was surprised to see sha1 and md5 still listed in the VOTE thread. > > They are actually created by one of the Maven plugins and uploaded to > Nexus. Nexus might generate its own to compare/validate them, but that > would be internal to Nexus. In any case, there is movement in the > Maven community to move these from one plugin to another (from > maven-install-plugin to maven-deploy-plugin, IIRC). Currently, Nexus > upload validation rules require .sha1 and .md5, and the Maven > dependency resolver uses these to validate downloads on the client. > These aren't just requirements for ASF's Nexus, but requirements to > sync artifacts to Maven Central also, and are core parts of the Maven2 > repository format. > > In order for Maven to discontinue these, 3 components need to > simultaneously (or coordinated-ly) update: maven-deploy-plugin, maven > dependency resolver, and Nexus. I'm sure that will happen at some > point, but it's going to take some time, and as I've said... all that > is outside our control. > > > > > When verifying a release, I want to verify the signatures are correct > and I > > want to use the asc and sha512 that will show up on the downloads page at > > https://accumulo.apache.org/downloads/. > > > > What about something like "Nexus generates an md5 and sha1 file but those > > are no longer recommended by Apache for verification of artifacts. For > > release verification use the 512 signature below" or something. > > > > Inaccuracies not withstanding (see above for clarification regarding > which component generates these), the situation is more complex than > merely not being recommended. First, it is currently not easy or > reasonable to stage checksums of better quality in Nexus at all. > Second, the release distribution policy applies to the ASF mirrors, > not Nexus. Since we're staging in Nexus, the SHA512 have been provided > directly in the email for the artifacts which will be placed on the > mirrors. Third, for all artifacts that are not going on the mirrors > (all the "convenience binary" jars that are only published to Maven), > the *only* way to verify these is with their corresponding .sha1/.md5 > (or .asc) files uploaded to Nexus. > > This current situation is complex... and it's very hard to encapsulate > all the relevant details in a vote email, but I believe we are doing > the right thing: > > 1. We are in compliance with the updated release distribution policy > (as I said, one of the first to be), > 2. We include SHA512 sums directly in our vote thread for provenance, > which almost nobody else in ASF does (b/c we don't need to: the GPG > signatures satisfy anything these could, plus some, and they satisfy > their purpose just as well detached), and > 3. We acknowledge the existence of the location of the Maven hashes, > for all artifacts stored in Maven to ensure there are no errors with > the staging area, and for other Maven-specific purposes. > > Short of explaining this over and over and over again, so that > everybody else is as familiar with all these details as I am (which I > can do, but it is time consuming, and lengthy), I don't know what we > can do better. The only thing I can think of is possibly improve the > wording in the template, but it's hard to be accurate without being > excessively verbose. When I wrote the original wording for the > template, I opted for brevity, but it seems to have only raised > questions from those unfamiliar with all the relevant complexities, > since this is the second vote thread in which we've discussed this. > I'm open to suggestions, but I think your above suggestion diverges > from accuracy a bit too far, since: > > 1) Nexus doesn't generate them, > 2) it's not the case that the relevant issue is whether they are > recommended by Apache, > 3) it's further not the case that Apache has completely discontinued > use of sha1/md5 for all purposes, since the policy update only applies > to the mirrors, not Nexus, and > 4) the *only* mandatory mechanism required by ASF that can be used to > verify a release is the cryptographic signature (GPG) > > http://www.apache.org/legal/release-policy.html#what-must-every-release-contain > ; all others are requirements of the distribution mirrors and are > fixed characteristics of the release artifacts, not something that is > itself voted on beyond the contents of the artifacts (a checksum is a > fixed characteristic, like file size; we vote on the contents of the > artifacts, not these traits); those other requirements for the mirrors > are not release vote requirements (which makes sense... we're voting > on the contents of the artifacts... not fixed, deterministic traits > like their file size or checksums). > > > > Thanks Christopher. > > > No problem. I'm happy to explain further or clarify anything, as > needed. And, if I've said something wrong, I welcome corrections. > > > > > > On Mon, Apr 1, 2019 at 10:21 PM Christopher <[email protected]> wrote: > > > > > Apologies, but I was confused because you said the template was > > > wrong... but you quoted a portion of the template which was not wrong, > > > and which I've already explained in my original response to Mike. > > > > > > Once again, those are references to files generated by the Maven > > > tooling, outside our control. Granted, we don't have to mention them > > > at all. However, the template merely acknowledges their existence. > > > Providing information for how those files are named still has value in > > > the Maven context, and is useful for any artifact downloaded from > > > Maven, not just our tarballs. > > > > > > Would there be less confusion over this if the template was a bit more > > > verbose, saying something like: > > > (Append ".asc" to download an artifact's corresponding GPG signature, > > > or ".sha1" or ".md5" to verify the checksums generated by Maven.) > > > > > > If you'd prefer this, or an alternative wording, please change it in > > > the repo... or let me know and I'll change it. > > > > > > On Mon, Apr 1, 2019 at 2:22 PM Josh Elser <[email protected]> wrote: > > > > > > > > Again, like I included earlier: > > > > > > > > > (Append ".sha1", ".md5", or ".asc" to download the signature/hash > for > > > > a given artifact.) > > > > > > > > On 4/1/19 1:56 PM, Christopher wrote: > > > > > In what way? > > > > > > > > > > On Mon, Apr 1, 2019 at 1:54 PM Josh Elser <[email protected]> > wrote: > > > > >> > > > > >> Your email template is wrong. > > > > >> > > > > >> On 4/1/19 1:33 PM, Christopher wrote: > > > > >>> Sorry, I don't understand what you mean by 'retelling of > "checksums > > > of old"'. > > > > >>> > > > > >>> On Mon, Apr 1, 2019 at 12:30 PM Josh Elser <[email protected]> > > > wrote: > > > > >>>> > > > > >>>> I think Mike's point was your VOTE template does not reflect the > > > > >>>> retelling of "checksums of old" > > > > >>>> > > > > >>>> > (Append ".sha1", ".md5", or ".asc" to download the > > > signature/hash for > > > > >>>> a given artifact.) > > > > >>>> > > > > >>>> On 3/31/19 10:54 PM, Christopher wrote: > > > > >>>>> Mike, > > > > >>>>> > > > > >>>>> We already stopped using md5 and sha1 for the release > artifacts on > > > the > > > > >>>>> mirrors. I did this some time ago, and we discussed it on list > on > > > > >>>>> previous vote threads (last year)... which resulted in me > changing > > > the > > > > >>>>> release candidate build script automated tooling to embed the > > > SHA512 > > > > >>>>> sums for the tarballs directly in the release vote message. I > even > > > > >>>>> went back and updated the downloads page for the previous > releases > > > and > > > > >>>>> updated the mirrors to be SHA512 only. Because of these steps I > > > took, > > > > >>>>> Accumulo was one of the first projects across the entire ASF > who > > > were > > > > >>>>> 100% compliant immediately after INFRA VP updated the release > > > > >>>>> distribution policy you linked. > > > > >>>>> > > > > >>>>> *This is a resolved action for Accumulo.* > > > > >>>>> > > > > >>>>> FWIW, SHA512 was also used as the hash algorithm in the GPG > > > signature > > > > >>>>> (same as every RC I've ever prepped for ASF). The only > remaining > > > md5 > > > > >>>>> and sha1 reference are Maven-specific tooling, and we have no > > > control > > > > >>>>> over that tooling. We could change the vote template to no > longer > > > > >>>>> mention them, but I don't see the point since they're still > > > relevant > > > > >>>>> within the context of Maven artifact hosting, and that's the > > > context > > > > >>>>> in which they are presented in the vote email. > > > > >>>>> > > > > >>>>> On Sun, Mar 31, 2019 at 1:59 PM Michael Wall <[email protected] > > > > > wrote: > > > > >>>>>> > > > > >>>>>> -1 for the issue with commons config > > > > >>>>>> > > > > >>>>>> I check the signatures, they are good. We should stop using > md5 > > > and sha1 > > > > >>>>>> though, see > > > https://www.apache.org/dev/release-distribution#sigs-and-sums. > > > > >>>>>> Has anyone looked at moving to sha256 and/org sha512? > > > > >>>>>> Successful run of mvn clean verify -Psunny > > > > >>>>>> > > > > >>>>>> On Sat, Mar 30, 2019 at 11:31 PM Keith Turner < > [email protected]> > > > wrote: > > > > >>>>>> > > > > >>>>>>> I completed a continuous ingest run on a 10 node cluster > running > > > > >>>>>>> Centos 7. I used the native map. I had to rebuild Accumulo > to > > > work > > > > >>>>>>> around #1065 inorder to get the verify M/R job to run. > > > > >>>>>>> > > > > >>>>>>> > > > org.apache.accumulo.test.continuous.ContinuousVerify$Counts > > > > >>>>>>> REFERENCED=34417110819 > > > > >>>>>>> UNREFERENCED=9097524 > > > > >>>>>>> > > > > >>>>>>> On Wed, Mar 27, 2019 at 7:57 PM Christopher < > [email protected]> > > > wrote: > > > > >>>>>>>> > > > > >>>>>>>> Accumulo Developers, > > > > >>>>>>>> > > > > >>>>>>>> Please consider the following candidate for Apache Accumulo > > > 1.9.3. > > > > >>>>>>>> > > > > >>>>>>>> This supersedes RC1 and contains the following change: > > > > >>>>>>>> https://github.com/apache/accumulo/pull/1057 > > > > >>>>>>>> > > > > >>>>>>>> Git Commit: > > > > >>>>>>>> 94f9782242a1f336e176c282f0f90063a21e361d > > > > >>>>>>>> Branch: > > > > >>>>>>>> 1.9.3-rc2 > > > > >>>>>>>> > > > > >>>>>>>> If this vote passes, a gpg-signed tag will be created using: > > > > >>>>>>>> git tag -f -m 'Apache Accumulo 1.9.3' -s rel/1.9.3 \ > > > > >>>>>>>> 94f9782242a1f336e176c282f0f90063a21e361d > > > > >>>>>>>> > > > > >>>>>>>> Staging repo: > > > > >>>>>>> > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077 > > > > >>>>>>>> Source (official release artifact): > > > > >>>>>>>> > > > > >>>>>>> > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077/org/apache/accumulo/accumulo/1.9.3/accumulo-1.9.3-src.tar.gz > > > > >>>>>>>> Binary: > > > > >>>>>>> > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077/org/apache/accumulo/accumulo/1.9.3/accumulo-1.9.3-bin.tar.gz > > > > >>>>>>>> (Append ".sha1", ".md5", or ".asc" to download the > > > signature/hash for > > > > >>>>>>>> a given artifact.) > > > > >>>>>>>> > > > > >>>>>>>> In addition to the tarballs, and their signatures, the > > > following checksum > > > > >>>>>>>> files will be added to the dist/release SVN area after > release: > > > > >>>>>>>> accumulo-1.9.3-src.tar.gz.sha512 will contain: > > > > >>>>>>>> SHA512 (accumulo-1.9.3-src.tar.gz) = > > > > >>>>>>>> > > > > >>>>>>> > > > > b366b89295b1835038cb242f8ad46b1d8455753a987333f0e15e3d89749540f2cd59db1bc6cf7100fc9050d3d0bc7340a3b661381549d40f2f0223d4120fd809 > > > > >>>>>>>> accumulo-1.9.3-bin.tar.gz.sha512 will contain: > > > > >>>>>>>> SHA512 (accumulo-1.9.3-bin.tar.gz) = > > > > >>>>>>>> > > > > >>>>>>> > > > > cc909296d9bbd12e08064fccaf21e81b754c183a8264dfa2575762c76705fd0c580b50c2b224c60feaeec120bd618fba4d6176d0f53e96e1ca9da0d9e2556f1f > > > > >>>>>>>> > > > > >>>>>>>> Signing keys are available at > > > https://www.apache.org/dist/accumulo/KEYS > > > > >>>>>>>> (Expected fingerprint: > 8CC4F8A2B29C2B040F2B835D6F0CDAE700B6899D) > > > > >>>>>>>> > > > > >>>>>>>> Release notes (in progress) can be found at: > > > > >>>>>>>> https://accumulo.apache.org/release/accumulo-1.9.3/ > > > > >>>>>>>> > > > > >>>>>>>> Release testing instructions: > > > > >>>>>>>> https://accumulo.apache.org/contributor/verifying-release > > > > >>>>>>>> > > > > >>>>>>>> Please vote one of: > > > > >>>>>>>> [ ] +1 - I have verified and accept... > > > > >>>>>>>> [ ] +0 - I have reservations, but not strong enough to vote > > > against... > > > > >>>>>>>> [ ] -1 - Because..., I do not accept... > > > > >>>>>>>> ... these artifacts as the 1.9.3 release of Apache Accumulo. > > > > >>>>>>>> > > > > >>>>>>>> This vote will remain open until at least Sun Mar 31 > 00:00:00 > > > UTC 2019. > > > > >>>>>>>> (Sat Mar 30 20:00:00 EDT 2019 / Sat Mar 30 17:00:00 PDT > 2019) > > > > >>>>>>>> Voting can continue after this deadline until the release > > > manager > > > > >>>>>>>> sends an email ending the vote. > > > > >>>>>>>> > > > > >>>>>>>> Thanks! > > > > >>>>>>>> > > > > >>>>>>>> P.S. Hint: download the whole staging repo with > > > > >>>>>>>> wget -erobots=off -r -l inf -np -nH \ > > > > >>>>>>>> > > > > >>>>>>> > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077/ > > > > >>>>>>>> # note the trailing slash is needed > > > > >>>>>>> > > > >
