On Tue, Apr 2, 2019 at 9:32 AM Michael Wall <[email protected]> wrote: > > If I understand this correctly, you are saying the sha1 and md5 are created > by nexus? I do recall the discussion about moving to the stronger hashes, > so I was surprised to see sha1 and md5 still listed in the VOTE thread.
They are actually created by one of the Maven plugins and uploaded to Nexus. Nexus might generate its own to compare/validate them, but that would be internal to Nexus. In any case, there is movement in the Maven community to move these from one plugin to another (from maven-install-plugin to maven-deploy-plugin, IIRC). Currently, Nexus upload validation rules require .sha1 and .md5, and the Maven dependency resolver uses these to validate downloads on the client. These aren't just requirements for ASF's Nexus, but requirements to sync artifacts to Maven Central also, and are core parts of the Maven2 repository format. In order for Maven to discontinue these, 3 components need to simultaneously (or coordinated-ly) update: maven-deploy-plugin, maven dependency resolver, and Nexus. I'm sure that will happen at some point, but it's going to take some time, and as I've said... all that is outside our control. > > When verifying a release, I want to verify the signatures are correct and I > want to use the asc and sha512 that will show up on the downloads page at > https://accumulo.apache.org/downloads/. > > What about something like "Nexus generates an md5 and sha1 file but those > are no longer recommended by Apache for verification of artifacts. For > release verification use the 512 signature below" or something. > Inaccuracies not withstanding (see above for clarification regarding which component generates these), the situation is more complex than merely not being recommended. First, it is currently not easy or reasonable to stage checksums of better quality in Nexus at all. Second, the release distribution policy applies to the ASF mirrors, not Nexus. Since we're staging in Nexus, the SHA512 have been provided directly in the email for the artifacts which will be placed on the mirrors. Third, for all artifacts that are not going on the mirrors (all the "convenience binary" jars that are only published to Maven), the *only* way to verify these is with their corresponding .sha1/.md5 (or .asc) files uploaded to Nexus. This current situation is complex... and it's very hard to encapsulate all the relevant details in a vote email, but I believe we are doing the right thing: 1. We are in compliance with the updated release distribution policy (as I said, one of the first to be), 2. We include SHA512 sums directly in our vote thread for provenance, which almost nobody else in ASF does (b/c we don't need to: the GPG signatures satisfy anything these could, plus some, and they satisfy their purpose just as well detached), and 3. We acknowledge the existence of the location of the Maven hashes, for all artifacts stored in Maven to ensure there are no errors with the staging area, and for other Maven-specific purposes. Short of explaining this over and over and over again, so that everybody else is as familiar with all these details as I am (which I can do, but it is time consuming, and lengthy), I don't know what we can do better. The only thing I can think of is possibly improve the wording in the template, but it's hard to be accurate without being excessively verbose. When I wrote the original wording for the template, I opted for brevity, but it seems to have only raised questions from those unfamiliar with all the relevant complexities, since this is the second vote thread in which we've discussed this. I'm open to suggestions, but I think your above suggestion diverges from accuracy a bit too far, since: 1) Nexus doesn't generate them, 2) it's not the case that the relevant issue is whether they are recommended by Apache, 3) it's further not the case that Apache has completely discontinued use of sha1/md5 for all purposes, since the policy update only applies to the mirrors, not Nexus, and 4) the *only* mandatory mechanism required by ASF that can be used to verify a release is the cryptographic signature (GPG) http://www.apache.org/legal/release-policy.html#what-must-every-release-contain ; all others are requirements of the distribution mirrors and are fixed characteristics of the release artifacts, not something that is itself voted on beyond the contents of the artifacts (a checksum is a fixed characteristic, like file size; we vote on the contents of the artifacts, not these traits); those other requirements for the mirrors are not release vote requirements (which makes sense... we're voting on the contents of the artifacts... not fixed, deterministic traits like their file size or checksums). > Thanks Christopher. No problem. I'm happy to explain further or clarify anything, as needed. And, if I've said something wrong, I welcome corrections. > > On Mon, Apr 1, 2019 at 10:21 PM Christopher <[email protected]> wrote: > > > Apologies, but I was confused because you said the template was > > wrong... but you quoted a portion of the template which was not wrong, > > and which I've already explained in my original response to Mike. > > > > Once again, those are references to files generated by the Maven > > tooling, outside our control. Granted, we don't have to mention them > > at all. However, the template merely acknowledges their existence. > > Providing information for how those files are named still has value in > > the Maven context, and is useful for any artifact downloaded from > > Maven, not just our tarballs. > > > > Would there be less confusion over this if the template was a bit more > > verbose, saying something like: > > (Append ".asc" to download an artifact's corresponding GPG signature, > > or ".sha1" or ".md5" to verify the checksums generated by Maven.) > > > > If you'd prefer this, or an alternative wording, please change it in > > the repo... or let me know and I'll change it. > > > > On Mon, Apr 1, 2019 at 2:22 PM Josh Elser <[email protected]> wrote: > > > > > > Again, like I included earlier: > > > > > > > (Append ".sha1", ".md5", or ".asc" to download the signature/hash for > > > a given artifact.) > > > > > > On 4/1/19 1:56 PM, Christopher wrote: > > > > In what way? > > > > > > > > On Mon, Apr 1, 2019 at 1:54 PM Josh Elser <[email protected]> wrote: > > > >> > > > >> Your email template is wrong. > > > >> > > > >> On 4/1/19 1:33 PM, Christopher wrote: > > > >>> Sorry, I don't understand what you mean by 'retelling of "checksums > > of old"'. > > > >>> > > > >>> On Mon, Apr 1, 2019 at 12:30 PM Josh Elser <[email protected]> > > wrote: > > > >>>> > > > >>>> I think Mike's point was your VOTE template does not reflect the > > > >>>> retelling of "checksums of old" > > > >>>> > > > >>>> > (Append ".sha1", ".md5", or ".asc" to download the > > signature/hash for > > > >>>> a given artifact.) > > > >>>> > > > >>>> On 3/31/19 10:54 PM, Christopher wrote: > > > >>>>> Mike, > > > >>>>> > > > >>>>> We already stopped using md5 and sha1 for the release artifacts on > > the > > > >>>>> mirrors. I did this some time ago, and we discussed it on list on > > > >>>>> previous vote threads (last year)... which resulted in me changing > > the > > > >>>>> release candidate build script automated tooling to embed the > > SHA512 > > > >>>>> sums for the tarballs directly in the release vote message. I even > > > >>>>> went back and updated the downloads page for the previous releases > > and > > > >>>>> updated the mirrors to be SHA512 only. Because of these steps I > > took, > > > >>>>> Accumulo was one of the first projects across the entire ASF who > > were > > > >>>>> 100% compliant immediately after INFRA VP updated the release > > > >>>>> distribution policy you linked. > > > >>>>> > > > >>>>> *This is a resolved action for Accumulo.* > > > >>>>> > > > >>>>> FWIW, SHA512 was also used as the hash algorithm in the GPG > > signature > > > >>>>> (same as every RC I've ever prepped for ASF). The only remaining > > md5 > > > >>>>> and sha1 reference are Maven-specific tooling, and we have no > > control > > > >>>>> over that tooling. We could change the vote template to no longer > > > >>>>> mention them, but I don't see the point since they're still > > relevant > > > >>>>> within the context of Maven artifact hosting, and that's the > > context > > > >>>>> in which they are presented in the vote email. > > > >>>>> > > > >>>>> On Sun, Mar 31, 2019 at 1:59 PM Michael Wall <[email protected]> > > wrote: > > > >>>>>> > > > >>>>>> -1 for the issue with commons config > > > >>>>>> > > > >>>>>> I check the signatures, they are good. We should stop using md5 > > and sha1 > > > >>>>>> though, see > > https://www.apache.org/dev/release-distribution#sigs-and-sums. > > > >>>>>> Has anyone looked at moving to sha256 and/org sha512? > > > >>>>>> Successful run of mvn clean verify -Psunny > > > >>>>>> > > > >>>>>> On Sat, Mar 30, 2019 at 11:31 PM Keith Turner <[email protected]> > > wrote: > > > >>>>>> > > > >>>>>>> I completed a continuous ingest run on a 10 node cluster running > > > >>>>>>> Centos 7. I used the native map. I had to rebuild Accumulo to > > work > > > >>>>>>> around #1065 inorder to get the verify M/R job to run. > > > >>>>>>> > > > >>>>>>> > > org.apache.accumulo.test.continuous.ContinuousVerify$Counts > > > >>>>>>> REFERENCED=34417110819 > > > >>>>>>> UNREFERENCED=9097524 > > > >>>>>>> > > > >>>>>>> On Wed, Mar 27, 2019 at 7:57 PM Christopher <[email protected]> > > wrote: > > > >>>>>>>> > > > >>>>>>>> Accumulo Developers, > > > >>>>>>>> > > > >>>>>>>> Please consider the following candidate for Apache Accumulo > > 1.9.3. > > > >>>>>>>> > > > >>>>>>>> This supersedes RC1 and contains the following change: > > > >>>>>>>> https://github.com/apache/accumulo/pull/1057 > > > >>>>>>>> > > > >>>>>>>> Git Commit: > > > >>>>>>>> 94f9782242a1f336e176c282f0f90063a21e361d > > > >>>>>>>> Branch: > > > >>>>>>>> 1.9.3-rc2 > > > >>>>>>>> > > > >>>>>>>> If this vote passes, a gpg-signed tag will be created using: > > > >>>>>>>> git tag -f -m 'Apache Accumulo 1.9.3' -s rel/1.9.3 \ > > > >>>>>>>> 94f9782242a1f336e176c282f0f90063a21e361d > > > >>>>>>>> > > > >>>>>>>> Staging repo: > > > >>>>>>> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077 > > > >>>>>>>> Source (official release artifact): > > > >>>>>>>> > > > >>>>>>> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077/org/apache/accumulo/accumulo/1.9.3/accumulo-1.9.3-src.tar.gz > > > >>>>>>>> Binary: > > > >>>>>>> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077/org/apache/accumulo/accumulo/1.9.3/accumulo-1.9.3-bin.tar.gz > > > >>>>>>>> (Append ".sha1", ".md5", or ".asc" to download the > > signature/hash for > > > >>>>>>>> a given artifact.) > > > >>>>>>>> > > > >>>>>>>> In addition to the tarballs, and their signatures, the > > following checksum > > > >>>>>>>> files will be added to the dist/release SVN area after release: > > > >>>>>>>> accumulo-1.9.3-src.tar.gz.sha512 will contain: > > > >>>>>>>> SHA512 (accumulo-1.9.3-src.tar.gz) = > > > >>>>>>>> > > > >>>>>>> > > b366b89295b1835038cb242f8ad46b1d8455753a987333f0e15e3d89749540f2cd59db1bc6cf7100fc9050d3d0bc7340a3b661381549d40f2f0223d4120fd809 > > > >>>>>>>> accumulo-1.9.3-bin.tar.gz.sha512 will contain: > > > >>>>>>>> SHA512 (accumulo-1.9.3-bin.tar.gz) = > > > >>>>>>>> > > > >>>>>>> > > cc909296d9bbd12e08064fccaf21e81b754c183a8264dfa2575762c76705fd0c580b50c2b224c60feaeec120bd618fba4d6176d0f53e96e1ca9da0d9e2556f1f > > > >>>>>>>> > > > >>>>>>>> Signing keys are available at > > https://www.apache.org/dist/accumulo/KEYS > > > >>>>>>>> (Expected fingerprint: 8CC4F8A2B29C2B040F2B835D6F0CDAE700B6899D) > > > >>>>>>>> > > > >>>>>>>> Release notes (in progress) can be found at: > > > >>>>>>>> https://accumulo.apache.org/release/accumulo-1.9.3/ > > > >>>>>>>> > > > >>>>>>>> Release testing instructions: > > > >>>>>>>> https://accumulo.apache.org/contributor/verifying-release > > > >>>>>>>> > > > >>>>>>>> Please vote one of: > > > >>>>>>>> [ ] +1 - I have verified and accept... > > > >>>>>>>> [ ] +0 - I have reservations, but not strong enough to vote > > against... > > > >>>>>>>> [ ] -1 - Because..., I do not accept... > > > >>>>>>>> ... these artifacts as the 1.9.3 release of Apache Accumulo. > > > >>>>>>>> > > > >>>>>>>> This vote will remain open until at least Sun Mar 31 00:00:00 > > UTC 2019. > > > >>>>>>>> (Sat Mar 30 20:00:00 EDT 2019 / Sat Mar 30 17:00:00 PDT 2019) > > > >>>>>>>> Voting can continue after this deadline until the release > > manager > > > >>>>>>>> sends an email ending the vote. > > > >>>>>>>> > > > >>>>>>>> Thanks! > > > >>>>>>>> > > > >>>>>>>> P.S. Hint: download the whole staging repo with > > > >>>>>>>> wget -erobots=off -r -l inf -np -nH \ > > > >>>>>>>> > > > >>>>>>> > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1077/ > > > >>>>>>>> # note the trailing slash is needed > > > >>>>>>> > >
