> On Apr 17, 2022, at 4:02 PM, sebb <seb...@gmail.com> wrote: > > On Sun, 17 Apr 2022 at 23:31, Christopher <ctubb...@apache.org > <mailto:ctubb...@apache.org>> wrote: >> >> I don't think this is proposing to have links to the >> checksums/signatures on the CDN, but rather to have the closer.lua >> template, which is still controlled by INFRA, include links to those >> files at the recommended location at downloads.apache.org. closer.lua >> can easily link to $x at dlcdn.apache.org, but $x.sha512 and $x.asc >> and KEYS at downloads.apache.org. > > Yes, the idea is to extend the existing page to create a minimal valid > download page for projects to use. > The links to KEYS, sigs and hashes would use the same URLs as a > regular download page
Finding the .asc and either .sha256 or .sha512 (or both) is relatively straightforward. As Sebb noted earlier, finding the KEYS file may be a bit harder. > > FTR, I have got a working Docker image which I will upload shortly. Looking forward to seeing the patch to closer.lua and closer.html. Craig > >> On Sun, Apr 17, 2022 at 5:46 PM Jarek Potiuk <ja...@potiuk.com> wrote: >>> >>> Sebb, >>> >>> I think it would be great to see some result analysis on >>> security/integrity of the proposal. I wonder, did you make any >>> analysis on the integrity aspect? What's the result of it? >>> >>> I believe originally the requirement for separating file downloads and >>> sha/checksum were to make sure that the checksum/sha files are always >>> served directly from the fully apache-controlled delivery chain. >>> Previously, when there were mirrors, this was the only way to make >>> sure you could validate the provenience of the files - the .sha and >>> .sum files were always served from "https://download.a.o"; so - unless >>> your local certificate authority source were compromised, you could be >>> sure that the downloaded files were not tampered with. It's all too >>> easy to modify both - the file and their checksum/sha to "pretend it >>> is valid". >>> >>> I understand the main reason why this was done this way was that >>> directly the Apache-owned and fully controlled servers were able to >>> serve all those files - because they are very small and can be served >>> directly by the Apache server under the infra control, while using >>> mirrors/cdn is absolutely necessary to serve the binary content which >>> is huge, and we cannot really route all the users to hit directly the >>> apache download server. This is at least when I tried to understand >>> the context and reasoning why it was implemented this way. >>> >>> I am not sure how dlcdn works and how "trustful" it is for the Apache >>> to be 100% sure the content has not been tampered with by the CDN >>> provider. >>> Do you know what is the consequence of your proposal on the integrity >>> check in case they are part of close.lua? Does it mean that it will be >>> served from a "fully owned" apache page ? Is it something that ASF can >>> "delegate" to the CDN provider? Is it something that closer.lua can >>> handle in the "provenience sure way" ? Is the content served by >>> closer.lua fully served by the ASF-owned server and cannot be tampered >>> with by the CDN provider? >>> >>> I do not know the answer to those questions (I do not know too much >>> about the closer.lua scripts and what's the level of trust between the >>> ASF and CDN) - but I am sure that when you proposed the changes, you >>> considered the context and reasoning why it was separated - so can you >>> tell us all if all the integrity and "provenience" aspects are handled >>> well by your proposal? >>> >>> J. >>> >>> On Sun, Apr 17, 2022 at 1:06 PM sebb <seb...@gmail.com> wrote: >>>> >>>> On Sun, 17 Apr 2022 at 11:56, sebb <seb...@gmail.com> wrote: >>>>> >>>>> On Sun, 17 Apr 2022 at 10:36, Greg Stein <gst...@gmail.com> wrote: >>>>>> >>>>>> Hi Craig, >>>>>> >>>>>> The closer.lua, its prior name closer.cgi. and all predecessors have >>>>>> been fine with the Foundation's consumers liking its functionality. This >>>>>> has supported 200+ projects (or more, depending on how you'd like to >>>>>> count). Changes to that system are (thus) demand-based, which we have >>>>>> not seen. Please provide some PRs with the changes you would like to >>>>>> make. >>>>>> >>>>>> I do believe that others agree with your ideas on "automated hashes", >>>>>> though I really have no idea on the follow-on security details. Nor a >>>>>> mechanism for broad-based security chains of GPG keys and hashes for the >>>>>> artifacts. >>>>> >>>>> My idea here was just to link to the sig and hash by adding the >>>>> appropriate suffix to the artifact path name, and checking that it >>>>> exists. >>>>> e.g. look for db/jdo/3.2/jdo-3.2-source-release.tar.gz + .asc also >>>>> .sha256 or .sha512 etc. >>>>> The code already checks for the presence of the artifact. >>>>> >>>>> There aren't that many valid hash types to check; if the code cannot >>>>> find the required files, it could direct users back to the project. >>>>> >>>>>> Figure it out, discuss on the mailing list, file some Pull-Requests, and >>>>>> we can move forward. >>>> >>>> It looks like the code is currently at: >>>> >>>> https://github.com/apache/infrastructure-p6/blob/production/modules/closer_cgi/files/closer.lua >>>> (requires GitHub login linked to ASF login with appropriate karma) >>>> >>>>>> Cheers, >>>>>> Greg >>>>>> InfraAdmin, ASF >>>>>> >>>>>> >>>>>> On Sat, Apr 16, 2022 at 6:23 PM Craig Russell <apache....@gmail.com> >>>>>> wrote: >>>>>>> >>>>>>> I'd like to ask on behalf of the DB PMC for the infra tool closer.lua >>>>>>> to be enhanced a bit for use by projects. >>>>>>> >>>>>>> It will be much easier for us if we can simply link to e.g. >>>>>>> https://www.apache.org/dyn/closer.lua/db/jdo/3.2/jdo-3.2-source-release.tar.gz >>>>>>> and have that page contain the .asc and .sha512 as well. >>>>>>> >>>>>>> Even better if closer.lua can also find the relevant KEYS file and link >>>>>>> to it when describing how to check the release. >>>>>>> >>>>>>> With this approach, it is easier for the project and less error-prone >>>>>>> to maintain the download page(s). We can include the current release as >>>>>>> well as archived releases on the same download page and omit the links >>>>>>> to the .asc and .sha512 and KEYS files. >>>>>>> >>>>>>> Please let us know how we can help. >>>>>>> >>>>>>> Regards, >>>>>>> Craig >>>>>>> >>>>>>> >>>>>>> On Apr 16, 2022, at 2:01 AM, Greg Stein <gst...@gmail.com> wrote: >>>>>>> >>>>>>> On Thu, Apr 14, 2022 at 4:43 PM sebb <seb...@gmail.com> wrote: >>>>>>>> >>>>>>>> On Thu, 14 Apr 2022 at 20:52, Christopher <ctubb...@apache.org> wrote: >>>>>>>>> >>>>>>>>> Doing this would probably restrict INFRA's ability to adapt it to >>>>>>>>> their changing needs, as projects would become dependent on it. >>>>>>>> >>>>>>>> Projects are already dependent on the page. >>>>>>> >>>>>>> >>>>>>> Correct. We *want* projects to use closer.lua. It provides a control >>>>>>> point to direct users to the appropriate location to download. Today, >>>>>>> it primarily sends them to our CDN and to an EU download location. >>>>>>> >>>>>>> Should those decisions ever change, we want projects to be using >>>>>>> closer.lua to effect those changes. >>>>>>> >>>>>>>> >>>>>>>> I think it would make it easier for Infra to make changes, as they >>>>>>>> would be able to adjust it. >>>>>>> >>>>>>> >>>>>>> Yes. >>>>>>> >>>>>>>> >>>>>>>>> Probably best to leave the mirror-management page independent from the >>>>>>>>> project download pages, since they serve the needs of separate groups. >>>>>>>> >>>>>>>> I was not suggesting replacing these, for those projects that want to >>>>>>>> customise the pages. >>>>>>> >>>>>>> >>>>>>> Today, we no longer have a mirror system. But requiring projects to use >>>>>>> closer.lua ensures that we can swap in that future option. >>>>>>> >>>>>>> All that said, as I recall: closer.lua provides support for .ezt pages >>>>>>> so that projects can provide custom download pages. Maybe there is a >>>>>>> way to provide the needed variables to projects' custom download page >>>>>>> templates. >>>>>>> >>>>>>> Or, they can just keep using their pages. >>>>>>> >>>>>>> The download pages are owned by the TLPs. Infra isn't gonna interfere >>>>>>> with them. Should any TLPs want more features, then they can ask. We >>>>>>> haven't seen any requests in years from the TLPs. >>>>>>> >>>>>>> Cheers, >>>>>>> Greg >>>>>>> InfraAdmin, ASF >>>>>>> >>>>>>> >>>>>>> >>>>>>> Craig L Russell >>>>>>> c...@apache.org Craig L Russell c...@apache.org
