On 22/03/2022 22.23, Ryan Schmidt wrote: > On Mar 22, 2022, at 13:08, Daniel J. Luke wrote: > >> On Mar 21, 2022, at 9:20 PM, Ryan Schmidt wrote: >>> Ports that fetch their sources from a revision control system do not enjoy >>> the protection of checksums. Although ports that fetch source from a >>> revision control system specify which tag or commit hash to fetch, it is >>> conceivable that a developer with sufficient access to that repository >>> could delete an old tag and replace it with a new tag of the same name that >>> contains different software. This is one of the reasons why we recommend >>> ports fetch using distfiles, and the vast majority do. We might consider >>> recommending that ports that fetch directly from a git repository >>> (fetch.type git) never use a tag, and always use the commit hash >>> corresponding to that tag, since replacing the contents of the repository >>> while keeping the same sha1 hash is, as far as I know, still impossible in >>> the general case. (Yes, it is possible to engineer a sha1 collision, but >>> only if you can carefully control both the old and new files. Generating a >>> sha1 collision against some existing old file is a very different matter.) >> >> I haven't reviewed the current literature but one thing is certain - attacks >> only get better. We should strongly discourage any use of revision control >> as the source. > > As I said we do already and have for a long time strongly discouraged using > revision control systems for downloads.
As these days most web frontends allow to download a tarball, we could also think about discouraging fetching from VCS completely. >> We could shorten the window where a person could get sources that don't >> match what the maintainer expected with a little infrastructure magic. >> Roughly in order of presumed level of effort: >> >> - provide a place for maintainers to manually upload distfiles that they can >> generate from a source checkout >> - provide some port command for maintainers to upload distfiles (as a >> convenience for the above upload process) >> - have a process that automatically generates distfiles from the checked out >> source and puts it on our mirrors (builder could probably do this since it's >> already going to check out the source), then have base look for the distfile >> on our mirrors first (even if the portfile specifies to check out from a >> revision control system) > > Rainer did some work on automating the creation of distfiles from revision > control systems but it has not been touched in some years. Since Rainer > hasn't been very active here lately maybe someone else would like to pick it > up and finish the work. > > https://github.com/macports/macports-base/tree/vcs-fetch Right, what Daniel describes in the last point is basically what is implemented in the vcs-fetch branch. I think the compare view is the better starting point for the code: https://github.com/macports/macports-base/compare/vcs-fetch MacPorts will fetch from a VCS, but creates a reproducible tarball with bsdtar. This allows to mirror a source tarball for all of fetch.type git/svn/hg/bzr/cvs if they point to a fixed revision as they should. The checksum of this tarball is in the Portfile as usual. The general idea is that the existing mirroring job would create such a reproducible tarball from the VCS that is distributed normally. On the user end, fetch looks for a tarball on the mirrors first before attempting a fetch from VCS. For fetch.type git it also implements the additional feature to also download submodules, which has been a requirement that could not be specified with the fetch.* options [1]. The problem is that this branch has become stale, fully attributed to my lack of interest and shift in priorities. This feature was started back in 2016 at the MacPorts Meeting together with Mojca, but according to the git log I last worked on it in 2018 ... I am fairly confident the vcs-fetch branch could be rebased onto the current master. I really don't mind if someone else wants to pick this up, no, I would rather be glad to see this finished. Rainer [1] https://trac.macports.org/ticket/50708