On Mon, Jun 27, 2022 at 01:43:19AM +0200, Zoltan Puskas wrote: *snip*
> First of all one of the advantages of Gentoo is that it gets it's source > code from upstream (yes, I'm aware of mirrors acting as a cache layer), > which means that poisoning source code needs to be done at upstream > level (effectively means hacking GitHub, PyPi, or some standalone > project's Gitea/cgit/gitlab/etc. instance or similar), sources which > either have more scrutiny or have a limited blast radius. I don't quite follow what you mean. Upstream for go modules is actually proxy.golang.org, or some other similar proxy, which the go tooling knows how to access [1]. > Additionally if an upstream dependency has a security issue it's easier > to scan all EGO_SUM content and find packages that potentially depend on > a broken dependency and force a re-pinning and rebuild. The tarball > magic hides this completely and makes searching very expensive. I'm not comfortable at all with us changing the dependencies like this downstream for the same reason the Debian folks ultimately were against it for kubernetes. If you make these kinds of changes you are affectively creating a fork, and that would mean we would be building packages with untested libraries [2]. *snip* > Considering that BTRFS (and possibly other filesystems) support on the > fly compression the physical cost of a few inflated ebuilds and The problem here is the size of SRC_URI when you add the EGO_SUM_SRC_URI to it. SRC_URI gets exported to the environment, so it can crash portage if it is too big. > Manifests is actually way smaller than the logical size would indicate. > Compare that to the huge incompressible tarballs that now we need to > store. > > As a proxied maintainer or overlay owner hosting these huge tarballs > also becomes problem (i.e. we need some public space with potentially > gigabytes of free space and enough bandwidth to push that to users). > Pushing toward vendor tarballs creates an extra expense on every level > (Gentoo infra, mirrors, proxy maintainers, overlay owners, users). I agree that creating the dependency tarballs is not ideal. We asked for another option [3], but as you can see from the bug this was refused by the PMS team. That refusal is the only reason we have to worry about dependency tarballs. > It also breaks reproducibility. With EGO_SUM I can check out an older > version of portage tree (well to some extent) and rebuild packages since > dependency upstream is very likely to host old versions of their source. > With the tarballs this breaks since as soon as an ebuild is dropped from > mainline portage the vendor tarballs follow them too. There is no way > for the user to roll back a package a few weeks back (e.g. if new > version has bugs), unlike with EGO_SUM. The contents of a dependency tarball is created using "go mod download", which is controlled by the go.mod/go.sum files in the package. So, it is possible to recreate the dependency tarball any time. I do not see any advantage EGO_SUM offers over the dependency tarballs in this space. > Finally with EGO_SUM we had a nice tool get-ego-vendor which produced > the EGO_SUM for maintainers which has made maintenance easier. However I > haven't found any new guidance yet on how to maintain go packages with > the new tarball method (e.g. what needs to go into the vendor tarball, > what changes are needed in ebuilds). Overall this complifates further > ebuild development and verification of PRs. The documentation for how to build dependency tarballs is in the eclass. The GOMODCACHE environment variable is used in the eclass to point to the location where the dependency tarball is unpacked, and that location is read by the normal go tooling. > In summary, IMHO the EGO_SUM way of handling of go packages has more > benefits than drawbacks compared to the vendor tarballs. EGO_SUM can cause portage to break; that is the primary reason support is going away. We attempted another solution that was refused, so the only option we have currently is to build the dependency tarballs. > > Cheers, > Zoltan > > [1] > https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packagers-security-nightmare/ > [1] https://go.dev/ref/mod [2] https://lwn.net/Articles/835599/ [3] https://bugs.gentoo.org/833567
signature.asc
Description: PGP signature
