On May 24, 2022, at 12:07 PM, Ryan Schmidt <ryandes...@macports.org> wrote: > I'm not convinced that I would be happy to see such a feature. Just a > moment's consideration shows how many different parts of MacPorts would need > modifications to support it, not only in base but also in the buildbot and in > the package hosting infrastructure. And in the end, what we end up with is a > rather fundamental violation of a principle we currently rely on, which is > that a given version and revision of a port is (unless there is a bug) > reproducible, immutable, deterministic.
The assumption you're making is that any possible implementation will be bad? ;-) I don't think it's worth bikeshedding this when no one has done any work - but if we step back and consider that all we'd be trying to do is automate the process you already go through when reading a comment and rev-bumping a port, it doesn't seem like it's a noncomputable problem. > To get there, we'd have to modify the registry to store what version of > curl-ca-bundle was installed at the time that privoxy-pki-bundle was > installed, and we'd have to extend the portindex to hold information about > which ports should trigger a reinstall for another port, and we'd have to > enhance base's understanding of outdatedness to make use of this new > information, and enhance "port outdated" to display that information. There are a lot of possible implementation paths. Yes, there would be additional information we'd need to store, and yes base would have to understand this information. > And suppose we have modified the buildbot to know that it must rebuild > privoxy-pki-bundle and upload a new package. (Currently it decides whether to > build a new package in part based on whether a package with the correct > filename already exists on the server, which now won't be sufficient > criteria.) One possible implementation would be to use one of the existing discriminators (epoch, version, revision) or add an additional one to indicate that a new package is necessary - yes this transition would involve a lot of moving pieces - but the buildbot could use the same (or very similar) criteria. > It takes awhile for newly uploaded packages to sync from the private server > to the public one, so let's say by 2pm the new curl-ca-bundle and > privoxy-pki-bundle packages have synced to the public server (although > there's no guarantee that they'd both be there at the same time -- the new > curl-ca-bundle package, having been built first, might arrive 30-60 minutes > ahead of privoxy-pki-bundle). Our mirrors sync from the public server at > varying intervals, so let's say that by 10pm the new packages are on all the > mirrors. Prior to that time, old packages of the same filename might still be > on some mirrors. What happens if a user upgrades to the new curl-ca-bundle at > 5pm? If the new curl-ca-bundle package is available on any mirror they > access, that'll be used, otherwise the new curl-ca-bundle will be built from > source. Fine. Then MacPorts will want to reinstall privoxy-pki-bundle, even > though its version and revision are the same, due to the new hypothetical > feature. It will check for a package on a mirror and it will find one! But > will it be an old package or a newly rebuilt one? This is assuming that any possible implementation would result in non-reproducible builds, which I also wouldn't support. > No one knows. ... because it doesn't exist yet. I think it would be more useful to try to specify what we'd want from a hypothetical feature like this rather than just claim that it's not possible for it to work, maybe something like: - builds must still be reproducible - buildbot must be able to determine it needs to build a new package - a new package must not replace a previously built package - end users must still be able to use pre-built packaages (probably more) On the other hand, if no one is actually doing the work /and/ prominent project leaders are saying they could never support such a feature, it's unlikely that anyone will ever try to make improvements in this area, so it's probably not even worth time defining what it would take for it to be useful and added to MacPorts. > Or perhaps the proposal is that package filenames should grow even longer > with an additional identifier -- which would hopefully be empty for ports not > opting in to this new thing so that all our existing packages are not > invalidated. If so, what would the new identifier be? There isn't really a proposal yet (I don't think anyone is working on code) - but there are of course possibilities here. We could (ab)use epoch, just have portindex increment the revision for the ports that request to be rebuilt when a new version of something is added, add a new int that we auto-increment, hash over the (portname, epoch, version, revision) of deps that a port says it cares about [and countless others that would maybe take me more than 5 minutes to come up with ;-)] -- Daniel J. Luke
