On 10/29/2016 03:11 AM, Rich Freeman wrote: > On Sat, Oct 29, 2016 at 5:42 AM, Neil Bothwick <n...@digimed.co.uk> wrote: >> On Fri, 28 Oct 2016 23:53:03 -0700, Daniel Campbell wrote: >> >>>> Anyone seeing this again? I've just sync'd to two servers in >>>> Australia, and then, for the hell of it, one in Canada and am getting >>>> it for: >>>> >>>> dev-libs/botan >>>> app-arch/tar >>>> media-video/libav >>>> app-crypt/qca >>>> net-print/cups-filters >>>> >>>> I suppose time will sort it out..... >>>> >>>> Andrew >>>> >>> This shouldn't happen unless the distfiles aren't found, or someone (a >>> dev) didn't use repoman to commit, leaving an old Manifest around. >> >> It looks like repoman is the culprit >> >> https://bugs.gentoo.org/show_bug.cgi?id=598376 >> > > This is probably not the issue here, since Gentoo uses thin manifests > (there is nothing for repoman to update). The manifests that are > causing the problem aren't created by regular Gentoo developers. > They're created by a script that runs as a part of the rsync mirror > process. > > This is a fairly well-known problem that has been around for over a year. > > You will only run into this problem if you use rsync to update your > repository, since the problem is created when creating the master > rsync mirror. The original git repository doesn't contain the error, > and the git mirror on github doesn't mess with the Manifests. > > The issue apparently has to do with Changelog generation. In April > the Council gave Infra the option to stop generating Changelogs, which > would eliminate the problem. I suspect those maintaining the scripts > prefer to keep them around, and I don't think anybody on the Council > has access to change the scripts. > > I switched to git syncing eons ago, so I've never seen this bug. I > recognize it has been a source of frustration for a lot of users, and > a bit of frustration for the Council, since there doesn't seem to be a > lot we can do to change it in practice. > > zlg is of course right that these kinds of problems can also be caused > by maintainer failure to use repoman/etc or if an upstream distfile > changes. If that is the problem then you'll see it no matter how you > sync your repo. However, when you get a bunch of these after syncing > it is almost always a result of the mirror creation process. I can't > remember the last time I saw a manifest error (granted, I'm also using > mgorny's stable mirror branch, which I think screens for these kinds > of errors). > > While there can be some latency I do in general recommend syncing from > https://github.com/gentoo-mirror/gentoo . This is a mirror of the > Gentoo developer git repository with two changes: > 1. Metadata is added to the mirror, which greatly speeds things up > compared to using the raw git repository (you can do this yourself, it > is one of the steps done by the rsync generation process as well, but > this one is not buggy). > 2. The default stable branch of this mirror screens for numerous > issues before accepting commits. That means it is generally a little > behind the main branch (at this moment the main branch is 2 minutes > old, and the default stable branch is 20min old), but a lot of the > really annoying issues that are caused by devs skipping repoman won't > be seen. Now, if a maintainer breaks a package then this mirror will > quickly get out of date until the problem is corrected, but Gentoo QA > gets warnings when this is happening and usually the maintainer is > being pestered or somebody else is fixing it. I suspect this process > has probably reduced the error rate for everybody. I have seen this > get a few days old though, which is something to keep in mind. > > It does not contain Changelogs, though if you use it you'll have a > full history so you can just run git whatchanged <path> to get > something pretty close to a changelog. > > To use it just put this in /etc/portage/repos.conf/gentoo.conf: > [DEFAULT] > main-repo = gentoo > > [gentoo] > location = /usr/portage > sync-type = git > sync-uri = https://github.com/gentoo-mirror/gentoo.git > auto-sync = yes > > If you want to git-sync from some other mirror, just change the url > accordingly. > > If you switch your mirror I suggest just renaming /usr/portage and > letting portage re-create it. > > The other big benefit of git syncing is that if you sync every day it > is a lot faster. If you sync less often it will become slower > compared to rsync. git is much more efficient at finding what has > changed, but rsync is not burdened with transferring a complete > history. If you only sync once every few months rsync will be a lot > faster. > Wow, quite the mail there! I actually wasn't aware of thin vs. thick manifest issues (or mgorny's work on a stable mirror). Thanks for sharing and teaching.
-- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6
signature.asc
Description: OpenPGP digital signature
