On Tuesday, 17 January 2017 20:51:38 GMT Bill Kenworthy wrote: > On 17/01/17 18:03, Peter Humphrey wrote: > > Hello list, > > > > Does anyone have some pointers to setting up a local portage mirror for > > git to serve? I'm looking at http://www.funtoo.org/Portage_Git_Mirror > > but some details are obscure to me. I hope I'm not going to have to > > roll my sleeves up and learn the whole of git; I just want to cut down > > the time my boxes take to sync via git - some of them have their > > portage trees nfs-mounted in chroots on my workstation and take > > anything up to 20 minutes to sync. > > > > One question comes up straight away: should the git-served repository be > > the host's own /usr/portage tree? And what cron task should I run to > > refresh the mirror? > > > > Any help would be gratefully received. > > > > (I can still read e-mails via web-mail even when KMail is playing up.) > > I moved my systems to a local git repos some time ago - it works (fast - > much faster than rsync!)but its not perfect: > > It would be nice if someone can share how it really should be done!
It seems to me to be ideal material for a wiki page. Any offers? > Below is the contents of an email to the list in 2015 when I got it > working: > > * problems: > the files in /usr/share/portage sometimes get overridden and give an > error message > renaming "gentoo" to "olympus" (the server host) made sense at the time > but creates odd errors at times requiring files to be restored. > git doesnt use the portage checksums which besides odd errors at times > required using "thin-manifests = true" in > "/usr/portage/metadata/layout.conf" Many thanks, Bill. That'll give me something to think about! -- Regards Peter > ++++++++++++++++++ > Replace your rsync infrastructure with a git one. After set up, there > is faster syncing than rsync, both externally to the gentoo > infrastructure (reasonable ADSL link), and really fast compared to rsync > internally. Not simple, but its a one off hit on just a few machines > for me. Works for me, no warranty etc. > > Thanks to Michael and Hasufell for the hints needed to get this working. > Apologies for the line wrap. > > > > Replace with your own values! > repo name: olympus > repo master host: olympus.lan.localdomain > > On the master: > > install git !!! > > Download https://github.com/hasufell/portage-gentoo-git-config > > unzip > > cp -v portage-gentoo-git-config-master/etc/portage/repo.postsync.d/* > /etc/portage/repo.postsync.d/ > > cp portage-gentoo-git-config-master/etc/portage/repos.conf/gentoo.conf > /etc/portage/repos.conf/ > > if needed: chmod +x /etc/portage/repo.postsync.d/sync_overlay_cache > cd /usr/ > mv portage portage.old > > emerge --sync > > > To use as the master for your own mirror system: > Uncomment the lines in /etc/rsyncd.conf to set up your own > gentoo-portage rsync server - still needed for now > > rc-update add rsyncd > > /etc/init.d/rsyncd start > > > > On the internal clients wanting to use the mirror: > reccomended: set up passwordless ssh between clients and the master > > emerge dev-vcs/git > > create /etc/portage/repos.conf/olympus.conf as follows > -------------------- > [DEFAULT] > main-repo = olympus > > [olympus] > location = /usr/portage > sync-type = git > sync-uri = git://olympus.lan.localdomain:/usr/portage > auto-sync = yes > -------------------- > > create /etc/portage/repo.postsync.d/sync_olympus > -------------------- > #!/bin/bash > > repository_name="${1}" > repository_path="${3}" > > [[ ${repository_name} == "olympus" ]] || exit 0 > > source /lib/gentoo/functions.sh > > # Number of jobs for egencache, default is number or processors. > parallel_jobs="$(nproc)" > > # caches > ebegin "Fetching pre-generated metadata cache for ${repository_name}" > rsync -a > rsync://olympus.lan.localdomain/gentoo-portage/metadata/md5-cache/ > "${repository_path}"/metadata/md5-cache/ > eend $? > > ebegin "Updating metadata cache for ${repository_name}" > egencache --jobs="${parallel_jobs}" --repo="${repository_name}" > --update --update-use-local-desc > eend $? > > #DTD's > DTDDIR="${repository_path}"/metadata/dtd > ebegin "Updating DTDs" > if [[ -e ${DTDDIR} ]]; then > git -C "${DTDDIR}" pull -q --ff > else > git clone olympus.lan.localdomain:/usr/portage/metadata/dtd > "${DTDDIR}" > fi > eend "$?" > > #GLSA's > GLSADIR="${repository_path}"/metadata/glsa > ebegin "Updating GLSAs" > if [[ -e ${GLSADIR} ]]; then > git -C "${GLSADIR}" pull -q --ff > else > git clone olympus.lan.localdomain:/usr/portage/metadata/glsa > "${GLSADIR}" > fi > eend "$?" > > # herds > ebegin "Updating herds.xml" > rsync -a > rsync://olympus.lan.localdomain/gentoo-portage/metadata/herds.xml > "${repository_path}"/metadata/md5-cache/ > eend $? > > #news > NEWSDIR="${repository_path}"/metadata/news > ebegin "Updating news items" > if [[ -e ${NEWSDIR} ]]; then > git -C "${NEWSDIR}" pull -q --ff > else > git clone olympus.lan.localdomain:/usr/portage/metadata/news > "${NEWSDIR}" > fi > eend $? "Try to remove ${NEWSDIR}" > -------------------- > > > mv /usr/portage /usr/portage.old > > git clone olympus.lan.localdomain:/usr/portage /usr/portage > > echo 'olympus' > /usr/portage/profiles/repo_name > > echo 'masters = olympus' > /usr/local/portage/metadata/layout.conf > > echo 'thin-manifests = true' >> /usr/local/portage/metadata/layout.conf > > cp /usr/portage/metadata/layout.conf > /usr/local/portage/metadata/layout.conf > > cp /etc/portage/repos.conf/olympus.conf > /usr/share/portage/config/repos.conf > > emerge --sync > emerge --sync > > * the first sync will complete the process, the second is to check there > are no errors.
