On Mon, Sep 19, 2011 at 10:53 AM, Michał Górny <mgo...@gentoo.org> wrote:
> On Mon, 19 Sep 2011 10:39:11 +0200
> Corentin Chary <corentin.ch...@gmail.com> wrote:
>
>> ## Also update eix database, because we use eix internaly
>> ## Bottleneck: disk and cpu
>> ##Time: 30mn ~ 1h
>> eix-update
>
> Using egencache to keep caches for overlays will make eix updates much
> faster.
>
> Here's my code for it (it uses overlays in /usr/portage/local):
>
> cd /usr/portage/local && \
> for O in */; do
> echo "${O}"
> egencache --jobs=8 --update --update-use-local-desc --rsync \
> --repo=$(cat ${O}profiles/repo_name)
> done
>
Just done that, and indeed, it's much faster.
Also I switched to PostgreSQL, and since that avoids a lot of deadlock
situations when making queries in parallel, I'm now able to do that:
eix --only-names -x | gparallel --eta --load 8 --jobs 400%
--max-args=64 python manage.py scan-metadata
And now it takes less than 30mn instead of more than 1h, and more
importantly, it scales.
--
Corentin Chary
http://xf.iksaif.net
