Re: p2k11 changes
... and with the just committed prepare change, which makes it about twice as fast in the common case (already installed deps), this should be all for now (of course, I'm just saying that to trick my brain into finding more speed-ups).
p2k11 changes
I spent most of last week working on speed improvements in the package creation process, with nice results. print-package-signature and out-of-date are now about twice as fast. packages with big dependency tree and no wantlib build very much faster (the new meta/* stuff is a proof of that). ports with lots of multi-packages will also see big improvements. port-lib-depends-check will also benefit. I think we're about back to the speeds before all the sanity checks, but with all the sanity checks. Technical details: - every user-visible operation that wants a port's libraries will create a cache directory and cache the results. - resolve-lib has some new options specifically for lib-depends-args, so that it gets invoked once instead of once per library. - pkg_create runs a specialized solver, that will only solve its own libraries instead of the full tree (and thus stops much earlier in many cases) - pkg_create is aware of the cache directory and caches plists in there. Thus, when they are already there, the next package in a multi-package situation no longer needs to regenerate it. This cache is destroyed after each user-invoked operation, because out-of-synch issues would be really bad. Users that really know what they are doing (e.g., a build without any change whatsoever) can try creating a directory and setting _DEPENDS_CACHE pointing to that directory.
Re: p2k11 changes
Of course, it's always better with actual benchmarks. In normal cases, the dependency checking will be often twice as fast. Ways worse in some pathological cases: make package in lang/php/5.2 before: 3m06s after: 1m06s - /3 make print-package-signature from the top-level: before: 230m21s after: 30m52s - /7 !! make package in meta/gnome before: 11m43s after: 32s - /20 !! typical out-of-date run (300 packages) before: 4m53s after: 1m40s - /3
Re: p2k11 changes
Of course, it's always better with actual benchmarks. In normal cases, the dependency checking will be often twice as fast. Ways worse in some pathological cases: make package in lang/php/5.2 before: 3m06s after: 1m06s - /3 make print-package-signature from the top-level: before: 230m21s after: 30m52s - /7 !! this is very very useful to make sure dpb doesn't rebuild needlessly. make package in meta/gnome before: 11m43s after: 32s - /20 !! typical out-of-date run (300 packages) before: 4m53s after: 1m40s - /3 the make package speedup will boost packaging quickly :-) thanks very much for that! several observations 1) when you restart a errored out package, dpb takes about 10-15 secs (or usually more) to scan make depends, and then very quickly it goes through configure, make, make fake by observing the cookies. 2) make plist on huge packages like kde-l10n is slow. will the speedups benefit in both the above cases? i decided to hold off rebuilding until hackathon is finished :-) otherwise you never get caught up as dpb rebuilds after each cvs up. thanks a bunch
