On Sat, 1 Nov 2014 15:46:18 -0700 Zac Medico <zmed...@gentoo.org> wrote:
> In addition to indexed search, this patch series implements > stream-based incremental display of search results (fixing bug > #412471). > > I'll be maintaining this patch series in the following branch: > > https://github.com/zmedico/portage/tree/bug_525718_stream > > The old non-incremental version of this patch series is still > available here: > > https://github.com/zmedico/portage/tree/bug_525718 > > Zac, while the code looks good, testing has showed a significant drop in performance for a -s search while it is greatly improves for a -S search. big_daddy portage # time emerge -s porthole !!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' !!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' [ Results for search key : porthole ] Searching... * app-portage/porthole Latest version available: 0.6.1-r3 Latest version installed: 0.6.1-r3 Size of files: 937 KiB Homepage: http://porthole.sourceforge.net Description: A GTK+-based frontend to Portage License: GPL-2 [ Applications found : 1 ] real 0m5.509s user 0m5.181s sys 0m0.321s big_daddy portage # time /usr/bin/emerge -s porthole !!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' !!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' Searching... [ Results for search key : porthole ] [ Applications found : 1 ] * app-portage/porthole Latest version available: 0.6.1-r3 Latest version installed: 0.6.1-r3 Size of files: 937 KiB Homepage: http://porthole.sourceforge.net Description: A GTK+-based frontend to Portage License: GPL-2 real 0m3.431s user 0m3.024s sys 0m0.401s big_daddy portage # emerge --version !!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' !!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' Portage 2.2.14_p28 (python 3.3.5-final-0, default/linux/amd64/13.0/desktop, gcc-4.8.3, glibc-2.19-r1, 3.16.1-gentoo x86_64) big_daddy portage # /usr/bin/emerge --version !!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' !!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' Portage 2.2.14 (python 3.3.5-final-0, default/linux/amd64/13.0/desktop, gcc-4.8.3, glibc-2.19-r1, 3.16.1-gentoo x86_64) big_daddy portage # time esearch porthole [ Results for search key : porthole ] [ Applications found : 1 ] * app-portage/porthole Latest version available: 0.6.1-r3 Latest version installed: 0.6.1-r3 Size of downloaded files: 936 kB Homepage: http://porthole.sourceforge.net Description: A GTK+-based frontend to Portage License: GPL-2 real 0m0.189s user 0m0.149s sys 0m0.038s big_daddy portage # ======================== times for -S search ======================== big_daddy portage # time /usr/bin/emerge -S porthole !!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' !!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' Searching... / * ERROR: app-portage/c-layman-9999::gentoo-guis failed (depend phase): * git.eclass could not be found by inherit() * * Call stack: * ebuild.sh, line 550: Called source '/home/brian/Dev/git/gentoo-guis/app-portage/c-layman/c-layman-9999.ebuild' * c-layman-9999.ebuild, line 9: Called inherit 'git' * ebuild.sh, line 257: Called die * The specific snippet of code: * [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" * * If you need support, post the output of `emerge --info '=app-portage/c-layman-9999::gentoo-guis'`, * the complete build log and the output of `emerge -pqv '=app-portage/c-layman-9999::gentoo-guis'`. * Working directory: '/usr/lib64/python3.3/site-packages' * S: '/var/tmp/portage/app-portage/c-layman-9999/work/c-layman-9999' emerge: search: aux_get() failed, skipping | * Manifest not found for '/home/brian/Dev/git/gentoo-guis/app-portage/ufed/ufed-9999.ebuild' | * ERROR: dev-vcs/git-bzr-9999::The-Pit failed (depend phase): * git.eclass could not be found by inherit() * * Call stack: * ebuild.sh, line 550: Called source '/usr/local/portage/dev-vcs/git-bzr/git-bzr-9999.ebuild' * git-bzr-9999.ebuild, line 4: Called inherit 'git' * ebuild.sh, line 257: Called die * The specific snippet of code: * [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" * * If you need support, post the output of `emerge --info '=dev-vcs/git-bzr-9999::The-Pit'`, * the complete build log and the output of `emerge -pqv '=dev-vcs/git-bzr-9999::The-Pit'`. * Working directory: '/usr/lib64/python3.3/site-packages' * S: '/var/tmp/portage/dev-vcs/git-bzr-9999/work/git-bzr-9999' emerge: search: aux_get() failed, skipping [ Results for search key : porthole ] [ Applications found : 1 ] * app-portage/porthole Latest version available: 0.6.1-r3 Latest version installed: 0.6.1-r3 Size of files: 937 KiB Homepage: http://porthole.sourceforge.net Description: A GTK+-based frontend to Portage License: GPL-2 real 1m15.121s user 1m1.664s sys 0m12.891s big_daddy portage # time emerge -S porthole !!! Repository 'vdr-devel' has sync-type attribute set to unsupported value: 'layman' !!! Repository 'wtk' has sync-type attribute set to unsupported value: 'layman' [ Results for search key : porthole ] Searching... * ERROR: app-portage/c-layman-9999::gentoo-guis failed (depend phase): * git.eclass could not be found by inherit() * * Call stack: * ebuild.sh, line 575: Called source '/home/brian/Dev/git/gentoo-guis/app-portage/c-layman/c-layman-9999.ebuild' * c-layman-9999.ebuild, line 9: Called inherit 'git' * ebuild.sh, line 257: Called die * The specific snippet of code: * [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" * * If you need support, post the output of `emerge --info '=app-portage/c-layman-9999::gentoo-guis'`, * the complete build log and the output of `emerge -pqv '=app-portage/c-layman-9999::gentoo-guis'`. * Working directory: '/home/brian/Dev/git/portage/pym' * S: '/var/tmp/portage/app-portage/c-layman-9999/work/c-layman-9999' emerge: search: aux_get() failed, skipping * app-portage/porthole Latest version available: 0.6.1-r3 Latest version installed: 0.6.1-r3 Size of files: 937 KiB Homepage: http://porthole.sourceforge.net Description: A GTK+-based frontend to Portage License: GPL-2 * ERROR: dev-vcs/git-bzr-9999::The-Pit failed (depend phase): * git.eclass could not be found by inherit() * * Call stack: * ebuild.sh, line 575: Called source '/usr/local/portage/dev-vcs/git-bzr/git-bzr-9999.ebuild' * git-bzr-9999.ebuild, line 4: Called inherit 'git' * ebuild.sh, line 257: Called die * The specific snippet of code: * [[ -z ${location} ]] && die "${1}.eclass could not be found by inherit()" * * If you need support, post the output of `emerge --info '=dev-vcs/git-bzr-9999::The-Pit'`, * the complete build log and the output of `emerge -pqv '=dev-vcs/git-bzr-9999::The-Pit'`. * Working directory: '/home/brian/Dev/git/portage/pym' * S: '/var/tmp/portage/dev-vcs/git-bzr-9999/work/git-bzr-9999' emerge: search: aux_get() failed, skipping [ Applications found : 1 ] real 0m12.515s user 0m11.795s sys 0m0.660s big_daddy portage # I know the above times are probably slowed by not having overlays indexed, but it is still a significant speedup. esearch is only marginally slower doing a -S search than a regular pkg-name search, typically just over 0.2 seconds The above was done with only the gentoo repo indexed, plus I need to clean out some old ebuilds in overlays. But I was shocked to see the normal -s searches going from 3.4s to 5.5s with your new index. Also I did not see a way to specify all repos to be index updated. It is I believe a requirement of this new system. Re-running is for each repo installed individually is something to be reserved for the new postsync() to do when it lands in master. -- Brian Dolbec <dolsen>
