Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Petr Pisar writes: > On Sat, May 16, 2020 at 12:56:06PM +0200, Dominique Martinet wrote: > >> How should I go about with that? Open bz bugs to all the packages I >> listed? strongly suggesting to get things to move to /usr/share (17) >> and flatpak (suggest some kind of cache? not sure they'll be >> interested...) and environment-modules (not sure what to suggest >> there, I only have environment-modules because I need to test >> something with openmpi from time to time and it comes with it...) >> >> It might also make sense to have a packaging guideline suggesting to >> avoid /etc/bash_completion.d in favor of the /usr/share variant, I >> couldn't find anything here[1] but I might not have looked thoroughly >> enough... [1] >> https://docs.fedoraproject.org/en-US/packaging-guidelines/ > > When you are at fixing a packaging of the completion scripts, please > make the dependency on bash-completion optional (Recommends: > bash-completion). It used to be a good habit to weaken the dependency > because not everyone is willing to pay the slow-down tax if he has no > use of the completion: I would suggest that at the point where you're sacrificing shell friendliness in favor of performance, it might be a good time to switch over to dash https://src.fedoraproject.org/rpms/dash Thanks, --Robbie signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
On Mon, May 18, 2020 at 6:29 AM Dominique Martinet wrote: > > On the pagure side, I assume the process is something along the lines of: > - fedpkg clone pkg && cd pkg > - modify things > - fedpkg fork > - push in fork > - got create PR in the web interface? > I would think you should fork first, otherwise aren't you making changes to the original repo? Amazingly I've been using the src.fp.o fork option, I wasn't even aware that there was a fork option in fedpkg. I need to play around with it. Does it fork in place or create a new directory? Thanks, Richard ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Zbigniew Jędrzejewski-Szmek wrote on Mon, May 18, 2020: > > > How should I go about with that? Open bz bugs to all the packages I > > > listed? > > > > I would suggest to directly create pull requests on pagure (with a > > reference to this thread), as that will make it more likely that this > > change will actually happen. > > +1 I feel like I'm being dumped the work wholesale... :) This unfortunately might not be that simple, I've just checked from the first I'm most likely to use (perf) and the install path is decided by the kernel makefiles. The kernel ships three completion scripts, two have been updated to install to /usr/share/bash-completion/completions by default (so bpftool will just be a matter of NOT setting bash_compdir in the specfile and just a spec update) but perf will need a kernel patch first. I believe this was not fluke the first project I looked that does it that way, there really is some work required... I don't mind going through it one item at a time, including the kernel patch that will break packaging anyway, but this really isn't as simple as pagure PRs, it might be possible to just 'mv' the file there after install but that's not a good solution. On the pagure side, I assume the process is something along the lines of: - fedpkg clone pkg && cd pkg - modify things - fedpkg fork - push in fork - got create PR in the web interface? I'll work on it as time permits... > Please also consider submitting a PR for the packaging guidelines. > I think this is a change we generally agree on, and a PR should make > the whole process faster. This file ? https://pagure.io/packaging-committee/blob/master/f/guidelines/modules/ROOT/pages/index.adoc Should it be something specific about bash-completion, or something more generic about avoiding files in etc when possible? To give a few examples: - /etc/systemd/system -> /usr/lib/systemd/system - /etc/udev/rules.d -> /usr/lib/udev/rules.d - /etc/tmpfiles.d -> /usr/lib/tmpfiles.d - /etc/bash_completion.d -> /usr/share/bash-completion/completions I'm sure there are more and don't see anything about any of these... Oh actually tmpfiles.d is mentionned but not about avoiding /etc, and all the others have a macro (%{_unitdir}, %{_udevrulesdir} and %{_tmpfilesdir} respectively) so can be considered as 'autodocumented' ? > Also: > $ repoquery --whatprovides '/etc/bash_completion.d/*' --qf '%{name}' Well... I'll start with what I have on my system as the egoist person I am :) Thanks for the full list though. -- Dominique ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
On Sat, May 16, 2020 at 01:35:52PM +0200, Dan Čermák wrote: > Hi Dominique, > > Dominique Martinet writes: > > > *snip* > > > > 341 to 130ms is a good start I guess, the rest of the waiting time > > probably now outweights bash and will get some looking at at a later > > point, but might as well start somewhere. > > That's quite the improvement! Good job and thanks for looking into that! > > > > > How should I go about with that? Open bz bugs to all the packages I > > listed? > > I would suggest to directly create pull requests on pagure (with a > reference to this thread), as that will make it more likely that this > change will actually happen. +1 > > strongly suggesting to get things to move to /usr/share (17) and > > flatpak (suggest some kind of cache? not sure they'll be > > interested...) and environment-modules (not sure what to suggest > > there, I only have environment-modules because I need to test > > something with openmpi from time to time and it comes with it...) > > > > > > It might also make sense to have a packaging guideline suggesting to > > avoid /etc/bash_completion.d in favor of the /usr/share variant, I > > couldn't find anything here[1] but I might not have looked thoroughly > > enough... > > Afaik we don't have an entry in the packaging guidelines about bash > completion files. Maybe the packaging committee should look into that? Please also consider submitting a PR for the packaging guidelines. I think this is a change we generally agree on, and a PR should make the whole process faster. Also: $ repoquery --whatprovides '/etc/bash_completion.d/*' --qf '%{name}' Last metadata expiration check: 0:01:09 ago on Mon 18 May 2020 12:42:34 PM CEST. RBTools abrt-tui authselect bash-completion beesu boinc-client bpftool bti cargo cekit-bash-completion ceph-common clusterssh composer-cli condor dahdi-tools dbus-glib-devel drbd-bash-completion drush epix-bash-completion fcoe-utils freeipa-client fzf gdal git-extras glusterfs-cli gmic icedtea-web iprutils kdocker koschei-admin ledger lilv lizardfs-client lldpad lnst-ctl lnst-slave lttng-tools lyx-common monotone mpc openscap-scanner openvswitch origin origin-clients pass-otp pass-pwned pdc-client perf perl-App-Cme perl-Config-Model-Itself perl-Dist-Zilla phoronix-test-suite pmempool publican python3-argcomplete python3-catkin_lint python3-cinderclient python3-doit python3-glanceclient python3-heatclient python3-manilaclient python3-mistralclient python3-novaclient python3-wstool quilt ratools reptyr salt scl-utils sheepdog singularity stgit tarsnap-bash-completion tmt topgit torsocks trace-cmd tracer-common tuxpaint udisks vrms-rpm xen-runtime Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
On Sat, May 16, 2020 at 12:56:06PM +0200, Dominique Martinet wrote: > How should I go about with that? Open bz bugs to all the packages I > listed? strongly suggesting to get things to move to /usr/share (17) and > flatpak (suggest some kind of cache? not sure they'll be interested...) > and environment-modules (not sure what to suggest there, I only have > environment-modules because I need to test something with openmpi from > time to time and it comes with it...) > > > It might also make sense to have a packaging guideline suggesting to > avoid /etc/bash_completion.d in favor of the /usr/share variant, I > couldn't find anything here[1] but I might not have looked thoroughly > enough... > [1] https://docs.fedoraproject.org/en-US/packaging-guidelines/ > When you are at fixing a packaging of the completion scripts, please make the dependency on bash-completion optional (Recommends: bash-completion). It used to be a good habit to weaken the dependency because not everyone is willing to pay the slow-down tax if he has no use of the completion: # dnf repoquery --whatrequires 'bash-completion' | grep -v bash-completion frama-c-0:20.0-3.fc33.x86_64 hashcat-0:5.1.0-7.fc33.i686 hashcat-0:5.1.0-7.fc33.x86_64 kiwi-cli-0:9.20.5-1.fc33.noarch perl-Config-Model-Itself-0:2.020-2.fc32.noarch snapd-0:2.43.3-1.fc33.x86_64 toolbox-experience-0:0.0.18-3.fc33.noarch votca-csg-bash-0:1.6-1.fc33.noarch -- Petr signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Samuel Sieb wrote on Sat, May 16, 2020: > On 5/16/20 3:56 AM, Dominique Martinet wrote: >> - down to 0.130s after moving /etc/bash_completion.d/* to >> /usr/share/bash-completion/completions/ > > I thought that generally, the /etc versions of config directories > were intended for the purpose of local overrides of the /usr/share > versions. Well that is for sure where I would install my own completion scripts not in rpms, similarily to how just about everything else works with /etc vs /usr (systemd, udev and friends at least) >> This one is not actually a no-op: bash-completion loads things from /etc >> at shell startup time, but things in /usr at first tab time, so if the >> file in /usr/share is not named by the same prefix as the command it >> help completes it won't work anymore, but in most case here it will >> still work just the same (slightly slower on first use) > > That's a strange separation of functionality. Maybe that's why some > of them are in there? The usual "solution" in this case for products that want to maintain a single script is just to add symlinks to the main one, for example see lvm: lvchange lvcreate lvdisplay etc etc all symlink to 'lvm' in /usr/share/bash-completion/completions. Although in this case very few would need to make one, in the full list above they almost all complete a single command so just renaming the file for some would be enough. In the list I gave, the only two exceptions are lilv which completes two commands so would need one link, and fzf which overrides the existing completion for kill so I guess that one has a valid reason to stay in /etc even if I hadn't noticed until now... -- Dominique ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
On 5/16/20 3:56 AM, Dominique Martinet wrote: - down to 0.130s after moving /etc/bash_completion.d/* to /usr/share/bash-completion/completions/ I thought that generally, the /etc versions of config directories were intended for the purpose of local overrides of the /usr/share versions. This one is not actually a no-op: bash-completion loads things from /etc at shell startup time, but things in /usr at first tab time, so if the file in /usr/share is not named by the same prefix as the command it help completes it won't work anymore, but in most case here it will still work just the same (slightly slower on first use) That's a strange separation of functionality. Maybe that's why some of them are in there? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Hi Dominique, Dominique Martinet writes: > *snip* > > 341 to 130ms is a good start I guess, the rest of the waiting time > probably now outweights bash and will get some looking at at a later > point, but might as well start somewhere. That's quite the improvement! Good job and thanks for looking into that! > > How should I go about with that? Open bz bugs to all the packages I > listed? I would suggest to directly create pull requests on pagure (with a reference to this thread), as that will make it more likely that this change will actually happen. > strongly suggesting to get things to move to /usr/share (17) and > flatpak (suggest some kind of cache? not sure they'll be > interested...) and environment-modules (not sure what to suggest > there, I only have environment-modules because I need to test > something with openmpi from time to time and it comes with it...) > > > It might also make sense to have a packaging guideline suggesting to > avoid /etc/bash_completion.d in favor of the /usr/share variant, I > couldn't find anything here[1] but I might not have looked thoroughly > enough... Afaik we don't have an entry in the packaging guidelines about bash completion files. Maybe the packaging committee should look into that? Cheers, Dan signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Vascom wrote on Sat, May 16, 2020: > Are you use SSD? > > I have 0.087s on it. Yes, I have a SSD. The bottleneck here really is cpu speed; I have my laptop's frequency limited to 1GHz in normal circumstances (when not building code or similar activities) so the fan doesn't turn up With max (4.6GHz) clocking the whole time is a bit less drastic, but still worth more than half the time: going from 0.097s to 0.037s I'd rather not use "I have a performant machine" as an excuse, though - my next laptop will likely be based on a low power ARM soc and I might not be that aggressive on throttling since it won't have a fan in the first place, but it will likely be much slower than this :P That's precisely the kind of thinking that makes everything feel just as slow today than they were 20 years ago despite having much more powerful computers... -- Dominique ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Are you use SSD? I have 0.087s on it. сб, 16 мая 2020 г., 13:57 Dominique Martinet : > Hi, > > once in a while I get annoyed at how slow bash is to startup; using a > tiling wm poping a new shell is supposed to be quite fast but I'm > staring at a new empty window for ~1s everytime and it gets annoying... > > According to https://xkcd.com/1205/ wasting 1s 50+ times a day is worth > spending some time to try to improve it so let's see what we can do :) > > > Here's the baseline on my machine: > $ time bash -l < /dev/null > > real0m0.341s > user0m0.198s > sys 0m0.146s > > > And a few low hanging fruits I could find adding `-x 2>&1 | ts "%.s"`: > - down to 0.288s after removing /etc/profile.d/flatpak.sh > (flatpak-1.6.3-1.fc32.x86_64) > > - down to 0.225s after removing /etc/profile.d/modules.sh > (environment-modules-4.4.1-2.fc32.x86_64) > > - down to 0.130s after moving /etc/bash_completion.d/* to > /usr/share/bash-completion/completions/ > This one is not actually a no-op: bash-completion loads things from /etc > at shell startup time, but things in /usr at first tab time, so if the > file in /usr/share is not named by the same prefix as the command it > help completes it won't work anymore, but in most case here it will > still work just the same (slightly slower on first use) > > Here's the list of files I had in there and their packages: > authselect-completion.sh authselect-1.2.1-1.fc32.x86_64 > fcoeadm fcoe-utils-1.0.32-9.git9834b34.fc31.x86_64 > javaws.bash icedtea-web-2.0.0-pre.0.2.alpha13.patched1.fc32.x86_64 > perf perf-5.6.7-300.fc32.x86_64 > trace-cmd.bash trace-cmd-2.8.3-1.fc32.x86_64 > bpftool bpftool-5.6.7-300.fc32.x86_64 > fcoemon fcoe-utils-1.0.32-9.git9834b34.fc31.x86_64 > policyeditor.bash icedtea-web-2.0.0-pre.0.2.alpha13.patched1.fc32.x86_64 > xl.sh xen-runtime-4.13.0-7.fc32.x86_64 > cargo cargo-1.43.1-1.fc32.x86_64 > fzf fzf-0.21.1-1.fc32.x86_64 > lilv lilv-0.24.6-2.fc32.x86_64 > python-argcomplete.sh python3-argcomplete-1.10.0-4.fc32.noarch > dbus-bash-completion.sh dbus-glib-devel-0.110-7.fc32.x86_64 > gluster glusterfs-cli-7.5-1.fc32.x86_64 > lldpad lldpad-1.0.1-16.git036e314.fc32.x86_64 > redefine_filedir bash-completion-2.8-8.fc32.noarch > dog sheepdog-1.0.1-10.fc31.x86_64 > itweb-settings.bash icedtea-web-2.0.0-pre.0.2.alpha13.patched1.fc32.x86_64 > lldptool lldpad-1.0.1-16.git036e314.fc32.x86_64 > torsocks torsocks-2.3.0-5.fc32.x86_64 > > > 341 to 130ms is a good start I guess, the rest of the waiting time > probably now outweights bash and will get some looking at at a later > point, but might as well start somewhere. > > How should I go about with that? Open bz bugs to all the packages I > listed? strongly suggesting to get things to move to /usr/share (17) and > flatpak (suggest some kind of cache? not sure they'll be interested...) > and environment-modules (not sure what to suggest there, I only have > environment-modules because I need to test something with openmpi from > time to time and it comes with it...) > > > It might also make sense to have a packaging guideline suggesting to > avoid /etc/bash_completion.d in favor of the /usr/share variant, I > couldn't find anything here[1] but I might not have looked thoroughly > enough... > [1] https://docs.fedoraproject.org/en-US/packaging-guidelines/ > > > Would anyone be willing to help, something is telling me that doing this > alone would take more time than what I'd save in the end, but a few > people considering it'll help everyone might be ;) > > > Thanks! > -- > Dominique > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Speed up bash starting time (slow scripts in profile.d, /etc/bash_completion.d)
Hi, once in a while I get annoyed at how slow bash is to startup; using a tiling wm poping a new shell is supposed to be quite fast but I'm staring at a new empty window for ~1s everytime and it gets annoying... According to https://xkcd.com/1205/ wasting 1s 50+ times a day is worth spending some time to try to improve it so let's see what we can do :) Here's the baseline on my machine: $ time bash -l < /dev/null real0m0.341s user0m0.198s sys 0m0.146s And a few low hanging fruits I could find adding `-x 2>&1 | ts "%.s"`: - down to 0.288s after removing /etc/profile.d/flatpak.sh (flatpak-1.6.3-1.fc32.x86_64) - down to 0.225s after removing /etc/profile.d/modules.sh (environment-modules-4.4.1-2.fc32.x86_64) - down to 0.130s after moving /etc/bash_completion.d/* to /usr/share/bash-completion/completions/ This one is not actually a no-op: bash-completion loads things from /etc at shell startup time, but things in /usr at first tab time, so if the file in /usr/share is not named by the same prefix as the command it help completes it won't work anymore, but in most case here it will still work just the same (slightly slower on first use) Here's the list of files I had in there and their packages: authselect-completion.sh authselect-1.2.1-1.fc32.x86_64 fcoeadm fcoe-utils-1.0.32-9.git9834b34.fc31.x86_64 javaws.bash icedtea-web-2.0.0-pre.0.2.alpha13.patched1.fc32.x86_64 perf perf-5.6.7-300.fc32.x86_64 trace-cmd.bash trace-cmd-2.8.3-1.fc32.x86_64 bpftool bpftool-5.6.7-300.fc32.x86_64 fcoemon fcoe-utils-1.0.32-9.git9834b34.fc31.x86_64 policyeditor.bash icedtea-web-2.0.0-pre.0.2.alpha13.patched1.fc32.x86_64 xl.sh xen-runtime-4.13.0-7.fc32.x86_64 cargo cargo-1.43.1-1.fc32.x86_64 fzf fzf-0.21.1-1.fc32.x86_64 lilv lilv-0.24.6-2.fc32.x86_64 python-argcomplete.sh python3-argcomplete-1.10.0-4.fc32.noarch dbus-bash-completion.sh dbus-glib-devel-0.110-7.fc32.x86_64 gluster glusterfs-cli-7.5-1.fc32.x86_64 lldpad lldpad-1.0.1-16.git036e314.fc32.x86_64 redefine_filedir bash-completion-2.8-8.fc32.noarch dog sheepdog-1.0.1-10.fc31.x86_64 itweb-settings.bash icedtea-web-2.0.0-pre.0.2.alpha13.patched1.fc32.x86_64 lldptool lldpad-1.0.1-16.git036e314.fc32.x86_64 torsocks torsocks-2.3.0-5.fc32.x86_64 341 to 130ms is a good start I guess, the rest of the waiting time probably now outweights bash and will get some looking at at a later point, but might as well start somewhere. How should I go about with that? Open bz bugs to all the packages I listed? strongly suggesting to get things to move to /usr/share (17) and flatpak (suggest some kind of cache? not sure they'll be interested...) and environment-modules (not sure what to suggest there, I only have environment-modules because I need to test something with openmpi from time to time and it comes with it...) It might also make sense to have a packaging guideline suggesting to avoid /etc/bash_completion.d in favor of the /usr/share variant, I couldn't find anything here[1] but I might not have looked thoroughly enough... [1] https://docs.fedoraproject.org/en-US/packaging-guidelines/ Would anyone be willing to help, something is telling me that doing this alone would take more time than what I'd save in the end, but a few people considering it'll help everyone might be ;) Thanks! -- Dominique ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org