Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On 2018-04-29 15:08, SZEDER Gábor wrote: On Sun, Apr 29, 2018 at 1:15 PM, Florian Gamböckwrote: I sense a problem here. If I have a directory with a file xyzfoobar in it, and I type `git xyz`, with no defined subcommand that starts with these letters, then minimal bashcomp would give me `git xyzfoobar`, which can of course not execute. This can be unintuitive for users, as in: "If it can't be executed correctly, then why does it even suggest such a completion?" I'm not sure I understand the problem. After 'git xyz' (note there is no space between 'xyz' and ) we try to complete the name of a git command, not options of a git command. This means: - At this point we don't look for a _git_xyz() function, so we'll return from __git_main() before even reaching the piece of code your patch modifies. - There are (presumably) no commands starting with 'xyz', so we don't list any commands. Bash will then fall back to its own filename completion, and that's where that 'xyzfoobar' will come from. It has been behaving like this basically since forever. And after 'git xyz ' (this time with space) we already complete the next word, not 'xyz'. You are absolutely right! I don't know what my brain was making up here, I am sorry. The minimal completion will come up regardless if no valid completion can be found. I think I mixed up the meaning of $cword in __git_main. It is correct, if I want to complete `git xyz`, then my patch is never reached. I think all you need to do is run a s/__load_completion/_completion_loader/ on your patch and update the commit message with relevant bits from the above discussion. I can do that, no problem. But prior to that I want to be sure that you are okay with the above mentioned drawback. Will the behavior be acceptable in this case? Or should we try to somehow "undo" the minimal completion afterwards? As explained above, I don't think there is any drawback here. Or at least not any new drawback that your patch is introducing. Or I'm completely missing your point; certainly a possibility, it's early Sunday afternoon, after all :) Okay, then I'll prepare the next round. Thank you very much for your helpful feedback!
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On Sun, Apr 29, 2018 at 1:15 PM, Florian Gamböckwrote: > On 2018-04-25 16:40, SZEDER Gábor wrote: >> >> In my previous emails I overlooked the _completion_loader() helper >> function. >> >> It seems that this function does almost exactly what we want. It was >> introduced along with dynamic completion loading back in 20c05b43, so it's >> available for us even in older LTS/Enterprise releases. Since cad3abfc it's >> a wrapper around __load_completion() and thus benefits from all the >> improvements, notably searching for completion scripts in a user-specified >> directory ($BASH_COMPLETION_USER_DIR) or in the user's home directory >> ($XDG_DATA_HOME or ~/.local/...) as well. It loads the matching completion >> script, but does not call the completion function unconditionally. > > > Sounds good so far. > >> The "almost" refers to he case when _completion_loader() can't find a >> completion script with a matching name to load, and then registers the >> _minimal() completion function for the given command to do basic path >> completion as fallback. I don't think this matters in practice, because in >> this case the given command is a git command in its dashed form, e.g. >> 'git-diff-index', and those have been deprecated for a long time. > > > I sense a problem here. If I have a directory with a file xyzfoobar in it, > and I type `git xyz`, with no defined subcommand that starts with these > letters, then minimal bashcomp would give me `git xyzfoobar`, which can of > course not execute. This can be unintuitive for users, as in: "If it can't > be executed correctly, then why does it even suggest such a completion?" I'm not sure I understand the problem. After 'git xyz' (note there is no space between 'xyz' and ) we try to complete the name of a git command, not options of a git command. This means: - At this point we don't look for a _git_xyz() function, so we'll return from __git_main() before even reaching the piece of code your patch modifies. - There are (presumably) no commands starting with 'xyz', so we don't list any commands. Bash will then fall back to its own filename completion, and that's where that 'xyzfoobar' will come from. It has been behaving like this basically since forever. And after 'git xyz ' (this time with space) we already complete the next word, not 'xyz'. >> I think all you need to do is run a >> s/__load_completion/_completion_loader/ on your patch and update the commit >> message with relevant bits from the above discussion. > > > I can do that, no problem. But prior to that I want to be sure that you are > okay with the above mentioned drawback. Will the behavior be acceptable in > this case? Or should we try to somehow "undo" the minimal completion > afterwards? As explained above, I don't think there is any drawback here. Or at least not any new drawback that your patch is introducing. Or I'm completely missing your point; certainly a possibility, it's early Sunday afternoon, after all :)
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On 2018-04-25 16:40, SZEDER Gábor wrote: In my previous emails I overlooked the _completion_loader() helper function. It seems that this function does almost exactly what we want. It was introduced along with dynamic completion loading back in 20c05b43, so it's available for us even in older LTS/Enterprise releases. Since cad3abfc it's a wrapper around __load_completion() and thus benefits from all the improvements, notably searching for completion scripts in a user-specified directory ($BASH_COMPLETION_USER_DIR) or in the user's home directory ($XDG_DATA_HOME or ~/.local/...) as well. It loads the matching completion script, but does not call the completion function unconditionally. Sounds good so far. The "almost" refers to he case when _completion_loader() can't find a completion script with a matching name to load, and then registers the _minimal() completion function for the given command to do basic path completion as fallback. I don't think this matters in practice, because in this case the given command is a git command in its dashed form, e.g. 'git-diff-index', and those have been deprecated for a long time. I sense a problem here. If I have a directory with a file xyzfoobar in it, and I type `git xyz`, with no defined subcommand that starts with these letters, then minimal bashcomp would give me `git xyzfoobar`, which can of course not execute. This can be unintuitive for users, as in: "If it can't be executed correctly, then why does it even suggest such a completion?" I think all you need to do is run a s/__load_completion/_completion_loader/ on your patch and update the commit message with relevant bits from the above discussion. I can do that, no problem. But prior to that I want to be sure that you are okay with the above mentioned drawback. Will the behavior be acceptable in this case? Or should we try to somehow "undo" the minimal completion afterwards?
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On Mon, Apr 23, 2018 at 5:12 PM, SZEDER Gáborwrote: > On Thu, Apr 19, 2018 at 9:07 PM, Florian Gamböck wrote: >> On 2018-04-18 21:51, SZEDER Gábor wrote: >>> I believe the main bash-completion repository can be found at: >>> >>> https://github.com/scop/bash-completion.git >>> >>> This repository still contains the branch 'dynamic-loading'; for the >>> record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2. >>> >>> Two commits on that branch are worth mentioning: >>> >>> 20c05b43 (Load completions in separate files dynamically, get rid of >>> have()., 2011-10-12) >>> 5baebf81 (Add _xfunc for loading and calling functions on demand, >>> use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13) I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion. >>> >>> This is wrong, __load_completion() was introduced in cad3abfc >>> (__load_completion: New function, use in _completion_loader and _xfunc, >>> 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03. >>> Would it be possible to use _xfunc() instead to plug that hole? It seems >>> the be tricky, because that function not only sources but also _calls_ the >>> completion function. >> >> But isn't this exactly what we want? > > No, that's definitely not what we want. In my previous emails I overlooked the _completion_loader() helper function. It seems that this function does almost exactly what we want. It was introduced along with dynamic completion loading back in 20c05b43, so it's available for us even in older LTS/Enterprise releases. Since cad3abfc it's a wrapper around __load_completion() and thus benefits from all the improvements, notably searching for completion scripts in a user-specified directory ($BASH_COMPLETION_USER_DIR) or in the user's home directory ($XDG_DATA_HOME or ~/.local/...) as well. It loads the matching completion script, but does not call the completion function unconditionally. The "almost" refers to he case when _completion_loader() can't find a completion script with a matching name to load, and then registers the _minimal() completion function for the given command to do basic path completion as fallback. I don't think this matters in practice, because in this case the given command is a git command in its dashed form, e.g. 'git-diff-index', and those have been deprecated for a long time. I think all you need to do is run a s/__load_completion/_completion_loader/ on your patch and update the commit message with relevant bits from the above discussion.
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On 2018-04-23 17:12, SZEDER Gábor wrote: On Thu, Apr 19, 2018 at 9:07 PM, Florian Gamböckwrote: On 2018-04-18 21:51, SZEDER Gábor wrote: Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function. But isn't this exactly what we want? No, that's definitely not what we want. The bash-completion project can get away with it, because they only use their _xfunc() to source a file they themselves ship and to call a completion function they know that that file contains. We, however, would like to load a file that might not exist and to call a function that might not be defined. Git has a lot of plumbing commands with neither a corresponding _git_plumbing_cmd() completion function in our completion script nor a corresponding 'git-plumbing-cmd' file that could be sourced dynamically to provide that function. The same applies to any 'git-foo' command in the user's $PATH (the user's own scripts or various git-related tools, e.g. Git LFS). So if someone tries e.g. 'git diff-index ' to complete files in the current directory, then it would result in an error message like _git_diff_index: command not found Furthermore, earlier versions of _xfunc(), I think until the introduction of __load_completion(), tried to source the file given as parameter without checking its existence beforehand, so on whatever LTS I happen to be currently using I would also get an error like bash: /usr/share/bash-completion/completions/git-diff-index: No such file or directory Finally, since all this is running in the user's interactive shell, Bash will even run the 'command_not_found_handle', if it's set (and most Linux distros do set it in their default configuration (OK, maybe not most, but at least some of the more popular do)), which may or may not have any suggestions, but at the very least it takes quite a while to scan through its database. You're right, this could be a problem. Then how about the following patch? This is one of my very first iterations of this patch, before even sending it to the mailing list. Actually this is similar to what the original _xfunc did, plus existence checking and minus premature function calling. In the directory of the git completion script, if there is an appropriate script for the current subcommand, if it is a regular file (or a valid symlink) and if it is readable, then source it and test the existence of the completion function again. Likewise for a possible alias. -- >8 -- diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash --- a/contrib/completion/git-completion.bash +++ b/contrib/completion/git-completion.bash @@ -2615,10 +2615,21 @@ __git_main () local completion_func="_git_${command//-/_}" declare -f $completion_func >/dev/null && $completion_func && return + local completion_dir="$(dirname ${BASH_SOURCE[0]})" + local completion_file="$completion_dir/git-$command" + [ -f "$completion_file" -a -r "$completion_file" ] && . "$completion_file" + declare -f $completion_func >/dev/null && $completion_func && return + local expansion=$(__git_aliased_command "$command") if [ -n "$expansion" ]; then words[1]=$expansion + completion_func="_git_${expansion//-/_}" + declare -f $completion_func >/dev/null && $completion_func && return + + completion_file="$completion_dir/git-$expansion" + [ -f "$completion_file" -a -r "$completion_file" ] && + . "$completion_file" declare -f $completion_func >/dev/null && $completion_func fi }
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On Thu, Apr 19, 2018 at 9:07 PM, Florian Gamböckwrote: > On 2018-04-18 21:51, SZEDER Gábor wrote: >> >> On Tue, Apr 10, 2018 at 10:28 PM, Florian Gamböck wrote: >>> >>> Adding external subcommands to Git is as easy as to put an executable >>> file git-foo into PATH. Packaging such subcommands for a Linux distribution >>> can be achieved by unpacking the executable into /usr/bin of the user's >>> system. Adding system-wide completion scripts for new subcommands, however, >>> can be a bit tricky. >>> >>> Since bash-completion started to use dynamical loading of completion >>> scripts since v1.90 (preview of v2.0), >> >> >> I believe the main bash-completion repository can be found at: >> >> https://github.com/scop/bash-completion.git >> >> This repository still contains the branch 'dynamic-loading'; for the >> record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2. >> >> Two commits on that branch are worth mentioning: >> >> 20c05b43 (Load completions in separate files dynamically, get rid of >> have()., 2011-10-12) >> 5baebf81 (Add _xfunc for loading and calling functions on demand, >> use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13) > > > Nice, thanks for the pointers! > >>> (...) >>> >>> I think the easiest method is to use a function that is defined by >>> bash-completion v2.0+, namely __load_completion. >> >> >> This is wrong, __load_completion() was introduced in cad3abfc >> (__load_completion: New function, use in _completion_loader and _xfunc, >> 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03. > > > Dang, I thought it was introduced at the same time. Sorry for that. I guess, > 2016 is a bit too young to take it for granted then? > >> The release tags '1.90' and '2.0' are from 2011-11-03 and 2012-06-17, >> respectively. This leaves a couple of years long hole where completions >> were already loaded dynamically but there was no __load_completion() >> function. >> >> Would it be possible to use _xfunc() instead to plug that hole? It seems >> the be tricky, because that function not only sources but also _calls_ the >> completion function. > > > But isn't this exactly what we want? No, that's definitely not what we want. The bash-completion project can get away with it, because they only use their _xfunc() to source a file they themselves ship and to call a completion function they know that that file contains. We, however, would like to load a file that might not exist and to call a function that might not be defined. Git has a lot of plumbing commands with neither a corresponding _git_plumbing_cmd() completion function in our completion script nor a corresponding 'git-plumbing-cmd' file that could be sourced dynamically to provide that function. The same applies to any 'git-foo' command in the user's $PATH (the user's own scripts or various git-related tools, e.g. Git LFS). So if someone tries e.g. 'git diff-index ' to complete files in the current directory, then it would result in an error message like _git_diff_index: command not found Furthermore, earlier versions of _xfunc(), I think until the introduction of __load_completion(), tried to source the file given as parameter without checking its existence beforehand, so on whatever LTS I happen to be currently using I would also get an error like bash: /usr/share/bash-completion/completions/git-diff-index: No such file or directory Finally, since all this is running in the user's interactive shell, Bash will even run the 'command_not_found_handle', if it's set (and most Linux distros do set it in their default configuration (OK, maybe not most, but at least some of the more popular do)), which may or may not have any suggestions, but at the very least it takes quite a while to scan through its database. > Lucky us, we can replace the whole > if-fi block with a simpler: > >_xfunc git-$command $completion_func 2>/dev/null && return > > If _xfunc is not defined -- as in, bashcomp is not installed / loaded -- > then the return will not get called and the original completion will > continue: > >declare -f $completion_func >/dev/null 2>/dev/null && >$completion_func && return > > Since this would be redundant, we could define a fall-back for _xfunc like > so: > >declare -f _xfunc || _xfunc() { >declare -f $completion_func >/dev/null 2>/dev/null && >$completion_func && return >} > > This way, we retain the "old" behavior and get dynamic loading if bashcomp > is available. The actual call to get the completions would just be _xfunc > like in my first example above. > > What do you think? > > -- > Regards > > Florian
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On 2018-04-18 21:51, SZEDER Gábor wrote: On Tue, Apr 10, 2018 at 10:28 PM, Florian Gamböckwrote: Adding external subcommands to Git is as easy as to put an executable file git-foo into PATH. Packaging such subcommands for a Linux distribution can be achieved by unpacking the executable into /usr/bin of the user's system. Adding system-wide completion scripts for new subcommands, however, can be a bit tricky. Since bash-completion started to use dynamical loading of completion scripts since v1.90 (preview of v2.0), I believe the main bash-completion repository can be found at: https://github.com/scop/bash-completion.git This repository still contains the branch 'dynamic-loading'; for the record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2. Two commits on that branch are worth mentioning: 20c05b43 (Load completions in separate files dynamically, get rid of have()., 2011-10-12) 5baebf81 (Add _xfunc for loading and calling functions on demand, use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13) Nice, thanks for the pointers! (...) I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion. This is wrong, __load_completion() was introduced in cad3abfc (__load_completion: New function, use in _completion_loader and _xfunc, 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03. Dang, I thought it was introduced at the same time. Sorry for that. I guess, 2016 is a bit too young to take it for granted then? The release tags '1.90' and '2.0' are from 2011-11-03 and 2012-06-17, respectively. This leaves a couple of years long hole where completions were already loaded dynamically but there was no __load_completion() function. Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function. But isn't this exactly what we want? Lucky us, we can replace the whole if-fi block with a simpler: _xfunc git-$command $completion_func 2>/dev/null && return If _xfunc is not defined -- as in, bashcomp is not installed / loaded -- then the return will not get called and the original completion will continue: declare -f $completion_func >/dev/null 2>/dev/null && $completion_func && return Since this would be redundant, we could define a fall-back for _xfunc like so: declare -f _xfunc || _xfunc() { declare -f $completion_func >/dev/null 2>/dev/null && $completion_func && return } This way, we retain the "old" behavior and get dynamic loading if bashcomp is available. The actual call to get the completions would just be _xfunc like in my first example above. What do you think? -- Regards Florian
Re: [PATCH v2 1/1] completion: load completion file for external subcommand
On Tue, Apr 10, 2018 at 10:28 PM, Florian Gamböckwrote: > Adding external subcommands to Git is as easy as to put an executable > file git-foo into PATH. Packaging such subcommands for a Linux > distribution can be achieved by unpacking the executable into /usr/bin > of the user's system. Adding system-wide completion scripts for new > subcommands, however, can be a bit tricky. > > Since bash-completion started to use dynamical loading of completion > scripts since v1.90 (preview of v2.0), I believe the main bash-completion repository can be found at: https://github.com/scop/bash-completion.git This repository still contains the branch 'dynamic-loading'; for the record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2. Two commits on that branch are worth mentioning: 20c05b43 (Load completions in separate files dynamically, get rid of have()., 2011-10-12) 5baebf81 (Add _xfunc for loading and calling functions on demand, use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13) > it is no longer sufficient to > drop a completion script of a subcommand into the standard completions > path, /usr/share/bash-completion/completions, since this script will not > be loaded if called as a git subcommand. > > For example, look at https://bugs.gentoo.org/544722. To give a short > summary: The popular git-flow subcommand provides a completion script, > which gets installed as /usr/share/bash-completion/completions/git-flow. > > If you now type into a Bash shell: > > git flow > > You will not get any completions, because bash-completion only loads > completions for git and git has no idea that git-flow is defined in > another file. You have to load this script manually or trigger the > dynamic loader with: > > git-flow # Please notice the dash instead of whitespace > > This will not complete anything either, because it only defines a Bash > function, without generating completions. But now the correct completion > script has been loaded and the first command can use the completions. > > So, the goal is now to teach the git completion script to consider the > possibility of external completion scripts for subcommands, but of > course without breaking current workflows. > > I think the easiest method is to use a function that is defined by > bash-completion v2.0+, namely __load_completion. This is wrong, __load_completion() was introduced in cad3abfc (__load_completion: New function, use in _completion_loader and _xfunc, 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03. The release tags '1.90' and '2.0' are from 2011-11-03 and 2012-06-17, respectively. This leaves a couple of years long hole where completions were already loaded dynamically but there was no __load_completion() function. Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function. > It will take care of > loading the correct script if present. Afterwards, the git completion > script behaves as usual. > > This way we can leverage bash-completion's dynamic loading for git > subcommands and make it easier for developers to distribute custom > completion scripts. > > Signed-off-by: Florian Gamböck > --- > contrib/completion/git-completion.bash | 10 ++ > 1 file changed, 10 insertions(+) > > diff --git a/contrib/completion/git-completion.bash > b/contrib/completion/git-completion.bash > index b09c8a236..09a820990 100644 > --- a/contrib/completion/git-completion.bash > +++ b/contrib/completion/git-completion.bash > @@ -3096,12 +3096,22 @@ __git_main () > fi > > local completion_func="_git_${command//-/_}" > + if ! declare -f $completion_func >/dev/null 2>/dev/null && > + declare -f __load_completion >/dev/null 2>/dev/null > + then > + __load_completion "git-$command" > + fi > declare -f $completion_func >/dev/null 2>/dev/null && > $completion_func && return > > local expansion=$(__git_aliased_command "$command") > if [ -n "$expansion" ]; then > words[1]=$expansion > completion_func="_git_${expansion//-/_}" > + if ! declare -f $completion_func >/dev/null 2>/dev/null && > + declare -f __load_completion >/dev/null 2>/dev/null > + then > + __load_completion "git-$expansion" > + fi > declare -f $completion_func >/dev/null 2>/dev/null && > $completion_func > fi > } > -- > 2.16.1 >
[PATCH v2 1/1] completion: load completion file for external subcommand
Adding external subcommands to Git is as easy as to put an executable file git-foo into PATH. Packaging such subcommands for a Linux distribution can be achieved by unpacking the executable into /usr/bin of the user's system. Adding system-wide completion scripts for new subcommands, however, can be a bit tricky. Since bash-completion started to use dynamical loading of completion scripts since v1.90 (preview of v2.0), it is no longer sufficient to drop a completion script of a subcommand into the standard completions path, /usr/share/bash-completion/completions, since this script will not be loaded if called as a git subcommand. For example, look at https://bugs.gentoo.org/544722. To give a short summary: The popular git-flow subcommand provides a completion script, which gets installed as /usr/share/bash-completion/completions/git-flow. If you now type into a Bash shell: git flow You will not get any completions, because bash-completion only loads completions for git and git has no idea that git-flow is defined in another file. You have to load this script manually or trigger the dynamic loader with: git-flow # Please notice the dash instead of whitespace This will not complete anything either, because it only defines a Bash function, without generating completions. But now the correct completion script has been loaded and the first command can use the completions. So, the goal is now to teach the git completion script to consider the possibility of external completion scripts for subcommands, but of course without breaking current workflows. I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion. It will take care of loading the correct script if present. Afterwards, the git completion script behaves as usual. This way we can leverage bash-completion's dynamic loading for git subcommands and make it easier for developers to distribute custom completion scripts. Signed-off-by: Florian Gamböck--- contrib/completion/git-completion.bash | 10 ++ 1 file changed, 10 insertions(+) diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash index b09c8a236..09a820990 100644 --- a/contrib/completion/git-completion.bash +++ b/contrib/completion/git-completion.bash @@ -3096,12 +3096,22 @@ __git_main () fi local completion_func="_git_${command//-/_}" + if ! declare -f $completion_func >/dev/null 2>/dev/null && + declare -f __load_completion >/dev/null 2>/dev/null + then + __load_completion "git-$command" + fi declare -f $completion_func >/dev/null 2>/dev/null && $completion_func && return local expansion=$(__git_aliased_command "$command") if [ -n "$expansion" ]; then words[1]=$expansion completion_func="_git_${expansion//-/_}" + if ! declare -f $completion_func >/dev/null 2>/dev/null && + declare -f __load_completion >/dev/null 2>/dev/null + then + __load_completion "git-$expansion" + fi declare -f $completion_func >/dev/null 2>/dev/null && $completion_func fi } -- 2.16.1