On Tue, Sep 26, 2023, 10:24 Chet Ramey <[email protected]> wrote: > > On 9/25/23 7:02 PM, Karl O. Pinc wrote: > > On Mon, 25 Sep 2023 18:35:53 +0200 > > "Stefan H. Holek" <[email protected]> wrote: > > > >> I am however not sure supporting different character sets is still > >> useful in 2023. Everything is UTF-8 these days, and your code is fine > >> if NFD/NFC is the only problem we have to solve. > > > > I am not following the details here but I've noticed some > > cloud providers provision C.UTF-8 by default. This may > > not be relevant, but is certainly ASCII-like so I thought > > you might want to know. > > Right now, this code is only active on macOS, which doesn't yet support > C.UTF-8 (at least as of Ventura).
I think the issue Stefan brings to light is that, while the filename rewrite hook installed by Bash is a noop other than on macOS, Readline will now apply any existing rewrite hook to the completion word, which existing hooks (if such exist) are likely not designed to deal with. E.g., if one already had a rewrite hook to convert UTF-8 file system names for matching (in as much as that is possible) against ISO-8859-1 terminal text, applying that same hook now to terminal input as well will produce garbage results. Not sure if this is kind of hook is actually part of something people are still building against new Readline versions but I would feel bad breaking backwards compatibility needlessly so here's a patch to split up the hook that gets applied to the filename portion of the input from the hook that gets applied to directory entries read from the file system, in case that seems reasonable to you.
From aa63aa0120d0875849f8b79ac0426312abeb2be9 Mon Sep 17 00:00:00 2001 From: Grisha Levit <[email protected]> Date: Tue, 26 Sep 2023 03:35:26 -0400 Subject: [PATCH] split rl_filename_{rewrite,completion}_hook Allow applying different transformations to completion word and directory entries prior to comparing them in filename completion, instead of using the same transformation for both. --- bashline.c | 1 + lib/readline/complete.c | 24 ++++++++++++++++++------ lib/readline/doc/rltech.texi | 31 +++++++++++++++++++------------ lib/readline/readline.h | 18 +++++++++++++++--- 4 files changed, 53 insertions(+), 21 deletions(-) diff --git a/bashline.c b/bashline.c index 2c0ffaa8..4930ed4a 100644 --- a/bashline.c +++ b/bashline.c @@ -3338,6 +3338,7 @@ bashline_set_filename_hooks (void) set_directory_hook (); rl_filename_rewrite_hook = bash_filename_rewrite_hook; + rl_filename_completion_hook = bash_filename_rewrite_hook; rl_filename_stat_hook = bash_filename_stat_hook; } diff --git a/lib/readline/complete.c b/lib/readline/complete.c index 9f170334..bf227fa6 100644 --- a/lib/readline/complete.c +++ b/lib/readline/complete.c @@ -246,11 +246,23 @@ rl_icppfunc_t *rl_filename_stat_hook = (rl_icppfunc_t *)NULL; either return its first argument (if no conversion takes place) or newly-allocated memory. This can, for instance, convert filenames between character sets for comparison against what's typed at the - keyboard. The returned value is what is added to the list of - matches. The second argument is the length of the filename to be - converted. */ + keyboard (after its potential modification by rl_filename_completion_hook). + The returned value is what is added to the list of matches. + The second argument is the length of the filename to be converted. */ rl_dequote_func_t *rl_filename_rewrite_hook = (rl_dequote_func_t *)NULL; +/* If non-zero, this is the address of a function to call before + comparing a partial word to be completed with directory entries from + the filesystem. This takes the address of the partial word to be + completed, after any rl_filename_dequoting_function has been applied. + The function should either return its first argument (if no conversion + takes place) or newly-allocated memory. This can, for instance, convert + the completion word to a character set suitable for comparison against + directory entries read from the filesystem (after their potential + modification by rl_filename_rewrite_hook). The second argument is the + length of the text to be converted. */ +rl_dequote_func_t *rl_filename_completion_hook = (rl_dequote_func_t *)NULL; + /* Non-zero means readline completion functions perform tilde expansion. */ int rl_complete_with_tilde_expansion = 0; @@ -2536,10 +2548,10 @@ rl_filename_completion_function (const char *text, int state) } filename_len = strlen (filename); - /* Normalize the filename if the application has set a rewrite hook. */ - if (*filename && rl_filename_rewrite_hook) + /* Convert the filename if the application has set a hook for that. */ + if (*filename && rl_filename_completion_hook) { - temp = (*rl_filename_rewrite_hook) (filename, filename_len); + temp = (*rl_filename_completion_hook) (filename, filename_len); if (temp != filename) { xfree (filename); diff --git a/lib/readline/doc/rltech.texi b/lib/readline/doc/rltech.texi index 3e136789..630d4165 100644 --- a/lib/readline/doc/rltech.texi +++ b/lib/readline/doc/rltech.texi @@ -2165,18 +2165,25 @@ The function should not modify the directory argument if it returns 0. @deftypevar {rl_dequote_func_t *} rl_filename_rewrite_hook If non-zero, this is the address of a function called when reading directory entries from the filesystem for completion and comparing -them to the partial word to be completed. The function should -perform any necessary application or system-specific conversion on -the filename, such as converting between character sets or converting -from a filesystem format to a character input format. -The function takes two arguments: @var{fname}, the filename to be converted, -and @var{fnlen}, its length in bytes. -It must either return its first argument (if no conversion takes place) -or the converted filename in newly-allocated memory. The converted -form is used to compare against the word to be completed, and, if it -matches, is added to the list of matches. Readline will free the -allocated string. -@end deftypevar +them to the partial word to be completed (after its potential modification +by rl_filename_completion_hook). The function should perform any necessary +application or system-specific conversion on the filename, such as converting +between character sets or converting from a filesystem format to a character +input format. The function takes two arguments: @var{fname}, the filename +to be converted, and @var{fnlen}, its length in bytes. +@end deftypevar + +@deftypevar {rl_dequote_func_t} *rl_filename_completion_hook +If non-zero, this is the address of a function to call before comparing +a partial word to be completed with directory entries from the filesystem. +The function takes two arguments: @var{fname}, the dequoted partial +filename portion of the word being completed, and @var{fnlen}, its length +in bytes. It must either return its first argument (if no conversion takes +place) or the converted text in newly-allocated memory. The function should +perform any necessary application or system-specific conversion on the text, +such that the converted form can be used to compare against directory entries +read from the filesystem (after their potential modification by +rl_filename_rewrite_hook). Readline will free the allocated string. @deftypevar {rl_compdisp_func_t *} rl_completion_display_matches_hook If non-zero, then this is the address of a function to call when diff --git a/lib/readline/readline.h b/lib/readline/readline.h index 0810f97d..b9bf62d7 100644 --- a/lib/readline/readline.h +++ b/lib/readline/readline.h @@ -766,11 +766,23 @@ extern rl_icppfunc_t *rl_filename_stat_hook; either return its first argument (if no conversion takes place) or newly-allocated memory. This can, for instance, convert filenames between character sets for comparison against what's typed at the - keyboard. The returned value is what is added to the list of - matches. The second argument is the length of the filename to be - converted. */ + keyboard (after its potential modification by rl_filename_completion_hook). + The returned value is what is added to the list of matches. + The second argument is the length of the filename to be converted. */ extern rl_dequote_func_t *rl_filename_rewrite_hook; +/* If non-zero, this is the address of a function to call before + comparing a partial word to be completed with directory entries from + the filesystem. This takes the address of the partial word to be + completed, after any rl_filename_dequoting_function has been applied. + The function should either return its first argument (if no conversion + takes place) or newly-allocated memory. This can, for instance, convert + the completion word to a character set suitable for comparison against + directory entries read from the filesystem (after their potential + modification by rl_filename_rewrite_hook). The second argument is the + length of the text to be converted. */ +extern rl_dequote_func_t *rl_filename_completion_hook; + /* Backwards compatibility with previous versions of readline. */ #define rl_symbolic_link_hook rl_directory_completion_hook -- 2.42.0
