Re: git-prompt.sh incompatible with non-basic global grep.patternType
Eric Sunshinewrites: >> Later when the grep API was used in revision graversal machinery, > > s/graversal/traversal/ Thanks. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
On Fri, Jul 22, 2016 at 3:21 PM, Junio C Hamanowrote: > Subject: [PATCH] grep: further simplify setting the pattern type > > When c5c31d33 (grep: move pattern-type bits support to top-level > grep.[ch], 2012-10-03) introduced grep_commit_pattern_type() helper > function, the intention was to allow the users of grep API to having > to fiddle only with .pattern_type_option (which can be set to "fixed", > "basic", "extended", and "pcre"), and then immediately before compiling > the pattern strings for use, call grep_commit_pattern_type() to have > it prepare various bits in the grep_opt structure (like .fixed, > .regflags, etc.). > > However, grep_set_pattern_type_option() helper function the grep API > internally uses were left as an external function by mistake. This > function shouldn't have been made callable by the users of the API. > > Later when the grep API was used in revision graversal machinery, s/graversal/traversal/ > the caller then mistakenly started calling the function around > 34a4ae55 (log --grep: use the same helper to set -E/-F options as > "git grep", 2012-10-03), instead of setting the .pattern_type_option > field and letting the grep_commit_pattern_type() to take care of the > details. > > This caused an unnecessary bug that made a configured > grep.patternType take precedence over the command line options > (e.g. --basic-regexp, --fixed-strings) in "git log" family of > commands. > > Signed-off-by: Junio C Hamano -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Jeff Kingwrites: > But even that is a lesser breakage than taking away grep.extendedRegexp > entirely. Taking it away breaks anybody who uses it; tweaking (2) only > breaks people who set both config variables. But why would anyone do > that? OK. Now we have an evidence that we have thought things through that we can point at when people later complain, I'd feel much safer to apply that "This will break t7810" patch, together with changes to t7810 to drop these crazy expectations ;-) -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
On Fri, Jul 22, 2016 at 12:51:00PM -0700, Junio C Hamano wrote: > Jeff Kingwrites: > > >> This comes from b22520a3 (grep: allow -E and -n to be turned on by > >> default via configuration, 2011-03-30) back when we didn't have a > >> more generic grep.patternType configuration mechanism in v1.7.5 > >> days, and it probably need to be deprecated to maintain our sanity. > >> ... > > I am not even sure we need to deprecate it. Once it becomes merely a > > historical synonym for "grep.patternType=extended" we can live with it > > indefinitely (and I do not think we need a deprecation period to go > > there; the existing behavior is simply buggy). > > I grossed over an important detail. > > Pretending as if grep.patternType=extended were given when we see > grep.extendedregexp=true and grep.patternType=basic is given when > grep.extendedregexp=false changes the behaviour in a way that can be > seen as the violation of (crazy) expectations t7810 makes. > > Any user who depends on that crazy expectation will be broken by > such a change, even if we do not deprecate and remove the > configuration variable. Ah. Reading 84befcd (grep: add a grep.patternType configuration setting, 2012-08-03) explains the rules, although I agree they are crazy (mostly because "basic" is different "fixed"). So I think there are two crazy things going on: 1. grep.extendedregexp takes precedence over the command-line (or at least "--basic" on the command line). 2. The weird rules in 84befcd, where patternType=fixed has higher precedence than extendedRegexp, but patternType=basic does not. It seems like (1) is a bug, and one we should not have to worry about compatibility while fixing. It stems from not telling the difference between "the user asked for nothing, so we defaulted to basic" and "the user explicitly asked for basic". I think (2) _is_ kind of crazy, and stems from similar confusion. I dunno. Part of me wants to say "I find it highly unlikely that anybody ever actually relied on this, and therefore it is OK to bring it closer to sanity without a deprecation period". But I admit that is just a gut feeling. But even that is a lesser breakage than taking away grep.extendedRegexp entirely. Taking it away breaks anybody who uses it; tweaking (2) only breaks people who set both config variables. But why would anyone do that? -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Jeff Kingwrites: >> This comes from b22520a3 (grep: allow -E and -n to be turned on by >> default via configuration, 2011-03-30) back when we didn't have a >> more generic grep.patternType configuration mechanism in v1.7.5 >> days, and it probably need to be deprecated to maintain our sanity. >> ... > I am not even sure we need to deprecate it. Once it becomes merely a > historical synonym for "grep.patternType=extended" we can live with it > indefinitely (and I do not think we need a deprecation period to go > there; the existing behavior is simply buggy). I grossed over an important detail. Pretending as if grep.patternType=extended were given when we see grep.extendedregexp=true and grep.patternType=basic is given when grep.extendedregexp=false changes the behaviour in a way that can be seen as the violation of (crazy) expectations t7810 makes. Any user who depends on that crazy expectation will be broken by such a change, even if we do not deprecate and remove the configuration variable. grep.c | 4 ++-- grep.h | 1 - 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/grep.c b/grep.c index 394c856..7b1d423 100644 --- a/grep.c +++ b/grep.c @@ -73,9 +73,9 @@ int grep_config(const char *var, const char *value, void *cb) if (!strcmp(var, "grep.extendedregexp")) { if (git_config_bool(var, value)) - opt->extended_regexp_option = 1; + opt->pattern_type_option = GREP_PATTERN_ERE; else - opt->extended_regexp_option = 0; + opt->pattern_type_option = GREP_PATTERN_BRE; return 0; } diff --git a/grep.h b/grep.h index cee4357..fc36c2a 100644 --- a/grep.h +++ b/grep.h @@ -119,7 +119,6 @@ struct grep_opt { int max_depth; int funcname; int funcbody; - int extended_regexp_option; int pattern_type_option; char color_context[COLOR_MAXLEN]; char color_filename[COLOR_MAXLEN]; -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
On Fri, Jul 22, 2016 at 12:21:31PM -0700, Junio C Hamano wrote: > One thing that I noticed is that there is this strange field > in grep_opt called .extended_regexp_option; it only is set from the > boolean configuration grep.extendedregexp and worse yet it takes > precedence over the command line option or grep.patternType, e.g. > t7810-grep expects crazy things like these: > > * "git -c grep.extendedregexp=true -c grep.patterntype=basic grep" >wants to be like "git grep -E" > > * "git -c grep.extendedregexp=false -c grep.patterntype=extended grep" >wants to be like "git grep -G" > > This comes from b22520a3 (grep: allow -E and -n to be turned on by > default via configuration, 2011-03-30) back when we didn't have a > more generic grep.patternType configuration mechanism in v1.7.5 > days, and it probably need to be deprecated to maintain our sanity. > I.e. when we see the configuration used, first we warn the user and > set grep.patternType to extended instead, and then eventually error > out in a backward-compatibility breaking release of Git we will make > in some future date, together with things like other compatibility > breaking topics like ex/deprecate-empty-pathspec-as-match-all. > > But that is a separate topic after this fix goes in anyway. I am not even sure we need to deprecate it. Once it becomes merely a historical synonym for "grep.patternType=extended" we can live with it indefinitely (and I do not think we need a deprecation period to go there; the existing behavior is simply buggy). Not that I mind eventually removing it, if you want to go through the steps. > -- >8 -- > From: Junio C Hamano> Date: Fri, 22 Jul 2016 11:43:14 -0700 > Subject: [PATCH] grep: further simplify setting the pattern type > [...] Thanks. This matches the cursory analysis I had done earlier, and the patch looks exactly as I had expected. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Jeff Kingwrites: > I gave a very cursory look when I wrote the other email, and your > alternative solution is what looked like the most elegant fix to me. > > I suspect this bug has been there quite a while, so no rush. :) Here is what I am going to queue. One thing that I noticed is that there is this strange field in grep_opt called .extended_regexp_option; it only is set from the boolean configuration grep.extendedregexp and worse yet it takes precedence over the command line option or grep.patternType, e.g. t7810-grep expects crazy things like these: * "git -c grep.extendedregexp=true -c grep.patterntype=basic grep" wants to be like "git grep -E" * "git -c grep.extendedregexp=false -c grep.patterntype=extended grep" wants to be like "git grep -G" This comes from b22520a3 (grep: allow -E and -n to be turned on by default via configuration, 2011-03-30) back when we didn't have a more generic grep.patternType configuration mechanism in v1.7.5 days, and it probably need to be deprecated to maintain our sanity. I.e. when we see the configuration used, first we warn the user and set grep.patternType to extended instead, and then eventually error out in a backward-compatibility breaking release of Git we will make in some future date, together with things like other compatibility breaking topics like ex/deprecate-empty-pathspec-as-match-all. But that is a separate topic after this fix goes in anyway. -- >8 -- From: Junio C Hamano Date: Fri, 22 Jul 2016 11:43:14 -0700 Subject: [PATCH] grep: further simplify setting the pattern type When c5c31d33 (grep: move pattern-type bits support to top-level grep.[ch], 2012-10-03) introduced grep_commit_pattern_type() helper function, the intention was to allow the users of grep API to having to fiddle only with .pattern_type_option (which can be set to "fixed", "basic", "extended", and "pcre"), and then immediately before compiling the pattern strings for use, call grep_commit_pattern_type() to have it prepare various bits in the grep_opt structure (like .fixed, .regflags, etc.). However, grep_set_pattern_type_option() helper function the grep API internally uses were left as an external function by mistake. This function shouldn't have been made callable by the users of the API. Later when the grep API was used in revision graversal machinery, the caller then mistakenly started calling the function around 34a4ae55 (log --grep: use the same helper to set -E/-F options as "git grep", 2012-10-03), instead of setting the .pattern_type_option field and letting the grep_commit_pattern_type() to take care of the details. This caused an unnecessary bug that made a configured grep.patternType take precedence over the command line options (e.g. --basic-regexp, --fixed-strings) in "git log" family of commands. Signed-off-by: Junio C Hamano --- grep.c | 22 +++--- grep.h | 1 - revision.c | 8 t/t4202-log.sh | 14 ++ 4 files changed, 29 insertions(+), 16 deletions(-) diff --git a/grep.c b/grep.c index b58c7c6..0bc5cc1 100644 --- a/grep.c +++ b/grep.c @@ -161,17 +161,7 @@ void grep_init(struct grep_opt *opt, const char *prefix) strcpy(opt->color_sep, def->color_sep); } -void grep_commit_pattern_type(enum grep_pattern_type pattern_type, struct grep_opt *opt) -{ - if (pattern_type != GREP_PATTERN_TYPE_UNSPECIFIED) - grep_set_pattern_type_option(pattern_type, opt); - else if (opt->pattern_type_option != GREP_PATTERN_TYPE_UNSPECIFIED) - grep_set_pattern_type_option(opt->pattern_type_option, opt); - else if (opt->extended_regexp_option) - grep_set_pattern_type_option(GREP_PATTERN_TYPE_ERE, opt); -} - -void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, struct grep_opt *opt) +static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, struct grep_opt *opt) { switch (pattern_type) { case GREP_PATTERN_TYPE_UNSPECIFIED: @@ -203,6 +193,16 @@ void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, struct gr } } +void grep_commit_pattern_type(enum grep_pattern_type pattern_type, struct grep_opt *opt) +{ + if (pattern_type != GREP_PATTERN_TYPE_UNSPECIFIED) + grep_set_pattern_type_option(pattern_type, opt); + else if (opt->pattern_type_option != GREP_PATTERN_TYPE_UNSPECIFIED) + grep_set_pattern_type_option(opt->pattern_type_option, opt); + else if (opt->extended_regexp_option) + grep_set_pattern_type_option(GREP_PATTERN_TYPE_ERE, opt); +} + static struct grep_pat *create_grep_pat(const char *pat, size_t patlen, const char *origin, int no, enum grep_pat_token t, diff --git a/grep.h b/grep.h index 95f197a..9e9ec5f 100644 --- a/grep.h +++ b/grep.h @@ -144,7 +144,6 @@
Re: git-prompt.sh incompatible with non-basic global grep.patternType
On Wed, Jul 20, 2016 at 01:10:42PM -0700, Junio C Hamano wrote: > This may fix it. I think the root cause is that logic to smear > "pattern type" into various broken-down fields in grep_opt for the > short-hands like --basic-regexp option needs to leave "I am setting > this short-hand" mark to allow the grep_commit_pattern_type() that > is done as the final step of the set-up sequence before we call > compile_grep_patterns() can take notice. The calls currently made > to grep_set_pattern_type_option() when we parse "--basic-regexp" and > friends forgets to override the "source of truth" field and only > updates the broken-down fields. > > An alternative may be to update places that parse "--basic-regexp" > and friends to just write to .pattern_type_option without calling > grep_set_pattern_type_option(); that might be a cleaner, but I am > not feeling well today so I won't be able to do a deeper analysis > right now. I gave a very cursory look when I wrote the other email, and your alternative solution is what looked like the most elegant fix to me. I suspect this bug has been there quite a while, so no rush. :) -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Jeff Kingwrites: > On Mon, Jul 18, 2016 at 03:56:09PM -0700, Richard Soderberg wrote: > >> ps. git log --basic-regexp does not fix the issue, as for unknown >> reasons (I'll start another thread) the command-line option doesn't >> override grep.patternType. > > Dscho gave a fix for your immediate issue, but this "ps" definitely > seems like a bug to me. Command-line options should always take > precedence over config. This may fix it. I think the root cause is that logic to smear "pattern type" into various broken-down fields in grep_opt for the short-hands like --basic-regexp option needs to leave "I am setting this short-hand" mark to allow the grep_commit_pattern_type() that is done as the final step of the set-up sequence before we call compile_grep_patterns() can take notice. The calls currently made to grep_set_pattern_type_option() when we parse "--basic-regexp" and friends forgets to override the "source of truth" field and only updates the broken-down fields. An alternative may be to update places that parse "--basic-regexp" and friends to just write to .pattern_type_option without calling grep_set_pattern_type_option(); that might be a cleaner, but I am not feeling well today so I won't be able to do a deeper analysis right now. grep.c | 1 + 1 file changed, 1 insertion(+) diff --git a/grep.c b/grep.c index 394c856..908ed3d 100644 --- a/grep.c +++ b/grep.c @@ -203,6 +203,7 @@ void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, struct gr opt->regflags &= ~REG_EXTENDED; break; } + opt->pattern_type_option = pattern_type; } static struct grep_pat *create_grep_pat(const char *pat, size_t patlen, -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
On Mon, Jul 18, 2016 at 03:56:09PM -0700, Richard Soderberg wrote: > ps. git log --basic-regexp does not fix the issue, as for unknown > reasons (I'll start another thread) the command-line option doesn't > override grep.patternType. Dscho gave a fix for your immediate issue, but this "ps" definitely seems like a bug to me. Command-line options should always take precedence over config. -Peff -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Hi Richard, On Tue, 19 Jul 2016, Richard Soderberg wrote: > Aha! Yes, this works precisely as intended: the prompt works > correctly, and quickly, with this change in place. Okay, next step for you: read http://github.com/git-for-windows/git/blob/master/Documentation/SubmittingPatches and submit this patch with an informative commit message. Ciao, Johannes -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Aha! Yes, this works precisely as intended: the prompt works correctly, and quickly, with this change in place. - R. On Tue, Jul 19, 2016 at 4:24 AM, Johannes Schindelinwrote: > Hi Richard, > > On Mon, 18 Jul 2016, Richard Soderberg wrote: > >> Hi, I wanted to report something interesting that I found while tracing >> a severe slowdown in git-prompt.sh. >> >> https://github.com/git/git/commit/6d158cba282f22fa1548af1188f78042fed30aed#diff-f37c4f4a898819f0ca4b5ff69e81d4d9R141 >> >> Way back in this commit, someone added a useful chunk of code that works >> perfectly with svn+ssh:// URLs under basic regexes: >> >> + local svn_upstream=($(git log --first-parent -1 \ + >> --grep="^git-svn-id: \(${svn_url_pattern:2}\)" 2>/dev/null)) >> >> However, if I switch over to Perl regexes (or Extended): >> >> git config --global grep.patternType perl >> >> Then the command runs for one wall clock second and shows incorrect >> results on my repository. I eventually traced this to an issue with the >> regular expression provided, assuming the svn repository url is >> "svn+ssh://...": >> >> git log ... --grep="^git-svn-id: \(svn+ssh://...\)" 2>/dev/null >> >> The + sign isn't escaped in git-prompt.sh, which under non-basic regexes >> causes the match to fail entirely. >> >> - R. >> >> ps. git log --basic-regexp does not fix the issue, as for unknown >> reasons (I'll start another thread) the command-line option doesn't >> override grep.patternType. > > Maybe this helps? > > -- snipsnap -- > diff --git a/contrib/completion/git-prompt.sh > b/contrib/completion/git-prompt.sh > index 97eacd7..74be907 100644 > --- a/contrib/completion/git-prompt.sh > +++ b/contrib/completion/git-prompt.sh > @@ -143,7 +143,7 @@ __git_ps1_show_upstream () > # get the upstream from the "git-svn-id: ..." in a commit > # message > # (git-svn uses essentially the same procedure internally) > local -a svn_upstream > - svn_upstream=($(git log --first-parent -1 \ > + svn_upstream=($(git -c grep.patternType=default log > --first-parent -1 \ > --grep="^git-svn-id: > \(${svn_url_pattern#??}\)" 2>/dev/null)) > if [[ 0 -ne ${#svn_upstream[@]} ]]; then > svn_upstream=${svn_upstream[${#svn_upstream[@]} - 2]} > -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git-prompt.sh incompatible with non-basic global grep.patternType
Hi Richard, On Mon, 18 Jul 2016, Richard Soderberg wrote: > Hi, I wanted to report something interesting that I found while tracing > a severe slowdown in git-prompt.sh. > > https://github.com/git/git/commit/6d158cba282f22fa1548af1188f78042fed30aed#diff-f37c4f4a898819f0ca4b5ff69e81d4d9R141 > > Way back in this commit, someone added a useful chunk of code that works > perfectly with svn+ssh:// URLs under basic regexes: > > + local svn_upstream=($(git log --first-parent -1 \ + > --grep="^git-svn-id: \(${svn_url_pattern:2}\)" 2>/dev/null)) > > However, if I switch over to Perl regexes (or Extended): > > git config --global grep.patternType perl > > Then the command runs for one wall clock second and shows incorrect > results on my repository. I eventually traced this to an issue with the > regular expression provided, assuming the svn repository url is > "svn+ssh://...": > > git log ... --grep="^git-svn-id: \(svn+ssh://...\)" 2>/dev/null > > The + sign isn't escaped in git-prompt.sh, which under non-basic regexes > causes the match to fail entirely. > > - R. > > ps. git log --basic-regexp does not fix the issue, as for unknown > reasons (I'll start another thread) the command-line option doesn't > override grep.patternType. Maybe this helps? -- snipsnap -- diff --git a/contrib/completion/git-prompt.sh b/contrib/completion/git-prompt.sh index 97eacd7..74be907 100644 --- a/contrib/completion/git-prompt.sh +++ b/contrib/completion/git-prompt.sh @@ -143,7 +143,7 @@ __git_ps1_show_upstream () # get the upstream from the "git-svn-id: ..." in a commit # message # (git-svn uses essentially the same procedure internally) local -a svn_upstream - svn_upstream=($(git log --first-parent -1 \ + svn_upstream=($(git -c grep.patternType=default log --first-parent -1 \ --grep="^git-svn-id: \(${svn_url_pattern#??}\)" 2>/dev/null)) if [[ 0 -ne ${#svn_upstream[@]} ]]; then svn_upstream=${svn_upstream[${#svn_upstream[@]} - 2]} -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
git-prompt.sh incompatible with non-basic global grep.patternType
Hi, I wanted to report something interesting that I found while tracing a severe slowdown in git-prompt.sh. https://github.com/git/git/commit/6d158cba282f22fa1548af1188f78042fed30aed#diff-f37c4f4a898819f0ca4b5ff69e81d4d9R141 Way back in this commit, someone added a useful chunk of code that works perfectly with svn+ssh:// URLs under basic regexes: + local svn_upstream=($(git log --first-parent -1 \ + --grep="^git-svn-id: \(${svn_url_pattern:2}\)" 2>/dev/null)) However, if I switch over to Perl regexes (or Extended): git config --global grep.patternType perl Then the command runs for one wall clock second and shows incorrect results on my repository. I eventually traced this to an issue with the regular expression provided, assuming the svn repository url is "svn+ssh://...": git log ... --grep="^git-svn-id: \(svn+ssh://...\)" 2>/dev/null The + sign isn't escaped in git-prompt.sh, which under non-basic regexes causes the match to fail entirely. - R. ps. git log --basic-regexp does not fix the issue, as for unknown reasons (I'll start another thread) the command-line option doesn't override grep.patternType. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html