Re: Inconsistent regex behaviour in 7.4.383

2014-07-30 Fir de Conversatie Bram Moolenaar

Tim Brosnan wrote:

 I initially send a version of this message through the google groups 
 interface, but it seems to have disappeared. Apologies if there is 
 duplication.
 
 When going through some regex examples from 
 VimRegexTutor(https://github.com/dahu/VimRegexTutor) I found that one of 
 the examples did not match the provided text when run in 7.4.383. Here 
 is the regex and the sample text:
 
   *30* : Find Text between HTML tags 
 
   /\%(\1\)\@=.*\%(\/\(\w\+\)\)\@=
 
   OR
 
   /\v%(\\1\)@=.*%(\\/(\w+)\)@=
 
   levelCan I play, daddy?/level
 
 Neither of the provided regexs would watch the sample text (it should 
 match the text between html tags). However, when I tried 7.1.42 (only 
 other version at hand) I found that the text between tags was matched 
 correctly. Both versions were fun as -u NONE and set nocompatible to 
 try to rule out option or plugin clashes.
 
 Is this is a bug or has regex behaviour changed between 7.1.42 and 7.4.383?

I don't think the pattern is supposed to work.  The backreference is
before the capturing group.  It's a suprise it works at all.  Probably
because in the old regex engine the \@= part is used only after
finding a match for the rest.

It's a very inefficient pattern too, better use:

   /\(\w\+\).\{-}\zs.*\ze\/\1


-- 
I once paid $12 to peer at the box that held King Tutankhamen's little
bandage-covered midget corpse at the De Young Museum in San Francisco.  I
remember thinking how pleased he'd be about the way things turned out in his
afterlife.
(Scott Adams - The Dilbert principle)

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Inconsistent regex behaviour in 7.4.383

2014-07-30 Fir de Conversatie Ingo Karkat
On 30-Jul-2014 12:53 +0200, Bram Moolenaar wrote:

 Tim Brosnan wrote:
 
 I initially send a version of this message through the google groups 
 interface, but it seems to have disappeared. Apologies if there is 
 duplication.

 When going through some regex examples from 
 VimRegexTutor(https://github.com/dahu/VimRegexTutor) I found that one of 
 the examples did not match the provided text when run in 7.4.383. Here 
 is the regex and the sample text:

   *30* : Find Text between HTML tags 

   /\%(\1\)\@=.*\%(\/\(\w\+\)\)\@=

   OR

   /\v%(\\1\)@=.*%(\\/(\w+)\)@=
 
   levelCan I play, daddy?/level

 Neither of the provided regexs would watch the sample text (it should 
 match the text between html tags). However, when I tried 7.1.42 (only 
 other version at hand) I found that the text between tags was matched 
 correctly. Both versions were fun as -u NONE and set nocompatible to 
 try to rule out option or plugin clashes.

 Is this is a bug or has regex behaviour changed between 7.1.42 and 7.4.383?
 
 I don't think the pattern is supposed to work.  The backreference is
 before the capturing group.

Well, it does work in the old engine.

 It's a suprise it works at all.  Probably because in the old regex
 engine the \@= part is used only after finding a match for the
 rest.
 
 It's a very inefficient pattern too, better use:
 
/\(\w\+\).\{-}\zs.*\ze\/\1

You cannot always use \zs in place of \@= (for example, with :syntax
match, or when already using a \zs elsewhere in the pattern).

Just changing the example in the help isn't going to solve the problem
of backwards compatibility. I think the new engine either has to support
it, or detect that special situation and then dynamically switch to the
old engine (like \%#=1 does).

-- regards, ingo

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Inconsistent regex behaviour in 7.4.383

2014-07-30 Fir de Conversatie Ben Fritz
On Wednesday, July 30, 2014 6:58:11 AM UTC-5, Ingo Karkat wrote:
  It's a suprise it works at all.  Probably because in the old regex
 
  engine the \@= part is used only after finding a match for the
 
  rest.
 
  
 

The help for /\@= pretty explicitly says it should work this way, and why:

The part of the pattern after \@= and \@! are checked for a
match first, thus things like \1 don't work to reference \(\) inside
the preceding atom.  It does work the other way around:
Example matches ~
\1\@=,\([a-z]\+\)  ,abc in abc,abc

It's weird but it's too late now. So I agree with Ingo: either the regex engine 
must be switched dynamically to use the old engine by default when an undefined 
backreference is used, or the new regex engine should support it in the same 
order.

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Inconsistent regex behaviour in 7.4.383

2014-07-29 Fir de Conversatie Tim Brosnan
I initially send a version of this message through the google groups 
interface, but it seems to have disappeared. Apologies if there is 
duplication.


When going through some regex examples from 
VimRegexTutor(https://github.com/dahu/VimRegexTutor) I found that one of 
the examples did not match the provided text when run in 7.4.383. Here 
is the regex and the sample text:


 *30* : Find Text between HTML tags 

 /\%(\1\)\@=.*\%(\/\(\w\+\)\)\@=

 OR

 /\v%(\\1\)@=.*%(\\/(\w+)\)@=

 levelCan I play, daddy?/level

Neither of the provided regexs would watch the sample text (it should 
match the text between html tags). However, when I tried 7.1.42 (only 
other version at hand) I found that the text between tags was matched 
correctly. Both versions were fun as -u NONE and set nocompatible to 
try to rule out option or plugin clashes.


Is this is a bug or has regex behaviour changed between 7.1.42 and 7.4.383?

--
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups vim_dev group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Inconsistent regex behaviour in 7.4.383

2014-07-29 Fir de Conversatie Ben Fritz
On Tuesday, July 29, 2014 3:10:56 PM UTC-5, Tim Brosnan wrote:
 I initially send a version of this message through the google groups 
 interface, but it seems to have disappeared. Apologies if there is 
 duplication.
 
 When going through some regex examples from 
 VimRegexTutor(https://github.com/dahu/VimRegexTutor) I found that one of 
 the examples did not match the provided text when run in 7.4.383. Here 
 is the regex and the sample text:
 
   *30* : Find Text between HTML tags 
 
   /\%(\1\)\@=.*\%(\/\(\w\+\)\)\@=
 
   OR
 
   /\v%(\\1\)@=.*%(\\/(\w+)\)@=
 
   levelCan I play, daddy?/level
 
 Neither of the provided regexs would watch the sample text (it should 
 match the text between html tags). However, when I tried 7.1.42 (only 
 other version at hand) I found that the text between tags was matched 
 correctly. Both versions were fun as -u NONE and set nocompatible to 
 try to rule out option or plugin clashes.
 
 Is this is a bug or has regex behaviour changed between 7.1.42 and 7.4.383?

I think this is the same issue reported here:

  https://groups.google.com/d/topic/vim_dev/07qFhT34nEI/discussion

Probably it is caused by the new regular expression engine introduced
in 7.4. See :help new-regexp-engine and :help two-engines for more
details. The new engine is designed in a way that backreferences are
really difficult. In the thread I link above Bram calls it a miracle
that they work at all.

:help 'regexpengine' provides a possible workaround for you until this
gets fixed. Try doing :set regexpengine=1 to see if that fixes it for
you temporarily until the problem is fixed.

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Inconsistent regex behaviour in 7.4.383

2014-07-29 Fir de Conversatie timothy . brosnan1
On Tuesday, July 29, 2014 9:20:02 PM UTC+1, Ben Fritz wrote:
 On Tuesday, July 29, 2014 3:10:56 PM UTC-5, Tim Brosnan wrote:
 
  I initially send a version of this message through the google groups 
 
  interface, but it seems to have disappeared. Apologies if there is 
 
  duplication.
 
  
 
  When going through some regex examples from 
 
  VimRegexTutor(https://github.com/dahu/VimRegexTutor) I found that one of 
 
  the examples did not match the provided text when run in 7.4.383. Here 
 
  is the regex and the sample text:
 
  
 
*30* : Find Text between HTML tags 
 
  
 
/\%(\1\)\@=.*\%(\/\(\w\+\)\)\@=
 
  
 
OR
 
  
 
/\v%(\\1\)@=.*%(\\/(\w+)\)@=
 
  
 
levelCan I play, daddy?/level
 
  
 
  Neither of the provided regexs would watch the sample text (it should 
 
  match the text between html tags). However, when I tried 7.1.42 (only 
 
  other version at hand) I found that the text between tags was matched 
 
  correctly. Both versions were fun as -u NONE and set nocompatible to 
 
  try to rule out option or plugin clashes.
 
  
 
  Is this is a bug or has regex behaviour changed between 7.1.42 and 7.4.383?
 
 
 
 I think this is the same issue reported here:
 
 
 
   https://groups.google.com/d/topic/vim_dev/07qFhT34nEI/discussion
 
 
 
 Probably it is caused by the new regular expression engine introduced
 
 in 7.4. See :help new-regexp-engine and :help two-engines for more
 
 details. The new engine is designed in a way that backreferences are
 
 really difficult. In the thread I link above Bram calls it a miracle
 
 that they work at all.
 
 
 
 :help 'regexpengine' provides a possible workaround for you until this
 
 gets fixed. Try doing :set regexpengine=1 to see if that fixes it for
 
 you temporarily until the problem is fixed.

Ah, it seems like this is the problem. :set regexpengine=1 gives the correct 
behaviour.

I didn't know about version7.txt, it certainly makes tracking differences 
between updates easier.

Thanks for your help.

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.