Re: Capture columns nummers of matches ending with double byte chars

[Bram Moolenaar]

Ken Takata wrote:

> (Cc-ing to vim_dev)
> 
> Hi,
> 
> 2016/4/26 Tue 15:00:33 UTC+9 rameo wrote:
> > > 
> > > It is much better to download sources from https://github.com/vim/vim.
> > > 
> > 
> > Found it:
> > https://github.com/vim/vim-win32-installer/releases
> > 
> > However...
> > Tried:
> > gvim_7.4.1782_x86.exe
> > gvim_7.4.1786_x86.exe
> > 
> > Both gives an error: 
> > Error detected while processing vimrc_example.vim
> > line 114:
> > E919: Directory not found in 'packpath': "pack/*/opt/matchit"
> 
> Oops!
> We should have updated the nsis script when we turned some scripts
> into packages.  Attached patch should fix the problem.

Thanks!  Took a while before someone noticed...

-- 
TALL KNIGHT: We are now no longer the Knights Who Say Ni!
ONE KNIGHT:  Ni!
OTHERS:  Sh!
ONE KNIGHT:  (whispers) Sorry.
 "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[Ken Takata]

(Cc-ing to vim_dev)

Hi,

2016/4/26 Tue 15:00:33 UTC+9 rameo wrote:
> > 
> > It is much better to download sources from https://github.com/vim/vim.
> > 
> 
> Found it:
> https://github.com/vim/vim-win32-installer/releases
> 
> However...
> Tried:
> gvim_7.4.1782_x86.exe
> gvim_7.4.1786_x86.exe
> 
> Both gives an error: 
> Error detected while processing vimrc_example.vim
> line 114:
> E919: Directory not found in 'packpath': "pack/*/opt/matchit"

Oops!
We should have updated the nsis script when we turned some scripts into 
packages.
Attached patch should fix the problem.

Regards,
Ken Takata

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
# HG changeset patch
# Parent  7dd3dce73b35f7846b891f59a2215cc59ffb0557

diff --git a/nsis/gvim.nsi b/nsis/gvim.nsi
--- a/nsis/gvim.nsi
+++ b/nsis/gvim.nsi
@@ -215,6 +215,28 @@ Section "Vim executables and runtime fil
 	SetOutPath $0\macros
 	File ${VIMRT}\macros\*.*
 
+	SetOutPath $0\pack\dist\opt\dvorak\dvorak
+	File ${VIMRT}\pack\dist\opt\dvorak\dvorak\*.*
+	SetOutPath $0\pack\dist\opt\dvorak\plugin
+	File ${VIMRT}\pack\dist\opt\dvorak\plugin\*.*
+
+	SetOutPath $0\pack\dist\opt\editexisting\plugin
+	File ${VIMRT}\pack\dist\opt\editexisting\plugin\*.*
+
+	SetOutPath $0\pack\dist\opt\justify\plugin
+	File ${VIMRT}\pack\dist\opt\justify\plugin\*.*
+
+	SetOutPath $0\pack\dist\opt\matchit\doc
+	File ${VIMRT}\pack\dist\opt\matchit\doc\*.*
+	SetOutPath $0\pack\dist\opt\matchit\plugin
+	File ${VIMRT}\pack\dist\opt\matchit\plugin\*.*
+
+	SetOutPath $0\pack\dist\opt\shellmenu\plugin
+	File ${VIMRT}\pack\dist\opt\shellmenu\plugin\*.*
+
+	SetOutPath $0\pack\dist\opt\swapmouse\plugin
+	File ${VIMRT}\pack\dist\opt\swapmouse\plugin\*.*
+
 	SetOutPath $0\plugin
 	File ${VIMRT}\plugin\*.*
 

Re: Capture columns nummers of matches ending with double byte chars

[rameo]

Op dinsdag 26 april 2016 09:14:52 UTC+2 schreef rameo:
> Op dinsdag 26 april 2016 08:14:33 UTC+2 schreef Christian Brabandt:
> > Hi rameo!
> > 
> > On Mo, 25 Apr 2016, rameo wrote:
> > 
> > > 
> > > > 
> > > > It is much better to download sources from https://github.com/vim/vim.
> > > > 
> > > 
> > > Found it:
> > > https://github.com/vim/vim-win32-installer/releases
> > > 
> > > However...
> > > Tried:
> > > gvim_7.4.1782_x86.exe
> > > gvim_7.4.1786_x86.exe
> > > 
> > > Both gives an error: 
> > > Error detected while processing vimrc_example.vim
> > > line 114:
> > > E919: Directory not found in 'packpath': "pack/*/opt/matchit"
> > 
> > Yes, your $VIMRUNTIME does not include the pack directory.
> > So, you need to install a new runtime. Check the zip file, it includes 
> > an updated runtime directory.
> > 
> > 
> > Best,
> > Christian
> > -- 
> > Falsche Vorstellung, dass man ein Phänomen durch Kalkül oder 
> > durch Worte abtun und beseitigen könne.
> > -- Goethe, Maximen und Reflektionen, Nr. 1004
> 
> I just commented out 'packadd matchit', now it works.
> 
> Thank you very much Christian for these new features.
> 
> Does matchstrpos not capture all matches on line?
> 
>for n in range(0, len(LinewithMatches)-1)
>  let s = getline(LinewithMatches[n])
>  let r= matchstrpos(s, @/)
>  echo s
>  echo r
>endfor
> 
> It only returns the first match on the 1st line.
> Did I do something wrong?

Found it, I had to add a startposition and adapt the counter in a while loop.

just an idea: add all matches found in entire string to a list
[['match1',2,7]['match2',12,17]..etc]

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[rameo]

Op dinsdag 26 april 2016 08:14:33 UTC+2 schreef Christian Brabandt:
> Hi rameo!
> 
> On Mo, 25 Apr 2016, rameo wrote:
> 
> > 
> > > 
> > > It is much better to download sources from https://github.com/vim/vim.
> > > 
> > 
> > Found it:
> > https://github.com/vim/vim-win32-installer/releases
> > 
> > However...
> > Tried:
> > gvim_7.4.1782_x86.exe
> > gvim_7.4.1786_x86.exe
> > 
> > Both gives an error: 
> > Error detected while processing vimrc_example.vim
> > line 114:
> > E919: Directory not found in 'packpath': "pack/*/opt/matchit"
> 
> Yes, your $VIMRUNTIME does not include the pack directory.
> So, you need to install a new runtime. Check the zip file, it includes 
> an updated runtime directory.
> 
> 
> Best,
> Christian
> -- 
> Falsche Vorstellung, dass man ein Phänomen durch Kalkül oder 
> durch Worte abtun und beseitigen könne.
>   -- Goethe, Maximen und Reflektionen, Nr. 1004

I just commented out 'packadd matchit', now it works.

Thank you very much Christian for these new features.

Does matchstrpos not capture all matches on line?

   for n in range(0, len(LinewithMatches)-1)
 let s = getline(LinewithMatches[n])
 let r= matchstrpos(s, @/)
 echo s
 echo r
   endfor

It only returns the first match on the 1st line.
Did I do something wrong?

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[Christian Brabandt]

Hi rameo!

On Mo, 25 Apr 2016, rameo wrote:

> 
> > 
> > It is much better to download sources from https://github.com/vim/vim.
> > 
> 
> Found it:
> https://github.com/vim/vim-win32-installer/releases
> 
> However...
> Tried:
> gvim_7.4.1782_x86.exe
> gvim_7.4.1786_x86.exe
> 
> Both gives an error: 
> Error detected while processing vimrc_example.vim
> line 114:
> E919: Directory not found in 'packpath': "pack/*/opt/matchit"

Yes, your $VIMRUNTIME does not include the pack directory.
So, you need to install a new runtime. Check the zip file, it includes 
an updated runtime directory.


Best,
Christian
-- 
Falsche Vorstellung, dass man ein Phänomen durch Kalkül oder 
durch Worte abtun und beseitigen könne.
-- Goethe, Maximen und Reflektionen, Nr. 1004

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[rameo]

> 
> It is much better to download sources from https://github.com/vim/vim.
> 

Found it:
https://github.com/vim/vim-win32-installer/releases

However...
Tried:
gvim_7.4.1782_x86.exe
gvim_7.4.1786_x86.exe

Both gives an error: 
Error detected while processing vimrc_example.vim
line 114:
E919: Directory not found in 'packpath': "pack/*/opt/matchit"

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[Nikolay Aleksandrovich Pavlov]

2016-04-26 0:45 GMT+03:00 rameo :
> Op maandag 25 april 2016 22:26:27 UTC+2 schreef Ken Takata:
>> Hi rameo,
>>
>> 2016/4/23 Sat 21:37:15 UTC+9 rameo wrote:
>> > Searchpos() doesn't return the right end value of a match if the match end 
>> > with a double byte character (èéòìùá...). (encoding utf-8)
>> > Isn't this a bug
>>
>> You might misunderstand the spec of searchpos().
>> When the 'e' flag is specified, the cursor moves to the last character of
>> the match and searchpos() returns the position of the cursor.  It means
>> that the returned value is a byte index of start of the last character,
>> not the end of the character.
>>
>
> Hello Ken,
>
> thank you for your reply :)
>
> Just a question, why does someone need the byte index of the start of the 
> last character and not simply the end of the last character?
>
>>
>> > The last few years I used match/matchend then I noted that it did not 
>> > capture correctly double byte characters within the string.
>>
>> Can you show an example?
>> match()/matchend() return a byte offset, not a character count.
>>
>> FYI, matchstrpos() can be used to get both start and end position
>> after 7.4.1685.
>>
>>
>> Regards,
>> Ken Takata
>
> That's great news.
> I waited long for such a feature.
>
> Can you please send me the url of the site where the patches can be 
> downloaded?
> I can't find it anymore.

It is much better to download sources from https://github.com/vim/vim.

>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups 
> "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to vim_use+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[rameo]

Op maandag 25 april 2016 22:26:27 UTC+2 schreef Ken Takata:
> Hi rameo,
> 
> 2016/4/23 Sat 21:37:15 UTC+9 rameo wrote:
> > Searchpos() doesn't return the right end value of a match if the match end 
> > with a double byte character (èéòìùá...). (encoding utf-8)
> > Isn't this a bug
> 
> You might misunderstand the spec of searchpos().
> When the 'e' flag is specified, the cursor moves to the last character of
> the match and searchpos() returns the position of the cursor.  It means
> that the returned value is a byte index of start of the last character,
> not the end of the character.
> 

Hello Ken,

thank you for your reply :)

Just a question, why does someone need the byte index of the start of the last 
character and not simply the end of the last character?

> 
> > The last few years I used match/matchend then I noted that it did not 
> > capture correctly double byte characters within the string.
> 
> Can you show an example?
> match()/matchend() return a byte offset, not a character count.
> 
> FYI, matchstrpos() can be used to get both start and end position
> after 7.4.1685.
> 
> 
> Regards,
> Ken Takata

That's great news.
I waited long for such a feature.

Can you please send me the url of the site where the patches can be downloaded?
I can't find it anymore.

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[Ken Takata]

Hi rameo,

2016/4/23 Sat 21:37:15 UTC+9 rameo wrote:
> Searchpos() doesn't return the right end value of a match if the match end 
> with a double byte character (èéòìùá...). (encoding utf-8)
> Isn't this a bug

You might misunderstand the spec of searchpos().
When the 'e' flag is specified, the cursor moves to the last character of
the match and searchpos() returns the position of the cursor.  It means
that the returned value is a byte index of start of the last character,
not the end of the character.


> The last few years I used match/matchend then I noted that it did not capture 
> correctly double byte characters within the string.

Can you show an example?
match()/matchend() return a byte offset, not a character count.

FYI, matchstrpos() can be used to get both start and end position
after 7.4.1685.


Regards,
Ken Takata

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[Nikolay Aleksandrovich Pavlov]

2016-04-21 22:03 GMT+03:00 rameo :
> Since I use Vim I have troubles with double byte characters.
>
> I want to capture all strings of matches together with startcolumn and 
> endcolumn of a match (line by line). I don't need only the strings but also 
> the columnnumbers for other functions.
>
> The last few years I used match/matchend then I noted that it did not capture 
> correctly double byte characters within the string.
> Then I adapted everything to use searchpos() but today I found out that it 
> gives troubles with string with a double byte character at the end.
>
>
>"mylist = list with all linenrs having matches
>for n in range(0, len(mylist)-1)
>  let idx  = []
>  let edx  = []
>  let matches_between_cols = []
>
>  "FIND ALL IDX MATCHES
>  "idx --> forward search
>  call cursor(mylist[n],1)
>  while line(".") == mylist[n]
>let S= searchpos(@/, '')
>if S[0] == mylist[n]
>  call add(idx, S[1]-1)
>endif
>  endwhile
>  "idx --> backward search (to include matches on first column)
>  call cursor(mylist[n],len(getline(mylist[n])))
>  while line(".") == mylist[n]
>let S= searchpos(@/, 'b')
>if S[0] == mylist[n]
>  call add(idx, S[1]-1)
>endif
>  endwhile
>
>  "FIND ALL EDX MATCHES
>  "edx --> forward search
>  call cursor(mylist[n],1)
>  while line(".") == mylist[n]
>let E= searchpos(@/, 'e')
>if E[0] == mylist[n]
>  call add(edx, E[1])
>endif
>  endwhile
>  "edx --> backward search (to include matches on first column)
>  call cursor(mylist[n],len(getline(mylist[n])))
>  while line(".") == mylist[n]
>let E= searchpos(@/, 'eb')
>if E[0] == mylist[n]
>  call add(edx, E[1])
>endif
>  endwhile
>
>  if len(idx) > 0
>for i in range(0,len(idx)-1)
> let r = strpart(getline(mylist[n]),idx[i], edx[i]-idx[i])
> call add(matches_between_cols, r)
>endfor
>  endif
>endfor
>
> ---
> Buffer:
> city | Felicità
> whatever | Peach
> pmg0001 | Perché
> text| Céline
> bMgbXuEWo | Université
>
>
> @/ = "| \zs\S\+"
> it captures:
> Felicit
> Peach
> Perch
> Céline
> Universit
>
> Expected:
> Felicità
> Peach
> Perché
> Céline
> Université
>
> Can you please tell me what I did wrong?

I cannot say what you did wrong, but calling `searchpos()` multiple
times for each occurence of a pattern is rather wasty. Check how I did
this in [formatvim][1], I collect matches there to highlight them
later, so it gets start and end positions of the match.

[1]: 
https://bitbucket.org/ZyX_I/formatvim/src/a00edc4c7032bde5c7e970bca7871e9317ee2265/autoload/format.vim#format.vim-1457

> (Is it not possible to let every character be a single byte char as in 
> languages as Python?)

This was already discussed many times. No, it is not: backward
compatibility, though there are special functions (useless because
column is byte index and not character index, virtual column is in
screen cells which also does not match characters).

Also any character above U+00FF in Python3 is *not* a single byte, it
is just as single byte character as 0x is single byte in `[0x,
0xFFFE, 0xFFFD][0]`. Simply different way of storing and indexing
strings.

>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups 
> "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to vim_use+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Capture columns nummers of matches ending with double byte chars

[rameo]

Searchpos() doesn't return the right end value of a match if the match end with 
a double byte character (èéòìùá...). (encoding utf-8)
Isn't this a bug

Would it be possible to add a feature in Vim like finditer in Python?
Searchpos() searches the entire file till stopline.
finditer returns all startvalues, endvalues and matches of a search in a string 
(or line).
This would be great!

>From Python website:
re.finditer(pattern, string[, flags])

Return an iterator yielding MatchObject instances over all non-overlapping 
matches for the RE pattern in string. The string is scanned left-to-right, and 
matches are returned in the order found. Empty matches are included in the 
result unless they touch the beginning of another match.

>>> text = "He was carefully disguised but captured quickly by police."
>>> for m in re.finditer(r"\w+ly", text):
... print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
07-16: carefully
40-47: quickly

https://docs.python.org/3/library/re.html

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Capture columns nummers of matches ending with double byte chars

[rameo]

Since I use Vim I have troubles with double byte characters.

I want to capture all strings of matches together with startcolumn and 
endcolumn of a match (line by line). I don't need only the strings but also the 
columnnumbers for other functions.

The last few years I used match/matchend then I noted that it did not capture 
correctly double byte characters within the string.
Then I adapted everything to use searchpos() but today I found out that it 
gives troubles with string with a double byte character at the end. 


   "mylist = list with all linenrs having matches
   for n in range(0, len(mylist)-1)
 let idx  = []
 let edx  = []
 let matches_between_cols = []

 "FIND ALL IDX MATCHES 
 "idx --> forward search
 call cursor(mylist[n],1)
 while line(".") == mylist[n]
   let S= searchpos(@/, '')
   if S[0] == mylist[n]
 call add(idx, S[1]-1)
   endif
 endwhile
 "idx --> backward search (to include matches on first column)
 call cursor(mylist[n],len(getline(mylist[n])))
 while line(".") == mylist[n]
   let S= searchpos(@/, 'b')
   if S[0] == mylist[n]
 call add(idx, S[1]-1)
   endif
 endwhile

 "FIND ALL EDX MATCHES
 "edx --> forward search
 call cursor(mylist[n],1)
 while line(".") == mylist[n]
   let E= searchpos(@/, 'e')
   if E[0] == mylist[n]
 call add(edx, E[1])
   endif
 endwhile
 "edx --> backward search (to include matches on first column)
 call cursor(mylist[n],len(getline(mylist[n])))
 while line(".") == mylist[n]
   let E= searchpos(@/, 'eb')
   if E[0] == mylist[n]
 call add(edx, E[1])
   endif
 endwhile

 if len(idx) > 0
   for i in range(0,len(idx)-1)
let r = strpart(getline(mylist[n]),idx[i], edx[i]-idx[i])
call add(matches_between_cols, r)
   endfor
 endif
   endfor

---
Buffer:
city | Felicità 
whatever | Peach 
pmg0001 | Perché 
text| Céline
bMgbXuEWo | Université 


@/ = "| \zs\S\+"
it captures:
Felicit
Peach
Perch
Céline
Universit

Expected:
Felicità 
Peach 
Perché
Céline 
Université 

Can you please tell me what I did wrong?
(Is it not possible to let every character be a single byte char as in 
languages as Python?)

-- 
-- 
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.