Re: Getting the byte index (column) given the character column number
Hi Bram, On Sun, Nov 20, 2022 at 4:04 PM Bram Moolenaar wrote: > > > Yegappan wrote: > > > The language server protocol messages use character column number > > whereas many of the built-in Vim functions (e.g. matchaddpos()) deal > > with byte column number. > > > > Several built-in functions were added to convert between the character > > and byte column numbers (byteidx(), charcol(), charidx(), > > getcharpos(), getcursorcharpos(), etc,). > > But these functions deal with strings, current cursor position or the > > position of a mark. > > > > We currently don't have a function to return the byte number given the > > character number in a line in a buffer. The workaround is to use > > getbufline() to get the entire buffer line and then use byteidx() to > > get the byte number from the character number. > > > > I am thinking of introducing a new function named charcol2bytecol() > > that accepts a buffer number, line number and the character number in > > the line and returns the corresponding byte number. Any > > suggestions/comments on this? > > > > We should also modify the matchaddpos() function to accept character numbers > > in a line in addition to the byte numbers. > > Just to make sure we understand what we are talking about: This is > always about text in a buffer? Thus the buffer text is somehow passed > through the LSP to a server, which then returns information with > character indexes. > Yes. The location information returned by the LSP server is about the text in the buffer. > > One detail that matters: Are composing characters counted separately, or > not counted (part of the base character)? > I think composing counters are not counted. But I couldn't find this mentioned in the LSP specification: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#position > > Also, I assume a Tab is counted as just one character, not the number of > display cells it occupies. > Yes. Tab is counted as one character. > > I wonder if it's really helpful to add a new function if it can > currently be done with two. You already mention that the text can be > obtained with getbufline(), and then get the byte index from the > character index with byteidx(). What is the problem with doing it that > way? > If the conversion has to be done too many times then it is not efficient. > > Other message: > > > Another alternative is to extend the col() function. The col() > > function currently accepts a list with two numbers (a line number and > > a byte number or "$") and returns the byte number. > > This can be modified to also accept a list with three numbers (line > > number, column number and a boolean indicating character column or > > byte column) and return the byte number. > > I don't like this, the first line for the col() help is: > > The result is a Number, which is the byte index of the column > > When the boolean is true this would be the character index, that is hard > to explain. A user would have to look really hard to find this > functionality. > The boolean doesn't change the return value of the col() function. It just changes how the col() function interprets the column number in the list. If it is true, then the col() function will use the column number as the character number. If it is false or not specified, then the col() function will use it as the byte number. In both cases the col() function will always return the byte index of the column. > > There is also charcol(), it appears to be doing what you want already. > The charcol() function returns the character number in a line. This function cannot be used to get the byte index given the character index. Regards, Yegappan -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/CAAW7x7kLMOszc%3DPSRVqoqw%3DuBkupgN4nNrkqUdrpepH%2Bt7qxhA%40mail.gmail.com.
Re: Getting the byte index (column) given the character column number
Yegappan wrote: > The language server protocol messages use character column number > whereas many of the built-in Vim functions (e.g. matchaddpos()) deal > with byte column number. > > Several built-in functions were added to convert between the character > and byte column numbers (byteidx(), charcol(), charidx(), > getcharpos(), getcursorcharpos(), etc,). > But these functions deal with strings, current cursor position or the > position of a mark. > > We currently don't have a function to return the byte number given the > character number in a line in a buffer. The workaround is to use > getbufline() to get the entire buffer line and then use byteidx() to > get the byte number from the character number. > > I am thinking of introducing a new function named charcol2bytecol() > that accepts a buffer number, line number and the character number in > the line and returns the corresponding byte number. Any > suggestions/comments on this? > > We should also modify the matchaddpos() function to accept character numbers > in a line in addition to the byte numbers. Just to make sure we understand what we are talking about: This is always about text in a buffer? Thus the buffer text is somehow passed through the LSP to a server, which then returns information with character indexes. One detail that matters: Are composing characters counted separately, or not counted (part of the base character)? Also, I assume a Tab is counted as just one character, not the number of display cells it occupies. I wonder if it's really helpful to add a new function if it can currently be done with two. You already mention that the text can be obtained with getbufline(), and then get the byte index from the character index with byteidx(). What is the problem with doing it that way? Other message: > Another alternative is to extend the col() function. The col() > function currently accepts a list with two numbers (a line number and > a byte number or "$") and returns the byte number. > This can be modified to also accept a list with three numbers (line > number, column number and a boolean indicating character column or > byte column) and return the byte number. I don't like this, the first line for the col() help is: The result is a Number, which is the byte index of the column When the boolean is true this would be the character index, that is hard to explain. A user would have to look really hard to find this functionality. There is also charcol(), it appears to be doing what you want already. -- hundred-and-one symptoms of being an internet addict: 92. It takes you two hours to check all 14 of your mailboxes. /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\ /// \\\ \\\sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\\help me help AIDS victims -- http://ICCF-Holland.org/// -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/20221121000400.7E0B41C12B9%40moolenaar.net.
Re: Getting the byte index (column) given the character column number
On Sun, Nov 20, 2022 at 9:05 AM Yegappan Lakshmanan wrote: > > Hi all, > > The language server protocol messages use character column number whereas many > of the built-in Vim functions (e.g. matchaddpos()) deal with byte column > number. > > Several built-in functions were added to convert between the character and > byte > column numbers (byteidx(), charcol(), charidx(), getcharpos(), > getcursorcharpos(), etc,). > But these functions deal with strings, current cursor position or the > position of a mark. > > We currently don't have a function to return the byte number given the > character > number in a line in a buffer. The workaround is to use getbufline() > to get the entire > buffer line and then use byteidx() to get the byte number from the > character number. > > I am thinking of introducing a new function named charcol2bytecol() that > accepts > a buffer number, line number and the character number in the line and > returns the > corresponding byte number. Any suggestions/comments on this? > Another alternative is to extend the col() function. The col() function currently accepts a list with two numbers (a line number and a byte number or "$") and returns the byte number. This can be modified to also accept a list with three numbers (line number, column number and a boolean indicating character column or byte column) and return the byte number. - Yegappan > We should also modify the matchaddpos() function to accept character numbers > in a line in addition to the byte numbers. > > Regards, > Yegappan -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/CAAW7x7%3D7najhNxO%3Dn5LzQ6dmO-mPhLnrzn2iO-AXKF4%2BxfszVA%40mail.gmail.com.
Getting the byte index (column) given the character column number
Hi all, The language server protocol messages use character column number whereas many of the built-in Vim functions (e.g. matchaddpos()) deal with byte column number. Several built-in functions were added to convert between the character and byte column numbers (byteidx(), charcol(), charidx(), getcharpos(), getcursorcharpos(), etc,). But these functions deal with strings, current cursor position or the position of a mark. We currently don't have a function to return the byte number given the character number in a line in a buffer. The workaround is to use getbufline() to get the entire buffer line and then use byteidx() to get the byte number from the character number. I am thinking of introducing a new function named charcol2bytecol() that accepts a buffer number, line number and the character number in the line and returns the corresponding byte number. Any suggestions/comments on this? We should also modify the matchaddpos() function to accept character numbers in a line in addition to the byte numbers. Regards, Yegappan -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/CAAW7x7%3DRz5kuO9h9Y9ksx7EbHOyLmWh%3DJv7UeCEq62hGA10BwA%40mail.gmail.com.
Re: Test Failure
> Environment: Linux > Terminal: tmux > Compiler: clang-15 > Version: 9.0.0915 > > There is a test failure: > Found errors in Test_autocmd_user_clear_group(): > Run 1, 15:38:56 - 15:39:01: > command line..script /source/vim/vim-9.0.0915/src/vim-motif/testdir= > /runtest.vim[515]..function RunTheTest[52]..Test_autocmd_user_clear_group[1= > 8]..StopVimInTerminal[14]..WaitForAssert[2]..5_WaitForCommon[11].. bda>47 line 1: Expected 'finished' but got 'running' It does not fail for me in a similar setup, and CI also doesn't show this failure. Can you think of anything that matters? I can't really fix a test failure if I can't reproduce it. You could try increasing the "50" argument of TermWait() in line 2239 and see if that makes a difference. -- >From "know your smileys": |-(Contact lenses, but has lost them /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\ /// \\\ \\\sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\\help me help AIDS victims -- http://ICCF-Holland.org/// -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/20221120152028.0FFC91C12B2%40moolenaar.net.
Test Failure
Hi all, Environment: Linux Terminal: tmux Compiler: clang-15 Version: 9.0.0915 There is a test failure: Found errors in Test_autocmd_user_clear_group(): Run 1, 15:38:56 - 15:39:01: command line..script /source/vim/vim-9.0.0915/src/vim-motif/testdir/runtest.vim[515]..function RunTheTest[52]..Test_autocmd_user_clear_group[18]..StopVimInTerminal[14]..WaitForAssert[2]..5_WaitForCommon[11]..47 line 1: Expected 'finished' but got 'running' Thanks for investigation Elimar -- The path to source is always uphill! -unknown- -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/20221120150518.cjvbewrxz2elv7ao%40baumbart.home.lxtec.de.
Patch 9.0.0915
Patch 9.0.0915
Problem:WinScrolled may trigger immediately when defined.
Solution: Initialize the fields in all windows. (closes #11582)
Files: src/autocmd.c, src/window.c, src/proto/window.pro,
src/testdir/test_autocmd.vim,
src/testdir/dumps/Test_winscrolled_not_when_defined_1.dump,
src/testdir/dumps/Test_winscrolled_not_when_defined_2.dump
*** ../vim-9.0.0914/src/autocmd.c 2022-11-13 17:53:42.283417583 +
--- src/autocmd.c 2022-11-20 12:10:13.844729626 +
***
*** 1264,1277
get_mode(last_mode);
#endif
// Initialize the fields checked by the WinScrolled trigger to
! // stop it from firing right after the first autocmd is defined.
if (event == EVENT_WINSCROLLED && !has_winscrolled())
{
! curwin->w_last_topline = curwin->w_topline;
! curwin->w_last_leftcol = curwin->w_leftcol;
! curwin->w_last_skipcol = curwin->w_skipcol;
! curwin->w_last_width = curwin->w_width;
! curwin->w_last_height = curwin->w_height;
}
if (is_buflocal)
--- 1264,1283
get_mode(last_mode);
#endif
// Initialize the fields checked by the WinScrolled trigger to
! // prevent it from firing right after the first autocmd is
! // defined.
if (event == EVENT_WINSCROLLED && !has_winscrolled())
{
! tabpage_T *save_curtab = curtab;
! tabpage_T *tp;
! FOR_ALL_TABPAGES(tp)
! {
! unuse_tabpage(curtab);
! use_tabpage(tp);
! snapshot_windows_scroll_size();
! }
! unuse_tabpage(curtab);
! use_tabpage(save_curtab);
}
if (is_buflocal)
*** ../vim-9.0.0914/src/window.c2022-11-19 21:17:48.841226535 +
--- src/window.c2022-11-20 12:01:47.440641805 +
***
*** 2846,2852
* Make a snapshot of all the window scroll positions and sizes of the current
* tab page.
*/
! static void
snapshot_windows_scroll_size(void)
{
win_T *wp;
--- 2846,2852
* Make a snapshot of all the window scroll positions and sizes of the current
* tab page.
*/
! void
snapshot_windows_scroll_size(void)
{
win_T *wp;
***
*** 3864,3869
--- 3864,3896
}
/*
+ * Store the relevant window pointers for tab page "tp". To be used before
+ * use_tabpage().
+ */
+ void
+ unuse_tabpage(tabpage_T *tp)
+ {
+ tp->tp_topframe = topframe;
+ tp->tp_firstwin = firstwin;
+ tp->tp_lastwin = lastwin;
+ tp->tp_curwin = curwin;
+ }
+
+ /*
+ * Set the relevant pointers to use tab page "tp". May want to call
+ * unuse_tabpage() first.
+ */
+ void
+ use_tabpage(tabpage_T *tp)
+ {
+ curtab = tp;
+ topframe = curtab->tp_topframe;
+ firstwin = curtab->tp_firstwin;
+ lastwin = curtab->tp_lastwin;
+ curwin = curtab->tp_curwin;
+ }
+
+ /*
* Allocate the first window and put an empty buffer in it.
* Called from main().
* Return FAIL when something goes wrong (out of memory).
***
*** 3877,3887
first_tabpage = alloc_tabpage();
if (first_tabpage == NULL)
return FAIL;
- first_tabpage->tp_topframe = topframe;
curtab = first_tabpage;
! curtab->tp_firstwin = firstwin;
! curtab->tp_lastwin = lastwin;
! curtab->tp_curwin = curwin;
return OK;
}
--- 3904,3911
first_tabpage = alloc_tabpage();
if (first_tabpage == NULL)
return FAIL;
curtab = first_tabpage;
! unuse_tabpage(first_tabpage);
return OK;
}
***
*** 4389,4398
win_T *next_prevwin = tp->tp_prevwin;
tabpage_T *last_tab = curtab;
! curtab = tp;
! firstwin = tp->tp_firstwin;
! lastwin = tp->tp_lastwin;
! topframe = tp->tp_topframe;
// We would like doing the TabEnter event first, but we don't have a
// valid current window yet, which may break some commands.
--- 4413,4419
win_T *next_prevwin = tp->tp_prevwin;
tabpage_T *last_tab = curtab;
! use_tabpage(tp);
// We would like doing the TabEnter event first, but we don't have a
// valid current window yet, which may break some commands.
*** ../vim-9.0.0914/src/proto/window.pro2022-11-19 21:17:48.841226535
+
--- src/proto/window.pro2022-11-20 11:53:02.444550244 +
***
*** 18,29
--- 18,32
void close_windows(buf_T *buf, int keep_curwin);
int one_window(void);
int win_close(win_T *win, int free_buf);
+ void snapshot_windows_scroll_size(void);
void
Patch 9.0.0914
Patch 9.0.0914
Problem:deletebufline() may move marks in the wrong window.
Solution: Find a window for the buffer being changed. (closes #11583)
Files: src/evalbuffer.c, src/testdir/test_bufline.vim
*** ../vim-9.0.0913/src/evalbuffer.c2022-11-19 13:59:39.511861139 +
--- src/evalbuffer.c2022-11-20 11:12:50.860191314 +
***
*** 119,124
--- 119,177
}
}
+ typedef struct {
+ win_T *cob_curwin_save;
+ aco_save_Tcob_aco;
+ int cob_using_aco;
+ int cob_save_VIsual_active;
+ } cob_T;
+
+ /*
+ * Used before making a change in "buf", which is not the current one: Make
+ * "buf" the current buffer and find a window for this buffer, so that side
+ * effects are done correctly (e.g., adjusting marks).
+ *
+ * Information is saved in "cob" and MUST be restored by calling
+ * change_other_buffer_restore().
+ */
+ static void
+ change_other_buffer_prepare(cob_T *cob, buf_T *buf)
+ {
+ CLEAR_POINTER(cob);
+
+ // Set "curbuf" to the buffer being changed. Then make sure there is a
+ // window for it to handle any side effects.
+ cob->cob_save_VIsual_active = VIsual_active;
+ VIsual_active = FALSE;
+ cob->cob_curwin_save = curwin;
+ curbuf = buf;
+ find_win_for_curbuf(); // simplest: find existing window for "buf"
+
+ if (curwin->w_buffer != buf)
+ {
+ // No existing window for this buffer. It is dangerous to have
+ // curwin->w_buffer differ from "curbuf", use the autocmd window.
+ curbuf = curwin->w_buffer;
+ aucmd_prepbuf(>cob_aco, buf);
+ cob->cob_using_aco = TRUE;
+ }
+ }
+
+ static void
+ change_other_buffer_restore(cob_T *cob)
+ {
+ if (cob->cob_using_aco)
+ {
+ aucmd_restbuf(>cob_aco);
+ }
+ else
+ {
+ curwin = cob->cob_curwin_save;
+ curbuf = curwin->w_buffer;
+ }
+ VIsual_active = cob->cob_save_VIsual_active;
+ }
+
/*
* Set line or list of lines in buffer "buf" to "lines".
* Any type is allowed and converted to a string.
***
*** 137,146
listitem_T*li = NULL;
long added = 0;
linenr_T append_lnum;
- win_T *curwin_save = NULL;
- aco_save_Taco;
- int using_aco = FALSE;
- int save_VIsual_active = VIsual_active;
// When using the current buffer ml_mfp will be set if needed. Useful
when
// setline() is used on startup. For other buffers the buffer must be
--- 190,195
***
*** 154,177
return;
}
if (!is_curbuf)
! {
! // Set "curbuf" to the buffer being changed. Then make sure there is a
! // window for it to handle any side effects.
! VIsual_active = FALSE;
! curwin_save = curwin;
! curbuf = buf;
! find_win_for_curbuf(); // simplest: find existing window for "buf"
!
! if (curwin->w_buffer != buf)
! {
! // No existing window for this buffer. It is dangerous to have
! // curwin->w_buffer differ from "curbuf", use the autocmd window.
! curbuf = curwin->w_buffer;
! aucmd_prepbuf(, buf);
! using_aco = TRUE;
! }
! }
if (append)
// appendbufline() uses the line number below which we insert
--- 203,213
return;
}
+ // After this don't use "return", goto "cleanup"!
+ cob_T cob;
if (!is_curbuf)
! // set "curbuf" to "buf" and find a window for this buffer
! change_other_buffer_prepare(, buf);
if (append)
// appendbufline() uses the line number below which we insert
***
*** 272,289
done:
if (!is_curbuf)
! {
! if (using_aco)
! {
! aucmd_restbuf();
! }
! else
! {
! curwin = curwin_save;
! curbuf = curwin->w_buffer;
! }
! VIsual_active = save_VIsual_active;
! }
}
/*
--- 308,314
done:
if (!is_curbuf)
! change_other_buffer_restore();
}
/*
***
*** 521,532
linenr_T lnum;
long count;
int is_curbuf;
- buf_T *curbuf_save = NULL;
- win_T *curwin_save = NULL;
tabpage_T *tp;
win_T *wp;
int did_emsg_before = did_emsg;
- int save_VIsual_active = VIsual_active;
rettv->vval.v_number = 1; // FAIL by default
--- 546,554
***
*** 539,545
buf = tv_get_buf([0], FALSE);
if (buf == NULL)
return;
- is_curbuf = buf == curbuf;
first = tv_get_lnum_buf([1], buf);
if (did_emsg > did_emsg_before)
--- 561,566
***
*** 554,567
return;
// After this don't use "return", goto "cleanup"!
if (!is_curbuf)
! {
! VIsual_active = FALSE;
! curbuf_save = curbuf;
!
