Re: Regex performance - SoC
On Tue, 2007-05-01 at 18:16 -0500, James Longstreet wrote: > Hello, > > I'm wondering if any of the Summer of Code winners are tackling the > regex performance thing suggested here: http://www.vim.org/soc/ideas.php Bram said two students are working on this, but you are welcome to help them (i think). Nikolai too would help anyone with this. > If anyone chose that project, I'll of course defer to them. But if > no one's working on it, I might give it a shot, I think it'd be fun. > Thanks, > James Longstreet - Asiri
Regex performance - SoC
Hello, I'm wondering if any of the Summer of Code winners are tackling the regex performance thing suggested here: http://www.vim.org/soc/ideas.php If anyone chose that project, I'll of course defer to them. But if no one's working on it, I might give it a shot, I think it'd be fun. Thanks, James Longstreet
patch 7.0.238
Patch 7.0.238 Problem:Crash when ":match" pattern runs into 'maxmempattern'. (Yakov Lerner) Solution: Don't free the regexp program of match_hl. Files: src/screen.c *** ../vim-7.0.237/src/screen.c Tue Nov 28 16:16:03 2006 --- src/screen.cTue May 1 21:36:50 2007 *** *** 6477,6485 if (called_emsg) { /* Error while handling regexp: stop using this regexp. */ ! vim_free(shl->rm.regprog); shl->rm.regprog = NULL; ! no_hlsearch = TRUE; break; } if (nmatched == 0) --- 6477,6491 if (called_emsg) { /* Error while handling regexp: stop using this regexp. */ ! if (shl == &search_hl) ! { ! /* don't free the regprog in match_hl[], it's a copy */ ! vim_free(shl->rm.regprog); ! no_hlsearch = TRUE; ! } shl->rm.regprog = NULL; ! shl->lnum = 0; ! got_int = FALSE; /* avoid the "Type :quit to exit Vim" message */ break; } if (nmatched == 0) *** ../vim-7.0.237/src/version.cTue May 1 19:06:39 2007 --- src/version.c Tue May 1 21:39:48 2007 *** *** 668,669 --- 668,671 { /* Add new patch number below this line */ + /**/ + 238, /**/ -- Change is inevitable, except from a vending machine. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: patch 7.0.237
Bram Moolenaar wrote: Patch 7.0.237 Problem:For root it is recommended to not use 'modeline', but in not-compatible mode the default is on. Solution: Let 'modeline' default to off for root. Files: runtime/doc/options.txt, src/option.c Note: :set ml&vim or :set nocompatible ... :set ml& still sets 'modeline' on, even when logged-in as root. Best regards, Tony. -- If you want to understand your government, don't begin by reading the Constitution. It conveys precious little of the flavor of today's statecraft. Instead, read selected portions of the Washington telephone directory containing listings for all the organizations with titles beginning with the word "National". -- George Will
patch 7.0.237
Patch 7.0.237 Problem:For root it is recommended to not use 'modeline', but in not-compatible mode the default is on. Solution: Let 'modeline' default to off for root. Files: runtime/doc/options.txt, src/option.c *** ../vim-7.0.236/runtime/doc/options.txt Tue Oct 10 18:43:50 2006 --- runtime/doc/options.txt Tue May 1 13:29:08 2007 *** *** 1,4 ! *options.txt* For Vim version 7.0. Last change: 2006 May 04 VIM REFERENCE MANUALby Bram Moolenaar --- 1,4 ! *options.txt* For Vim version 7.0. Last change: 2007 May 01 VIM REFERENCE MANUALby Bram Moolenaar *** *** 528,534 ':' is removed. Thus to include "\:" you have to specify "\\:". No other commands than "set" are supported, for security reasons (somebody ! might create a Trojan horse text file with modelines). Hint: If you would like to do something else than setting an option, you could define an autocommand that checks the file for a specific string. For --- 528,539 ':' is removed. Thus to include "\:" you have to specify "\\:". No other commands than "set" are supported, for security reasons (somebody ! might create a Trojan horse text file with modelines). And not all options ! can be set. For some options a flag is set, so that when it's used the ! |sandbox| is effective. Still, there is always a small risc that a modeline ! causes trouble. E.g., when some joker sets 'textwidth' to 5 all your lines ! are wrapped unexpectedly. So disable modelines before editing untrusted text. ! The mail ftplugin does this, for example. Hint: If you would like to do something else than setting an option, you could define an autocommand that checks the file for a specific string. For *** *** 4520,4526 languages, no matter what you set 'mkspellmem' to. *'modeline'* *'ml'* *'nomodeline'* *'noml'* ! 'modeline' 'ml' boolean (Vim default: on, Vi default: off) local to buffer *'modelines'* *'mls'* 'modelines' 'mls' number (default 5) --- 4557,4564 languages, no matter what you set 'mkspellmem' to. *'modeline'* *'ml'* *'nomodeline'* *'noml'* ! 'modeline' 'ml' boolean (Vim default: on (off for root), !Vi default: off) local to buffer *'modelines'* *'mls'* 'modelines' 'mls' number (default 5) *** ../vim-7.0.236/src/option.c Tue May 1 13:39:14 2007 --- src/option.cTue May 1 13:26:10 2007 *** *** 3429,3434 --- 3429,3439 /* the cast to long is required for Manx C, long_i is needed for * MSVC */ *(int *)varp = (int)(long)(long_i)options[opt_idx].def_val[dvi]; + #ifdef UNIX + /* 'modeline' defaults to off for root */ + if (options[opt_idx].indir == PV_ML && getuid() == ROOT_UID) + *(int *)varp = FALSE; + #endif /* May also set global value for local option. */ if (both) *(int *)get_varp_scope(&(options[opt_idx]), OPT_GLOBAL) = *** ../vim-7.0.236/src/version.cTue May 1 13:39:14 2007 --- src/version.c Tue May 1 19:03:09 2007 *** *** 668,669 --- 668,671 { /* Add new patch number below this line */ + /**/ + 237, /**/ -- The users that I support would double-click on a landmine to find out what happens. -- A system administrator /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: patch 7.0.236
scott wrote: i had to 'rm src/auto/config.cache before i could build with this one sc Yes, this is normal for a patch affecting the configure files: src/auto/configure src/configure.in src/config.h.in Me too: the first make ended in an error, suggesting to remove auto/config.cache; then after "rm -vf auto/config.cache" the next make rebuilt everything. Best regards, Tony. -- THE LESSER-KNOWN PROGRAMMING LANGUAGES #2: RENE Named after the famous French philosopher and mathematician Rene DesCartes, RENE is a language used for artificial intelligence. The language is being developed at the Chicago Center of Machine Politics and Programming under a grant from the Jane Byrne Victory Fund. A spokesman described the language as "Just as great as dis [sic] city of ours." The center is very pleased with progress to date. They say they have almost succeeded in getting a VAX to think. However, sources inside the organization say that each time the machine fails to think it ceases to exist.
Re: patch 7.0.236
i had to 'rm src/auto/config.cache before i could build with this one sc
patch 7.0.236
Patch 7.0.236 Problem:Linux 2.4 uses sysinfo() with a mem_unit field, which is not backwards compatible. Solution: Add an autoconf check for sysinfo.mem_unit. Let mch_total_mem() return Kbyte to avoid overflow. Files: src/auto/configure, src/configure.in, src/config.h.in, src/option.c, src/os_unix.c *** ../vim-7.0.235/src/auto/configure Tue Oct 17 11:50:45 2006 --- src/auto/configure Thu Apr 26 16:44:07 2007 *** *** 13685,13690 --- 13685,13746 echo "$as_me:$LINENO: result: not usable" >&5 echo "${ECHO_T}not usable" >&6 + fi + rm -f conftest.err conftest.$ac_objext conftest.$ac_ext + + echo "$as_me:$LINENO: checking for sysinfo.mem_unit" >&5 + echo $ECHO_N "checking for sysinfo.mem_unit... $ECHO_C" >&6 + cat >conftest.$ac_ext <<_ACEOF + /* confdefs.h. */ + _ACEOF + cat confdefs.h >>conftest.$ac_ext + cat >>conftest.$ac_ext <<_ACEOF + /* end confdefs.h. */ + #include + #include + int + main () + { + struct sysinfo sinfo; + sinfo.mem_unit = 1; + + ; + return 0; + } + _ACEOF + rm -f conftest.$ac_objext + if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && +{ ac_try='test -z "$ac_c_werror_flag" +|| test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && +{ ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + echo "$as_me:$LINENO: result: yes" >&5 + echo "${ECHO_T}yes" >&6; cat >>confdefs.h <<\_ACEOF + #define HAVE_SYSINFO_MEM_UNIT 1 + _ACEOF + + else + echo "$as_me: failed program was:" >&5 + sed 's/^/| /' conftest.$ac_ext >&5 + + echo "$as_me:$LINENO: result: no" >&5 + echo "${ECHO_T}no" >&6 fi rm -f conftest.err conftest.$ac_objext conftest.$ac_ext *** ../vim-7.0.235/src/configure.in Tue Oct 17 11:50:45 2006 --- src/configure.inThu Apr 26 16:40:18 2007 *** *** 2589,2594 --- 2589,2605 ], AC_MSG_RESULT(yes); AC_DEFINE(HAVE_SYSINFO), AC_MSG_RESULT(not usable)) + + dnl struct sysinfo may have the mem_unit field or not + AC_MSG_CHECKING(for sysinfo.mem_unit) + AC_TRY_COMPILE( + [#include + #include ], + [ struct sysinfo sinfo; + sinfo.mem_unit = 1; + ], + AC_MSG_RESULT(yes); AC_DEFINE(HAVE_SYSINFO_MEM_UNIT), + AC_MSG_RESULT(no)) dnl sysconf() may exist but not support what we want to use AC_MSG_CHECKING(for sysconf) *** ../vim-7.0.235/src/config.h.in Fri Apr 21 00:11:09 2006 --- src/config.h.in Thu Apr 26 16:40:34 2007 *** *** 176,181 --- 176,182 #undef HAVE_SYSCONF #undef HAVE_SYSCTL #undef HAVE_SYSINFO + #undef HAVE_SYSINFO_MEM_UNIT #undef HAVE_TGETENT #undef HAVE_TOWLOWER #undef HAVE_TOWUPPER *** ../vim-7.0.235/src/option.c Fri Mar 2 20:00:06 2007 --- src/option.cTue May 1 13:26:10 2007 *** *** 3030,3036 #else # ifdef HAVE_TOTAL_MEM /* Use amount of memory available to Vim. */ ! n = (mch_total_mem(FALSE) >> 11); # else n = (0x7fff >> 11); # endif --- 3030,3036 #else # ifdef HAVE_TOTAL_MEM /* Use amount of memory available to Vim. */ ! n = (mch_total_mem(FALSE) >> 1); # else n = (0x7fff >> 11); # endif *** ../vim-7.0.235/src/os_unix.cThu Apr 26 16:28:43 2007 --- src/os_unix.c Thu Apr 26 16:37:43 2007 *** *** 428,435 # endif /* ! * Return total amount of memory available. Doesn't change when memory has ! * been allocated. */ /* ARGSUSED */ long_u --- 428,435 # endif /* ! * Return total amount of memory available in Kbyte. ! * Doesn't change when memory has been allocated. */ /* ARGSUSED */ long_u *** *** 437,445 int special; { # ifdef __EMX__ ! return ulimit(3, 0L); /* always 32MB? */ # else long_umem = 0; # ifdef HAVE_SYSCTL int mib[2], physmem; --- 437,446 int special; { # ifdef __EMX__ ! return ulimit(3, 0L) >> 10; /* always 32MB? */ # else long_umem = 0; + long_ushiftright = 10; /* how much to shift "mem" right for Kbyte */ # ifdef HAVE_SYSCTL int mib[2], physmem; *** *** 460,466 --- 461,479 /* Linux way of getting amount of RAM available */ if (sysinfo(&sinfo) == 0) + { + # ifdef HAVE_SYSINFO_MEM_UNIT + /* avoid
Re: feedkeys() allowed in sandbox
Tony Mechelynck wrote: > Bram Moolenaar wrote: > [...] > > Modelines are default off when you are root. The mail filetype plugin > > also switches it off. > [...] > > Are you sure? In a terminal logged-in as root, using vim 7.0.235: > > vim -u NONE -N > :set ml? mls? >modeline >modelines=5 > > Modelines default off when 'compatible' is set, they default on (and 5) when > 'nocompatible' is set. Root login changes nothing to that AFAICT. Sorry, my mistake. There is a recommendation that when working as root you switch 'modeline' off, but it's not done automatically. I do think that it's a good idea to make it automatic. -- He who laughs last, thinks slowest. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: feedkeys() allowed in sandbox
Bram Moolenaar wrote: [...] Modelines are default off when you are root. The mail filetype plugin also switches it off. [...] Are you sure? In a terminal logged-in as root, using vim 7.0.235: vim -u NONE -N :set ml? mls? modeline modelines=5 Modelines default off when 'compatible' is set, they default on (and 5) when 'nocompatible' is set. Root login changes nothing to that AFAICT. Best regards, Tony. -- "You mean there really is an answer?" "Yes! But you're not going to like it!" "Oh do please tell us!" "You're really not going to like it!" "but we MUST know - tell us" "Alright, the answer is" "yes..." "... is ..." "yes... come on!" "is 42!" (Douglas Adams - The Hitchhiker's Guide to the Galaxy)
Re: feedkeys() allowed in sandbox
Matthew Winn wrote: If there was a security problem in Vim do you really think it couldn't be exploited in 100 characters? That's a pretty shaky foundation on which to build your security. I am quite surprised at the lack of appreciation for the merits of "defense in depth" here. I am not claiming that a length limit would preclude damage, just that a modeline should be sanity checked before execution, and a reasonable length would be the first check. It's certainly true that a modeline of 100 bytes could wreak havoc on an unpatched Vim. But it is quite possible that a future vulnerability might allow code to be injected from a modeline, and limiting its length *might* make the attacker's job harder. It's sensational that Vim can process files with very long lines, for the occasions we need that. But it would be absurd for Vim to process a multi-megabyte modeline. By all means abuse me for my cheeky suggestion to limit modelines to 100 bytes, but while doing that you might agree that some limit under 1MB should be enforced. A web browser should be able to handle anything thrown at it in a way that doesn't compromise security. _Every_ application should be able to handle anything thrown at it in a way that doesn't compromise security. Even if a program is perfect now, a later change can introduce a bug. Any program which can automatically execute untrusted code should sanity-check the input as a separate step from sandboxing. That is standard Security 101 stuff - not my idea. What you're suggesting isn't much different from security through obscurity. Jargon is great, but not when it's misused. Perl and Vim have exactly the same requirements: the need to safely handle code taken from an untrustworthy source. It makes no difference whether it comes directly from a network or from a disk. (If, like me, you use Vim as your source viewer for web pages, the need for the same level of security is obvious.) It doesn't matter, but for the record, Perl's tainting system is not related to the scenario you describe. Perl wants to make sure that untrusted input is not later used as the basis for some expression that could do harm, such as executing SQL code. John
Re: feedkeys() allowed in sandbox
Bram Moolenaar wrote: N times as safe still isn't 100% safe. I am not claiming that sanity-checking a modeline before execution would make it 100% safe. But there have been many examples in other software where minor bugs have turned into security disasters because some simple point that could have been checked, wasn't. While code is working correctly, a simple check is redundant, and indeed is offensive because it lengthens and obscures the code. But a few simple checks may prevent disaster at some future time, when Vim is further developed. The Google test (searching for past instances of trouble with Vim's modeline) proves the case that future problems are likely. Modelines are default off when you are root. The mail filetype plugin also switches it off. Good grief - I didn't know that. So you *have* got sanity checks built in! I'll go and sit in the corner now, but thanks for confirming that multiple layers of defence are desirable. John
Re: feedkeys() allowed in sandbox
John Beckett wrote: > A.J.Mechelynck wrote: > >> Is folding really needed in a default modeline? > > Folding may be useful in a modeline. > > (Don't know what you call a "default" modeline.) > > By "default modeline" I mean I would like Vim to be changed so > that its default behaviour is aggressively safe. If wanted, > there could be a new option to enable clever features, and a > user could choose to allow modelines with folding or expression > evaluation, etc. This is not true. It just reduces the chance of a mistake being made by an unknown factor. It's still possible to allow an option to be set, thinking that it is OK, but we later find out that it was not OK. Just like carefully removing mistakes and screening the options for mistakes does help to make it safer. Thus it doesn't make an essential difference. N times as safe still isn't 100% safe. In other words: If we have an option "run insecure" nobody would set it. Vim must be secure as-is. > But the only long-term safe procedure is to have Vim *default* > to work with only very restricted modelines (set tab and other > options - no way to even get near executing code). As they sometimes joke: The best way to protect your computer from malicious software is to switch it off. Likewise, the only really safe way is to disable modelines. Obviously you pay a price: restricted functionality. Options to partly disable modelines make it more complicated and don't help much for security. > I am wondering what the lack of comment on this topic indicates. > Do you understand that another modeline vulnerability could > allow the next file you open to overwrite all files under your > home folder? Or it might overwrite all sectors on your disk, if > you have sufficient privilege. Don't forget that this requires someone who intentionally wants this evil thing to happen. So far the only examples seen are jokes and proof of concept. I have never seen a file with a modeline that intentionally causes harm. > How about if you go to another computer that you rarely use. > Would you be happy using Vim on that computer? > Network admins in secure environments should be prohibited > from using Vim. Modelines are default off when you are root. The mail filetype plugin also switches it off. > If I am overlooking something, or am overly alarmist, please > tell me. For anyone new to this, enter following in Google: > vim vulnerability modeline Thanks for the advertisement! :-). -- Give a man a computer program and you give him a headache, but teach him to program computers and you give him the power to create headaches for others for the rest of his life... R. B. Forest /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: wish: show search progress on slow searches
[This is development, removed the Vim maillist] Yakov Lerner wrote: > On 4/29/07, Yakov Lerner <[EMAIL PROTECTED]> wrote: > > On 4/29/07, Bram Moolenaar <[EMAIL PROTECTED]> wrote: > > > > > > Yakov Lerner wrote: > > > > > > > Wish: when search is slow, show the progress line number > > > > every second on the bottom line (like, 12345 of 9). > > > > > > What is slow? > > To my taste, when something takes longer than 1-2 sec, > > I'd prefer some visual feedback on the progress. > > > > > Checking if the second passed will make the search even slower. > > > Checking time is quite slow on some systems (the check for CTRL-C > > > suffers from this). > > > > Checking for time every several hundred (N) lines will probably not slow > > the search perceptibly. N can be configurable, a parameter. > > Some value between 10 and 1000 will probaby be reasonable. > > I think it's possibe to check for time, which searching, not too > often and not too seldom, even without user-defined parameter. > Adaptive algorithm with two counters will find the right rate or > time-checking: > > - as we start search, we check time every 50 lines > (N=50 is initial value of N). We maintain counter M. M is how > many times we called time() between the seconds changed. > M is checked and reset every second. M is checked as folllows: > > - If M is too high (M>10), then we adjust N by increasing it. > If M is too low(M<10), then we adjust N by decreasing it. Ideally, > we want to check time() ~10 times per second. (overhead of 10 > calls to time() per second cannot he high, right ?) > > - if search progresses for several seconds, then N quickly converges > to the "ideal" value (~10 checks/sec). > > - we start every search with same value of N (say, 50). If search > is slow, then N will quickly converge to the "ideal" value for this regex, > the value in which where time() is checked ~10 times per second. It will help for differences in speed for various patterns, but it won't help for lines differing in length. Quite a few patterns with wildcards depend on the line length a lot. It might work better to adjust to the delay caused by the time function. This varies greatly between systems. When it's fast we can check the time often, when it's slow it delays the search more and should be called less often. - Bram -- hundred-and-one symptoms of being an internet addict: 33. You name your children Eudora, Mozilla and Dotcom. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: accessing vim's clipboard from java
java will have to do whatever any other application may do, without special knowledge about Vim. ... There's nothing Vim-specific there. Up to now jVi does exactly as you say. But now I want to get hold of the "structure" of a VIsual area, so a block paste can be implemented. In os_win32.c there is clip_star.format = RegisterClipboardFormat("VimClipboard2"); clip_star.format_raw = RegisterClipboardFormat("VimRawBytes"); In java I've registered "VimClipboard2", but I'm not picking it up. The name I'm using may not the right one, possible there's some translation going on. I've no experience with windows system programming. Or the java code is not quite right. I'm pursuing the java side. I'm wondering if any vim developers know the exact string that VimClipboard2 shows up as if you asked windows what formats were available. Thanks, -ernie A.J.Mechelynck wrote: Ernie Rael wrote: Hi all, I've just joined this list. I'm not a vim developer per se. However, I put together jVi, http://jvi.sourceforge.net , whose core is a port of some of vim to java. It runs on NetBeans ( and JBuilder, but not supported any more). jVi is based on vim-5.6. A jVi user has added visual char/line and is now tackling block. So now I want to setup access to the *real* vim clipboard, so blockmode paste works between vim and jVi. So I'm learning how to access "native" data formats in clipboard from java. I couldn't get anything to work, so I looked at the vim7 source, couldn't find windows source (I run cygwin, but not cygwin's vim, at home). I downloaded extras. I now see that the clipboard name has changed to VimClipboard2, and there's a whole lot more... So I changed the name, but still no luck. I thought I'd ping this list in case anyone has accessed the vim clipboard from java or has some clues. As an experiment I'm doing public static final String VIM_CLIPBOARD2 = "VimClipboard2"; SystemFlavorMap.addFlavorForUnencodedNative( ViManager.VIM_CLIPBOARD2, Misc.VimClipboardDataFlavor.get()); and the other way around. But after a paste into jVi from vim the DataFlavor I've defined is not available. Any clues? In os_mswin.c it looks like VimClipboard2 is always made available, so it should be there somewhere. Once I get something working, I'll look at doing all the various encodings... based on VimClipboard2. Or maybe, I'll use VimClipboard2 just to find out if it is MCHAR/MLINE/MBLOCK. Then go to the regular DataFlavors to pick the string type I'm looking for. -ernie IIUC, there is no specific interface between Vim and java, so java will have to do whatever any other application may do, without special knowledge about Vim. Whatever gvim writes to the + register is available in the system clipboard, just as if you'd used "Edit => Copy" in just about any application. There's nothing Vim-specific there. Best regards, Tony.
Re: wish: show search progress on slow searches
On 4/30/07, Bram Moolenaar <[EMAIL PROTECTED]> wrote: [This is development, removed the Vim maillist] Yakov Lerner wrote: > On 4/29/07, Yakov Lerner <[EMAIL PROTECTED]> wrote: > > On 4/29/07, Bram Moolenaar <[EMAIL PROTECTED]> wrote: > > > > > > Yakov Lerner wrote: > > > > > > > Wish: when search is slow, show the progress line number > > > > every second on the bottom line (like, 12345 of 9). > > > > > > What is slow? > > To my taste, when something takes longer than 1-2 sec, > > I'd prefer some visual feedback on the progress. > > > > > Checking if the second passed will make the search even slower. > > > Checking time is quite slow on some systems (the check for CTRL-C > > > suffers from this). > > > > Checking for time every several hundred (N) lines will probably not slow > > the search perceptibly. N can be configurable, a parameter. > > Some value between 10 and 1000 will probaby be reasonable. > > I think it's possibe to check for time, which searching, not too > often and not too seldom, even without user-defined parameter. > Adaptive algorithm with two counters will find the right rate or > time-checking: > > - as we start search, we check time every 50 lines > (N=50 is initial value of N). We maintain counter M. M is how > many times we called time() between the seconds changed. > M is checked and reset every second. M is checked as folllows: > > - If M is too high (M>10), then we adjust N by increasing it. > If M is too low(M<10), then we adjust N by decreasing it. Ideally, > we want to check time() ~10 times per second. (overhead of 10 > calls to time() per second cannot he high, right ?) > > - if search progresses for several seconds, then N quickly converges > to the "ideal" value (~10 checks/sec). > > - we start every search with same value of N (say, 50). If search > is slow, then N will quickly converge to the "ideal" value for this regex, > the value in which where time() is checked ~10 times per second. It will help for differences in speed for various patterns, but it won't help for lines differing in length. Quite a few patterns with wildcards depend on the line length a lot. It might work better to adjust to the delay caused by the time function. This varies greatly between systems. When it's fast we can check the time often, when it's slow it delays the search more and should be called less often. How about using SIGALRM when search is progressing, every second ? SIGALRM handler would store time() into global var (and reload alarm(1)). The search would check the global var for changes, every line. It is cheap to check global integer (time_t) var, no ? Yakov