Re: Patch 7.1.235

2008-01-20 Fir de Conversatie Bram Moolenaar


Alexei Alexandrov wrote:

> Bram Moolenaar wrote:
> > 
> > Patch 7.1.235
> > Problem:Pattern matching is slow when using a lot of simple patterns.
> > Solution:   Avoid allocating memory by not freeing it when it's not so much.
> > (Alexei Alexandrov)
> > Files:  src/regexp.c
> 
> Thanks a lot!
> You had to rework it somewhat as I can see, right?

The only essential thing I added was freeing "reg_tofree" when it's
long.

> I'm also curious what happens with the DFA regexp engine rework?

It's still being worked on.  It's probably not going to help for simple
patterns, might even make it slower.  But it will help a lot for
patterns with things like ".*".

-- 
(letter from Mark to Mike, about the film's probable certificate)
  For an 'A' we would have to: Lose as many shits as possible; Take Jesus
  Christ out, if possible; Loose "I fart in your general direction"; Lose
  "the oral sex"; Lose "oh, fuck off"; Lose "We make castanets out of your
  testicles"
 "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

 /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net   \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\download, build and distribute -- http://www.A-A-P.org///
 \\\help me help AIDS victims -- http://ICCF-Holland.org///

--~--~-~--~~~---~--~~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---



Re: Patch 7.1.235

2008-01-19 Fir de Conversatie Alexei Alexandrov

Bram Moolenaar wrote:
> 
> Patch 7.1.235
> Problem:Pattern matching is slow when using a lot of simple patterns.
> Solution:   Avoid allocating memory by not freeing it when it's not so much.
>   (Alexei Alexandrov)
> Files:src/regexp.c
> 

Thanks a lot!
You had to rework it somewhat as I can see, right?
I'm also curious what happens with the DFA regexp engine rework?

-- 
Alexei Alexandrov


--~--~-~--~~~---~--~~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---



Patch 7.1.235

2008-01-18 Fir de Conversatie Bram Moolenaar


Patch 7.1.235
Problem:Pattern matching is slow when using a lot of simple patterns.
Solution:   Avoid allocating memory by not freeing it when it's not so much.
(Alexei Alexandrov)
Files:  src/regexp.c


*** ../vim-7.1.234/src/regexp.c Wed Jan  2 15:34:48 2008
--- src/regexp.cFri Jan 18 20:35:21 2008
***
*** 378,391 
  
  static char_u *reg_prev_sub = NULL;
  
- #if defined(EXITFREE) || defined(PROTO)
- void
- free_regexp_stuff()
- {
- vim_free(reg_prev_sub);
- }
- #endif
- 
  /*
   * REGEXP_INRANGE contains all characters which are always special in a []
   * range after '\'.
--- 378,383 
***
*** 3206,3217 
  } backpos_T;
  
  /*
!  * regstack and backpos are used by regmatch().  They are kept over calls to
!  * avoid invoking malloc() and free() often.
   */
! static garray_T   regstack;   /* stack with regitem_T items, sometimes
!  preceded by regstar_T or regbehind_T. */
! static garray_T   backpos;/* table with backpos_T for BACK */
  
  /*
   * Get pointer to the line "lnum", which is relative to "reg_firstlnum".
--- 3198,3236 
  } backpos_T;
  
  /*
!  * "regstack" and "backpos" are used by regmatch().  They are kept over calls
!  * to avoid invoking malloc() and free() often.
!  * "regstack" is a stack with regitem_T items, sometimes preceded by regstar_T
!  * or regbehind_T.
!  * "backpos_T" is a table with backpos_T for BACK
!  */
! static garray_T   regstack = {0, 0, 0, 0, NULL};
! static garray_T   backpos = {0, 0, 0, 0, NULL};
! 
! /*
!  * Both for regstack and backpos tables we use the following strategy of
!  * allocation (to reduce malloc/free calls):
!  * - Initial size is fairly small.
!  * - When needed, the tables are grown bigger (8 times at first, double after
!  *   that).
!  * - After executing the match we free the memory only if the array has grown.
!  *   Thus the memory is kept allocated when it's at the initial size.
!  * This makes it fast while not keeping a lot of memory allocated.
!  * A three times speed increase was observed when using many simple patterns.
   */
! #define REGSTACK_INITIAL  2048
! #define BACKPOS_INITIAL   64
! 
! #if defined(EXITFREE) || defined(PROTO)
! void
! free_regexp_stuff()
! {
! ga_clear(®stack);
! ga_clear(&backpos);
! vim_free(reg_tofree);
! vim_free(reg_prev_sub);
! }
! #endif
  
  /*
   * Get pointer to the line "lnum", which is relative to "reg_firstlnum".
***
*** 3346,3360 
  char_u*s;
  long  retval = 0L;
  
! reg_tofree = NULL;
! 
! /* Init the regstack empty.  Use an item size of 1 byte, since we push
!  * different things onto it.  Use a large grow size to avoid reallocating
!  * it too often. */
! ga_init2(®stack, 1, 1);
! 
! /* Init the backpos table empty. */
! ga_init2(&backpos, sizeof(backpos_T), 10);
  
  if (REG_MULTI)
  {
--- 3365,3389 
  char_u*s;
  long  retval = 0L;
  
! /* Create "regstack" and "backpos" if they are not allocated yet.
!  * We allocate *_INITIAL amount of bytes first and then set the grow size
!  * to much bigger value to avoid many malloc calls in case of deep regular
!  * expressions.  */
! if (regstack.ga_data == NULL)
! {
!   /* Use an item size of 1 byte, since we push different things
!* onto the regstack. */
!   ga_init2(®stack, 1, REGSTACK_INITIAL);
!   ga_grow(®stack, REGSTACK_INITIAL);
!   regstack.ga_growsize = REGSTACK_INITIAL * 8;
! }
! 
! if (backpos.ga_data == NULL)
! {
!   ga_init2(&backpos, sizeof(backpos_T), BACKPOS_INITIAL);
!   ga_grow(&backpos, BACKPOS_INITIAL);
!   backpos.ga_growsize = BACKPOS_INITIAL * 8;
! }
  
  if (REG_MULTI)
  {
***
*** 3525,3533 
  }
  
  theend:
! vim_free(reg_tofree);
! ga_clear(®stack);
! ga_clear(&backpos);
  
  return retval;
  }
--- 3554,3570 
  }
  
  theend:
! /* Free "reg_tofree" when it's a bit big.
!  * Free regstack and backpos if they are bigger than their initial size. 
*/
! if (reg_tofreelen > 400)
! {
!   vim_free(reg_tofree);
!   reg_tofree = NULL;
! }
! if (regstack.ga_maxlen > REGSTACK_INITIAL)
!   ga_clear(®stack);
! if (backpos.ga_maxlen > BACKPOS_INITIAL)
!   ga_clear(&backpos);
  
  return retval;
  }
***
*** 3717,3724 
  #define RA_MATCH  4   /* successful match */
  #define RA_NOMATCH5   /* didn't match */
  
!   /* Init the regstack and backpos table empty.  They are initialized and
!* freed in vim_regexec_both() to reduce malloc()/free() calls. */
regstack.ga_len = 0;
backpos.ga_le