Re: Vim slow after big count insert

2013-07-30 Fir de Conversatie Christian Brabandt
Hi Mike!

On So, 21 Jul 2013, Mike Williams wrote:

 On 20/07/2013 10:13, LCD 47 wrote:
  I believe this can be fixed with a counter that means something
 along the lines of: this line is longer than tw, and it has no
 breaking point for the first X characters.  Then X would be updated
 every time more text is appended to that line.

Are you going to create a patch for that?

 Yup to all that.  And there is another O(n^2) operation is getvcol()
 rescanning the line after each insert.

Not only that, but getvcol() is called twice from the edit() function. 
Once from validate_cursor_col() at line 695 and once from 
validate_cursor() at line 725. It seems like that either one of those 
calls could be saved.

regards,
Christian
-- 
Wie man sein Kind nicht nennen sollte: 
  Hans Dampf 

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-23 Fir de Conversatie Nikolay Pavlov
On Jul 23, 2013 9:24 AM, Gary Johnson garyj...@spocom.com wrote:

 On 2013-07-23, Nikolay Pavlov wrote:
 
  On Jul 20, 2013 2:28 AM, Gary Johnson wrote:
  
   On 2013-07-19, Dimitar DIMITROV wrote:
  
   Hi Dimitar,
  
   It would be a big help to those of us with threading mail readers if
   you would be sure that your replies include Re:  at the start of
   the Subject.
 
  This has nothing to do with threads. There is a special field added to
messages
  when you reply that contains unique ID of the message being replied to
and
  *this* field makes mail readers able to arrange messages correctly.

 I'm well aware of those fields, and my mail reader (mutt) handles
 them quite well.  It is also capable of threading messages that lack
 those fields and have only Re:  in front of the original subject
 to go by.  Dimitar's messages did not contain those headers, though,
 and since his Subject lines didn't even include the Re: , I
 assumed it would be a stretch for his mail client to do any more
 than allow him to add the Re: .

Good to know something new. One my mailer ignores everything but special
header, another splits threads on theme change (I did not realize I was
getting separate threads for this very reason though), but does not join
messages in threads if they lack field in question. So I assumed this field
is required for all mailers without performing more research.

 Regards,
 Gary

 --
 --
 You received this message from the vim_dev maillist.
 Do not top-post! Type your reply below the text you are replying to.
 For more information, visit http://www.vim.org/maillist.php

 ---
 You received this message because you are subscribed to the Google Groups
vim_dev group.
 To unsubscribe from this group and stop receiving emails from it, send an
email to vim_dev+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.



-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-22 Fir de Conversatie Charles Campbell

Dimitar DIMITROV wrote:

  There is no use case
 If you do something stupid by accident most vim operations can be
 aborted by ctrl-c (exception: python, rbuy, .. scripts)

Try to abort it you will see the success you have.

 So there is still nothing to fix or talk about unless there is a use
 case.

 Marc Weber

I tried this problem with both

  100iHelloesc
  100iHello esc

In both cases ctrl-c worked just fine to break the operation.

I used vim 7.4a.35 on a linux system when trying this.

Regards,
C Campbell

--
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups vim_dev group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-22 Fir de Conversatie Nikolay Pavlov
On Jul 20, 2013 2:28 AM, Gary Johnson garyj...@spocom.com wrote:

 On 2013-07-19, Dimitar DIMITROV wrote:

 Hi Dimitar,

 It would be a big help to those of us with threading mail readers if
 you would be sure that your replies include Re:  at the start of
 the Subject.

This has nothing to do with threads. There is a special field added to
messages when you reply that contains unique ID of the message being
replied to and *this* field makes mail readers able to arrange messages
correctly.

 Thanks,
 Gary

 --
 --
 You received this message from the vim_dev maillist.
 Do not top-post! Type your reply below the text you are replying to.
 For more information, visit http://www.vim.org/maillist.php

 ---
 You received this message because you are subscribed to the Google Groups
vim_dev group.
 To unsubscribe from this group and stop receiving emails from it, send an
email to vim_dev+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.



-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




[OT] Mail threading (Was: Re: Vim slow after big count insert)

2013-07-22 Fir de Conversatie James McCoy
On Tue, Jul 23, 2013 at 07:39:12AM +0400, Nikolay Pavlov wrote:
 On Jul 20, 2013 2:28 AM, Gary Johnson garyj...@spocom.com wrote:
 
  On 2013-07-19, Dimitar DIMITROV wrote:
 
  Hi Dimitar,
 
  It would be a big help to those of us with threading mail readers if
  you would be sure that your replies include Re:  at the start of
  the Subject.
 
 This has nothing to do with threads. There is a special field added to
 messages when you reply that contains unique ID of the message being
 replied to and *this* field makes mail readers able to arrange messages
 correctly.

That depends on the mail reader.  Even with MUAs that follow the
Message-ID/In-Reply-To chains, it's not uncommon to treat a topic change
as a new thread.  For ones that don't honor the IDs, the subject is
usually used instead.

Regardless, Dimitar's emails don't even have the In-Reply-To header set.
The X-Mailer header claims to be YahooMailWebService, so I doubt there's
much that he can do to fix it short of deciding to use a more standard,
standalone MUA instead of (I assume) their web interface.

Cheers,
-- 
James
GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy james...@jamessan.com

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-22 Fir de Conversatie Gary Johnson
On 2013-07-23, Nikolay Pavlov wrote:
 
 On Jul 20, 2013 2:28 AM, Gary Johnson wrote:
 
  On 2013-07-19, Dimitar DIMITROV wrote:
 
  Hi Dimitar,
 
  It would be a big help to those of us with threading mail readers if
  you would be sure that your replies include Re:  at the start of
  the Subject.
 
 This has nothing to do with threads. There is a special field added to 
 messages
 when you reply that contains unique ID of the message being replied to and
 *this* field makes mail readers able to arrange messages correctly.

I'm well aware of those fields, and my mail reader (mutt) handles
them quite well.  It is also capable of threading messages that lack
those fields and have only Re:  in front of the original subject
to go by.  Dimitar's messages did not contain those headers, though,
and since his Subject lines didn't even include the Re: , I
assumed it would be a stretch for his mail client to do any more
than allow him to add the Re: .

Regards,
Gary

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-21 Fir de Conversatie Mike Williams

On 20/07/2013 10:13, LCD 47 wrote:

On 19 July 2013, Mike Williams mike.willi...@globalgraphics.com wrote:

On 19/07/2013 15:52, Mike Williams wrote:

On 19/07/2013 12:18, Dimitar DIMITROV wrote:

Hi,

Did a search on the vim_dev archives but couldn't find anything
related to this. Sorry if this is redundant.  Basically vim is
exponentially slow after 100iHello esc as mentionned in this
link:
http://www.galexander.org/vim_sucks.html


A quick run over with a profiler seems to show most of the time is
being spent in vim_strchr() called from has_format_option(), called

from internal_format().  If I first do :set paste then it is a lot

quicker - for 4000iHelloESC it goes from 16s down to 1s.

[...]

Textwidth.  Darn sight quicker with it set to 0, wish I had
remembered that.  Another factor of ~2 quicker for my final times,
16000iHelloESC now takes ~4s.

[...]

 Right, what you're seeing here is Vim backtracking to find a blank
to split that N * Hello line.  Since there is no such blank, you get
len(Hello) * N * (N + 1) / 2 failed comparisons, which is exactly the
O(N^2) behaviour claimed in the article mentioned above.

 If you replace your test by, say, 16000iHello ESC (with a space
after Hello), you get linear behaviour, namely len(Hello) * N failed
comparisons.

 I believe this can be fixed with a counter that means something
along the lines of: this line is longer than tw, and it has no
breaking point for the first X characters.  Then X would be updated
every time more text is appended to that line.


Yup to all that.  And there is another O(n^2) operation is getvcol() 
rescanning the line after each insert.



On 19 July 2013, Marc Weber marco-owe...@gmx.de wrote:

Let's discuss the use case, first. Is there one ?


 Yes, there is: you run through that piece of code every time you
press p to paste text.  It hits you particularly hard when you edit
files with very long lines.


If not let's forgett about it ..

[...]

 Perhaps we should try to understand the problem before dismissing it
as harmless?


Indeed.  Part of the problem is that most computers these days have 
ridiculous amounts of processing power and storage that slow algorithms 
for small to medium amounts of data appear near instantaneous.  On say 
an Atom or Arm based netbook then the problem becomes more apparent. 
Getting a faster machine is not always the solution.


Mike
--
Born naked, wet and hungry... and it gets worse?

--
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups vim_dev group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-20 Fir de Conversatie LCD 47
On 19 July 2013, Mike Williams mike.willi...@globalgraphics.com wrote:
 On 19/07/2013 15:52, Mike Williams wrote:
 On 19/07/2013 12:18, Dimitar DIMITROV wrote:
 Hi,
 
 Did a search on the vim_dev archives but couldn't find anything
 related to this. Sorry if this is redundant.  Basically vim is
 exponentially slow after 100iHello esc as mentionned in this
 link:
 http://www.galexander.org/vim_sucks.html
 
 A quick run over with a profiler seems to show most of the time is
 being spent in vim_strchr() called from has_format_option(), called
 from internal_format().  If I first do :set paste then it is a lot
 quicker - for 4000iHelloESC it goes from 16s down to 1s.
[...]
 Textwidth.  Darn sight quicker with it set to 0, wish I had
 remembered that.  Another factor of ~2 quicker for my final times,
 16000iHelloESC now takes ~4s.
[...]

Right, what you're seeing here is Vim backtracking to find a blank
to split that N * Hello line.  Since there is no such blank, you get
len(Hello) * N * (N + 1) / 2 failed comparisons, which is exactly the
O(N^2) behaviour claimed in the article mentioned above.

If you replace your test by, say, 16000iHello ESC (with a space
after Hello), you get linear behaviour, namely len(Hello) * N failed
comparisons.

I believe this can be fixed with a counter that means something
along the lines of: this line is longer than tw, and it has no
breaking point for the first X characters.  Then X would be updated
every time more text is appended to that line.

On 19 July 2013, Marc Weber marco-owe...@gmx.de wrote:
 Let's discuss the use case, first. Is there one ?

Yes, there is: you run through that piece of code every time you
press p to paste text.  It hits you particularly hard when you edit
files with very long lines.

 If not let's forgett about it ..
[...]

Perhaps we should try to understand the problem before dismissing it
as harmless?

/lcd

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-19 Fir de Conversatie Marc Weber
Excerpts from Dimitar DIMITROV's message of Fri Jul 19 13:18:09 +0200 2013:
 Hi,
 
 Did a search on the vim_dev archives but couldn't find anything related to 
 this. Sorry if this is redundant.
 Basically vim is exponentially slow after 100iHello esc as mentionned 
 in this link:
 http://www.galexander.org/vim_sucks.html
Let's discuss the use case, first. Is there one ? If not let's forgett
about it ..

Vim sucks in many ways - but on average it is very helpful for me.

You could try c-r=repeat('Hello', insanely-large-int)cr as
alternative - maybe its faster, maybe its not, I don't want to wait
50 min for this case unless there is a use case.

Marc Weber

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-19 Fir de Conversatie Mike Williams

On 19/07/2013 12:18, Dimitar DIMITROV wrote:

Hi,

Did a search on the vim_dev archives but couldn't find anything related to 
this. Sorry if this is redundant.
Basically vim is exponentially slow after 100iHello esc as mentionned in 
this link:
http://www.galexander.org/vim_sucks.html


A quick run over with a profiler seems to show most of the time is being 
spent in vim_strchr() called from has_format_option(), called from 
internal_format().  If I first do :set paste then it is a lot quicker - 
for 4000iHelloESC it goes from 16s down to 1s.


Can part of the conditional for the while() looking for a position to 
break at in internal_format() (line 6186) be hoisted out as constant for 
the duration of the outer while loop?  Thinking of the control 
expression clauses:


(!fo_ins_blank  !has_format_option(FO_INS_VI))
|| (flags  INSCHAR_FORMAT)

This doesn't change the cost of the insert function but removes a large 
constant factor.


Re-running with the profiler then has the time spent in in plines_win() 
and its children, which is a little surprising since nothing is 
displayed until all the inserts have been done.  Turning wrap off speeds 
things up again.  With :set paste then 1iHelloESC takes ~9s. 
With :set paste nowrap then 1iHelloESC takes ~2s and 
16000iHelloESC takes ~7s, or doing 4x the first test in less than half 
the time  Re-enabling warp afterwards is instantaneous yet costs 7s for 
no gain during the repeated insert operation.


So it looks like all the extra work VIM does for formatting inserted 
text and displaying it is where the time is going, sometimes not 
efficiently and sometimes for no actual benefit.  Then again, is it 
reasonable to expect 25000iStringESC to be a typical operation (up 
there with editing 1GB log files)?


HTH - TTFN

Mike
--
Sympathy: What one woman offers another in exchange for the details.

--
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups vim_dev group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-19 Fir de Conversatie Mike Williams

On 19/07/2013 15:52, Mike Williams wrote:

On 19/07/2013 12:18, Dimitar DIMITROV wrote:

Hi,

Did a search on the vim_dev archives but couldn't find anything related to 
this. Sorry if this is redundant.
Basically vim is exponentially slow after 100iHello esc as mentionned in 
this link:
http://www.galexander.org/vim_sucks.html


A quick run over with a profiler seems to show most of the time is being
spent in vim_strchr() called from has_format_option(), called from
internal_format().  If I first do :set paste then it is a lot quicker -
for 4000iHelloESC it goes from 16s down to 1s.

Can part of the conditional for the while() looking for a position to
break at in internal_format() (line 6186) be hoisted out as constant for
the duration of the outer while loop?  Thinking of the control
expression clauses:

  (!fo_ins_blank  !has_format_option(FO_INS_VI))
|| (flags  INSCHAR_FORMAT)

This doesn't change the cost of the insert function but removes a large
constant factor.

Re-running with the profiler then has the time spent in in plines_win()
and its children, which is a little surprising since nothing is
displayed until all the inserts have been done.  Turning wrap off speeds
things up again.  With :set paste then 1iHelloESC takes ~9s.
With :set paste nowrap then 1iHelloESC takes ~2s and
16000iHelloESC takes ~7s, or doing 4x the first test in less than half
the time  Re-enabling warp afterwards is instantaneous yet costs 7s for
no gain during the repeated insert operation.

So it looks like all the extra work VIM does for formatting inserted
text and displaying it is where the time is going, sometimes not
efficiently and sometimes for no actual benefit.  Then again, is it
reasonable to expect 25000iStringESC to be a typical operation (up
there with editing 1GB log files)?

HTH - TTFN


Textwidth.  Darn sight quicker with it set to 0, wish I had remembered 
that.  Another factor of ~2 quicker for my final times, 16000iHelloESC 
now takes ~4s.


Obviously the fact that these settings has such an influence for large 
repetition counts for single line updates is not documented anywhere. 
Then again there are better ways of generating a 4MB file containing a 
single line of data - just cos you hammer a nail into wood with a 
spanner doesn't mean you should.


Mike
--
Sympathy: What one woman offers another in exchange for the details.

--
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups vim_dev group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-19 Fir de Conversatie Marc Weber
Excerpts from Dimitar DIMITROV's message of Fri Jul 19 23:14:44 +0200 2013:
 Try to abort it you will see the success you have.
1600iAutoescc-c aborts almost instantly
  VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Jun  9 2013 16:56:22)
  Included patches: 1-1155  

vim and gvim tested .. Have I done something wrong ?

Marc Weber

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Vim slow after big count insert

2013-07-19 Fir de Conversatie Gary Johnson
On 2013-07-19, Dimitar DIMITROV wrote:

Hi Dimitar,

It would be a big help to those of us with threading mail readers if
you would be sure that your replies include Re:  at the start of
the Subject.

Thanks,
Gary

-- 
-- 
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
vim_dev group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.