Re: Vim slow after big count insert
Hi Mike! On So, 21 Jul 2013, Mike Williams wrote: On 20/07/2013 10:13, LCD 47 wrote: I believe this can be fixed with a counter that means something along the lines of: this line is longer than tw, and it has no breaking point for the first X characters. Then X would be updated every time more text is appended to that line. Are you going to create a patch for that? Yup to all that. And there is another O(n^2) operation is getvcol() rescanning the line after each insert. Not only that, but getvcol() is called twice from the edit() function. Once from validate_cursor_col() at line 695 and once from validate_cursor() at line 725. It seems like that either one of those calls could be saved. regards, Christian -- Wie man sein Kind nicht nennen sollte: Hans Dampf -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On Jul 23, 2013 9:24 AM, Gary Johnson garyj...@spocom.com wrote: On 2013-07-23, Nikolay Pavlov wrote: On Jul 20, 2013 2:28 AM, Gary Johnson wrote: On 2013-07-19, Dimitar DIMITROV wrote: Hi Dimitar, It would be a big help to those of us with threading mail readers if you would be sure that your replies include Re: at the start of the Subject. This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and *this* field makes mail readers able to arrange messages correctly. I'm well aware of those fields, and my mail reader (mutt) handles them quite well. It is also capable of threading messages that lack those fields and have only Re: in front of the original subject to go by. Dimitar's messages did not contain those headers, though, and since his Subject lines didn't even include the Re: , I assumed it would be a stretch for his mail client to do any more than allow him to add the Re: . Good to know something new. One my mailer ignores everything but special header, another splits threads on theme change (I did not realize I was getting separate threads for this very reason though), but does not join messages in threads if they lack field in question. So I assumed this field is required for all mailers without performing more research. Regards, Gary -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
Dimitar DIMITROV wrote: There is no use case If you do something stupid by accident most vim operations can be aborted by ctrl-c (exception: python, rbuy, .. scripts) Try to abort it you will see the success you have. So there is still nothing to fix or talk about unless there is a use case. Marc Weber I tried this problem with both 100iHelloesc 100iHello esc In both cases ctrl-c worked just fine to break the operation. I used vim 7.4a.35 on a linux system when trying this. Regards, C Campbell -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On Jul 20, 2013 2:28 AM, Gary Johnson garyj...@spocom.com wrote: On 2013-07-19, Dimitar DIMITROV wrote: Hi Dimitar, It would be a big help to those of us with threading mail readers if you would be sure that your replies include Re: at the start of the Subject. This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and *this* field makes mail readers able to arrange messages correctly. Thanks, Gary -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[OT] Mail threading (Was: Re: Vim slow after big count insert)
On Tue, Jul 23, 2013 at 07:39:12AM +0400, Nikolay Pavlov wrote: On Jul 20, 2013 2:28 AM, Gary Johnson garyj...@spocom.com wrote: On 2013-07-19, Dimitar DIMITROV wrote: Hi Dimitar, It would be a big help to those of us with threading mail readers if you would be sure that your replies include Re: at the start of the Subject. This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and *this* field makes mail readers able to arrange messages correctly. That depends on the mail reader. Even with MUAs that follow the Message-ID/In-Reply-To chains, it's not uncommon to treat a topic change as a new thread. For ones that don't honor the IDs, the subject is usually used instead. Regardless, Dimitar's emails don't even have the In-Reply-To header set. The X-Mailer header claims to be YahooMailWebService, so I doubt there's much that he can do to fix it short of deciding to use a more standard, standalone MUA instead of (I assume) their web interface. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy james...@jamessan.com -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On 2013-07-23, Nikolay Pavlov wrote: On Jul 20, 2013 2:28 AM, Gary Johnson wrote: On 2013-07-19, Dimitar DIMITROV wrote: Hi Dimitar, It would be a big help to those of us with threading mail readers if you would be sure that your replies include Re: at the start of the Subject. This has nothing to do with threads. There is a special field added to messages when you reply that contains unique ID of the message being replied to and *this* field makes mail readers able to arrange messages correctly. I'm well aware of those fields, and my mail reader (mutt) handles them quite well. It is also capable of threading messages that lack those fields and have only Re: in front of the original subject to go by. Dimitar's messages did not contain those headers, though, and since his Subject lines didn't even include the Re: , I assumed it would be a stretch for his mail client to do any more than allow him to add the Re: . Regards, Gary -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On 20/07/2013 10:13, LCD 47 wrote: On 19 July 2013, Mike Williams mike.willi...@globalgraphics.com wrote: On 19/07/2013 15:52, Mike Williams wrote: On 19/07/2013 12:18, Dimitar DIMITROV wrote: Hi, Did a search on the vim_dev archives but couldn't find anything related to this. Sorry if this is redundant. Basically vim is exponentially slow after 100iHello esc as mentionned in this link: http://www.galexander.org/vim_sucks.html A quick run over with a profiler seems to show most of the time is being spent in vim_strchr() called from has_format_option(), called from internal_format(). If I first do :set paste then it is a lot quicker - for 4000iHelloESC it goes from 16s down to 1s. [...] Textwidth. Darn sight quicker with it set to 0, wish I had remembered that. Another factor of ~2 quicker for my final times, 16000iHelloESC now takes ~4s. [...] Right, what you're seeing here is Vim backtracking to find a blank to split that N * Hello line. Since there is no such blank, you get len(Hello) * N * (N + 1) / 2 failed comparisons, which is exactly the O(N^2) behaviour claimed in the article mentioned above. If you replace your test by, say, 16000iHello ESC (with a space after Hello), you get linear behaviour, namely len(Hello) * N failed comparisons. I believe this can be fixed with a counter that means something along the lines of: this line is longer than tw, and it has no breaking point for the first X characters. Then X would be updated every time more text is appended to that line. Yup to all that. And there is another O(n^2) operation is getvcol() rescanning the line after each insert. On 19 July 2013, Marc Weber marco-owe...@gmx.de wrote: Let's discuss the use case, first. Is there one ? Yes, there is: you run through that piece of code every time you press p to paste text. It hits you particularly hard when you edit files with very long lines. If not let's forgett about it .. [...] Perhaps we should try to understand the problem before dismissing it as harmless? Indeed. Part of the problem is that most computers these days have ridiculous amounts of processing power and storage that slow algorithms for small to medium amounts of data appear near instantaneous. On say an Atom or Arm based netbook then the problem becomes more apparent. Getting a faster machine is not always the solution. Mike -- Born naked, wet and hungry... and it gets worse? -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On 19 July 2013, Mike Williams mike.willi...@globalgraphics.com wrote: On 19/07/2013 15:52, Mike Williams wrote: On 19/07/2013 12:18, Dimitar DIMITROV wrote: Hi, Did a search on the vim_dev archives but couldn't find anything related to this. Sorry if this is redundant. Basically vim is exponentially slow after 100iHello esc as mentionned in this link: http://www.galexander.org/vim_sucks.html A quick run over with a profiler seems to show most of the time is being spent in vim_strchr() called from has_format_option(), called from internal_format(). If I first do :set paste then it is a lot quicker - for 4000iHelloESC it goes from 16s down to 1s. [...] Textwidth. Darn sight quicker with it set to 0, wish I had remembered that. Another factor of ~2 quicker for my final times, 16000iHelloESC now takes ~4s. [...] Right, what you're seeing here is Vim backtracking to find a blank to split that N * Hello line. Since there is no such blank, you get len(Hello) * N * (N + 1) / 2 failed comparisons, which is exactly the O(N^2) behaviour claimed in the article mentioned above. If you replace your test by, say, 16000iHello ESC (with a space after Hello), you get linear behaviour, namely len(Hello) * N failed comparisons. I believe this can be fixed with a counter that means something along the lines of: this line is longer than tw, and it has no breaking point for the first X characters. Then X would be updated every time more text is appended to that line. On 19 July 2013, Marc Weber marco-owe...@gmx.de wrote: Let's discuss the use case, first. Is there one ? Yes, there is: you run through that piece of code every time you press p to paste text. It hits you particularly hard when you edit files with very long lines. If not let's forgett about it .. [...] Perhaps we should try to understand the problem before dismissing it as harmless? /lcd -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
Excerpts from Dimitar DIMITROV's message of Fri Jul 19 13:18:09 +0200 2013: Hi, Did a search on the vim_dev archives but couldn't find anything related to this. Sorry if this is redundant. Basically vim is exponentially slow after 100iHello esc as mentionned in this link: http://www.galexander.org/vim_sucks.html Let's discuss the use case, first. Is there one ? If not let's forgett about it .. Vim sucks in many ways - but on average it is very helpful for me. You could try c-r=repeat('Hello', insanely-large-int)cr as alternative - maybe its faster, maybe its not, I don't want to wait 50 min for this case unless there is a use case. Marc Weber -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On 19/07/2013 12:18, Dimitar DIMITROV wrote: Hi, Did a search on the vim_dev archives but couldn't find anything related to this. Sorry if this is redundant. Basically vim is exponentially slow after 100iHello esc as mentionned in this link: http://www.galexander.org/vim_sucks.html A quick run over with a profiler seems to show most of the time is being spent in vim_strchr() called from has_format_option(), called from internal_format(). If I first do :set paste then it is a lot quicker - for 4000iHelloESC it goes from 16s down to 1s. Can part of the conditional for the while() looking for a position to break at in internal_format() (line 6186) be hoisted out as constant for the duration of the outer while loop? Thinking of the control expression clauses: (!fo_ins_blank !has_format_option(FO_INS_VI)) || (flags INSCHAR_FORMAT) This doesn't change the cost of the insert function but removes a large constant factor. Re-running with the profiler then has the time spent in in plines_win() and its children, which is a little surprising since nothing is displayed until all the inserts have been done. Turning wrap off speeds things up again. With :set paste then 1iHelloESC takes ~9s. With :set paste nowrap then 1iHelloESC takes ~2s and 16000iHelloESC takes ~7s, or doing 4x the first test in less than half the time Re-enabling warp afterwards is instantaneous yet costs 7s for no gain during the repeated insert operation. So it looks like all the extra work VIM does for formatting inserted text and displaying it is where the time is going, sometimes not efficiently and sometimes for no actual benefit. Then again, is it reasonable to expect 25000iStringESC to be a typical operation (up there with editing 1GB log files)? HTH - TTFN Mike -- Sympathy: What one woman offers another in exchange for the details. -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On 19/07/2013 15:52, Mike Williams wrote: On 19/07/2013 12:18, Dimitar DIMITROV wrote: Hi, Did a search on the vim_dev archives but couldn't find anything related to this. Sorry if this is redundant. Basically vim is exponentially slow after 100iHello esc as mentionned in this link: http://www.galexander.org/vim_sucks.html A quick run over with a profiler seems to show most of the time is being spent in vim_strchr() called from has_format_option(), called from internal_format(). If I first do :set paste then it is a lot quicker - for 4000iHelloESC it goes from 16s down to 1s. Can part of the conditional for the while() looking for a position to break at in internal_format() (line 6186) be hoisted out as constant for the duration of the outer while loop? Thinking of the control expression clauses: (!fo_ins_blank !has_format_option(FO_INS_VI)) || (flags INSCHAR_FORMAT) This doesn't change the cost of the insert function but removes a large constant factor. Re-running with the profiler then has the time spent in in plines_win() and its children, which is a little surprising since nothing is displayed until all the inserts have been done. Turning wrap off speeds things up again. With :set paste then 1iHelloESC takes ~9s. With :set paste nowrap then 1iHelloESC takes ~2s and 16000iHelloESC takes ~7s, or doing 4x the first test in less than half the time Re-enabling warp afterwards is instantaneous yet costs 7s for no gain during the repeated insert operation. So it looks like all the extra work VIM does for formatting inserted text and displaying it is where the time is going, sometimes not efficiently and sometimes for no actual benefit. Then again, is it reasonable to expect 25000iStringESC to be a typical operation (up there with editing 1GB log files)? HTH - TTFN Textwidth. Darn sight quicker with it set to 0, wish I had remembered that. Another factor of ~2 quicker for my final times, 16000iHelloESC now takes ~4s. Obviously the fact that these settings has such an influence for large repetition counts for single line updates is not documented anywhere. Then again there are better ways of generating a 4MB file containing a single line of data - just cos you hammer a nail into wood with a spanner doesn't mean you should. Mike -- Sympathy: What one woman offers another in exchange for the details. -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
Excerpts from Dimitar DIMITROV's message of Fri Jul 19 23:14:44 +0200 2013: Try to abort it you will see the success you have. 1600iAutoescc-c aborts almost instantly VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Jun 9 2013 16:56:22) Included patches: 1-1155 vim and gvim tested .. Have I done something wrong ? Marc Weber -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Vim slow after big count insert
On 2013-07-19, Dimitar DIMITROV wrote: Hi Dimitar, It would be a big help to those of us with threading mail readers if you would be sure that your replies include Re: at the start of the Subject. Thanks, Gary -- -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups vim_dev group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.