Re: 2html optimization
On 16/06/11 21:33, Benjamin Fritz wrote: On Wed, Jun 15, 2011 at 1:22 PM, Benjamin Koltaibkolta...@gmail.com wrote: Also, the result of :version is: VIM - Vi IMproved 7.2 (2008 Aug 9, compiled Jul 5 2010 16:33:43) Included patches: 1-127, 257 Compiled by 'http://www.opensuse.org/' Is this out of date? What is the easiest way to update? I don't think we've responded to this, yet. Yes, this is out of date. The latest is 7.3.219, and a lot of new features have been added (to Vim itself as well as to 2html). Some of my other answers (especially the options which might affect speed) depend on this recent version of 2html. The easiest way, would be if your Linux distribution has up-to-date packages in their repository. But often these are fairly out of date. Check the repository first, but failing that, you can compile Vim yourself fairly easily. The source code is available through Mercurial from: http://code.google.com/p/vim There should be a readme there that says how to compile (it's pretty standard). If you get stuck you can google for building Vim or check the wiki: http://vim.wikia.com/wiki/Category:Building_Vim The current Vim in the openSUSE distribution is 7.3.62 which is still a lot behind the times. I recommend compiling your own, see http://vim.wikia.com/wiki/Getting_the_Vim_source_with_Mercurial http://users.skynet.be/antoine.mechelynck/vim/compunix.htm Best regards, Tony. -- Actor: So what do you do for a living? Doris: I work for a company that makes deceptively shallow serving dishes for Chinese restaurants. -- Woody Allen, Without Feathers -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Wed, Jun 15, 2011 at 1:22 PM, Benjamin Koltai bkolta...@gmail.com wrote: Also, the result of :version is: VIM - Vi IMproved 7.2 (2008 Aug 9, compiled Jul 5 2010 16:33:43) Included patches: 1-127, 257 Compiled by 'http://www.opensuse.org/' Is this out of date? What is the easiest way to update? I don't think we've responded to this, yet. Yes, this is out of date. The latest is 7.3.219, and a lot of new features have been added (to Vim itself as well as to 2html). Some of my other answers (especially the options which might affect speed) depend on this recent version of 2html. The easiest way, would be if your Linux distribution has up-to-date packages in their repository. But often these are fairly out of date. Check the repository first, but failing that, you can compile Vim yourself fairly easily. The source code is available through Mercurial from: http://code.google.com/p/vim There should be a readme there that says how to compile (it's pretty standard). If you get stuck you can google for building Vim or check the wiki: http://vim.wikia.com/wiki/Category:Building_Vim -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Great thanks for that.
Do you have any idea how I am get past the prompt to press enter with the last
command you suggested? Why is it happening?
Here is the prompt again:
$ HOME=/path/to/vimfiles/ vim -N -u /path/to/config.vim -i NONE --noplugin -c
'runtime syntax/2html.vim' -c 'xa' {file}
cookies.php.html [New File]
Press ENTER or type command to continue
With config.vim:
filetype on
runtime colors/simplewhite.vim
colorscheme simplewhite
set number
syntax on
runtime plugin/TOhtml.vim
Thanks again
On Jun 16, 2011, at 3:33 PM, Benjamin Fritz wrote:
On Wed, Jun 15, 2011 at 1:22 PM, Benjamin Koltai bkolta...@gmail.com wrote:
Also, the result of :version is:
VIM - Vi IMproved 7.2 (2008 Aug 9, compiled Jul 5 2010 16:33:43)
Included patches: 1-127, 257
Compiled by 'http://www.opensuse.org/'
Is this out of date? What is the easiest way to update?
I don't think we've responded to this, yet.
Yes, this is out of date. The latest is 7.3.219, and a lot of new
features have been added (to Vim itself as well as to 2html). Some of
my other answers (especially the options which might affect speed)
depend on this recent version of 2html.
The easiest way, would be if your Linux distribution has up-to-date
packages in their repository. But often these are fairly out of date.
Check the repository first, but failing that, you can compile Vim
yourself fairly easily. The source code is available through Mercurial
from:
http://code.google.com/p/vim
There should be a readme there that says how to compile (it's pretty
standard). If you get stuck you can google for building Vim or check
the wiki:
http://vim.wikia.com/wiki/Category:Building_Vim
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Thu, Jun 16, 2011 at 4:26 PM, Benjamin Koltai bkolta...@gmail.com wrote:
Do you have any idea how I am get past the prompt to press enter with the
last command you suggested? Why is it happening?
Here is the prompt again:
$ HOME=/path/to/vimfiles/ vim -N -u /path/to/config.vim -i NONE --noplugin -c
'runtime syntax/2html.vim' -c 'xa' {file}
cookies.php.html [New File]
Press ENTER or type command to continue
Try some of the hints here:
http://vim.wikia.com/wiki/Press_enter_to_continue
Setting nomore in your config.vim file might help as well.
I would also recommend some other options pointed out by Zyx,
especially undolevels=0 and nobackup, for performance reasons.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Will do. Thanks. I was meaning to add those extra options once I got it working
with the minimums.
Thanks everyone for all the help!
On Jun 16, 2011, at 5:42 PM, Benjamin Fritz wrote:
On Thu, Jun 16, 2011 at 4:26 PM, Benjamin Koltai bkolta...@gmail.com wrote:
Do you have any idea how I am get past the prompt to press enter with the
last command you suggested? Why is it happening?
Here is the prompt again:
$ HOME=/path/to/vimfiles/ vim -N -u /path/to/config.vim -i NONE --noplugin
-c 'runtime syntax/2html.vim' -c 'xa' {file}
cookies.php.html [New File]
Press ENTER or type command to continue
Try some of the hints here:
http://vim.wikia.com/wiki/Press_enter_to_continue
Setting nomore in your config.vim file might help as well.
I would also recommend some other options pointed out by Zyx,
especially undolevels=0 and nobackup, for performance reasons.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Jun 14, 11:00 pm, Andy Spencer andy753...@gmail.com wrote:
On 2011-06-14 16:23, Benjamin Koltai wrote:
I am currently using the command:
HOME=#{ CONFIG.paths.lib_dir }/vimfiles/ vi -n -X '+runtime!
syntax/2html.vim' '+xa' filename
Using '+runtime! syntax/2html.vim' is acting funny on my computer.
I haven't looked into it, but it's generating html for filename, and
then generating another html for filename.html and calling that one
filename.html.html.. Using '+runtime' instead of '+runtime!' or just
using +TOhtml seems to fix it..
On my system, I have a ~/.vim/syntax/2html.vim, and also the default
$VIMRUNTIME/syntax/2html.vim. When I use +runtime! syntax/2html.vim
I see what you describe, as I would expect, because the '!' in the
runtime command says source all files found. When I omit the '!' it
works as intended, and only does a single conversion.
I tried -e but I do not get the syntax highlighting with that option.
You can force ex to use syntax highlighting with a few tricks. You'll probably
need to run it in screen if it's being run by a daemon, and you'll need to
force the value of 'term' in vimrc. The gist of the command I use is:
HOME=/home/andy screen -D -m ex -nXZ -i NONE -u vimrc '+sil! TOhtml' ...
I have not tested 2html with Vim in ex mode. I would not be surprised
if there are some problems with this. If it works, that's pretty cool!
Do let me know if you see problems running 2html in ex mode, I imagine
that might speed things up some.
You can also speed it up a little but by using:
let g:html_no_progress = 1
Other options which might speed things up slightly (if their effects
are acceptable):
g:html_ignore_folding = 1 or g:html_dynamic_folds = 0 (otherwise Vim
scans every line for fold information, twice, before starting the
conversion)
g:html_use_encoding = UTF-8 (or whatever encoding you desire) to
avoid Vim needing to determine the encoding to use (probably VERY
minor savings, but it can also avoid a conversion on write if the
specified encoding matches your 'encoding' setting)
g:html_expand_tabs = 0 (probably very minor savings, but this will
prevent Vim from doing processing on tab characters to replace them
with spaces; mostly this is useful if you do NOT use line numbers or a
fold column in the generated output)
More savings can be had by making sure that folding (especially syntax-
based folding) is disabled, especially for HTML files. Disabling
syntax completely for HTML files will also speed things up. Maybe
after loading the file to convert you can set the eventignore option
to disable filetype and syntax events from triggering on the HTML file
being generated.
For reference, the full command I use and the vimrc are posted here:
-http://vpaste.net/index.cgi?ft=sh
-http://vpaste.net/vimrc?ft=vim
I notice that these references use the dynamic_folds option, but there
are no folds in the document. See my note above, you could probably
make this faster by setting the ignore_folding option!
Some of these options I suggest above were added/augmented in later
versions of the TOhtml plugin than you are using. I think that v9 is
currently in the Hg repository, and I've given v10 to Bram, but you're
still using v6 according to the meta tags in the generated output.
Also note the html_use_encoding option takes an IANA name (i.e. UTF-8
rather than utf8). Perhaps I should add an auto-correction for
that...I'll think about it.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Jun 14, 3:23 pm, Benjamin Koltai bkolta...@gmail.com wrote: I am using vim to syntax highlight and convert plain text files to html. What sort of syntax highlighting are you using for your plain text files? By default plain text receives no syntax highlighting. TOhtml's primary purpose is to put the syntax-highlighting colors into the generated HTML. Does the file have to load in order for it to get highlighted? Yes, a file is only syntax highlighted if Vim has loaded it. Perhaps if you know files will always be of a certain type, you could speed things along by turning off filetype/syntax detection and sourcing the correct scripts yourself (maybe in file-extension based autocmds even) to avoid the filetype detection logic. This would make it easier to implement the turn off syntax highlighting for HTML I suggested in my previous response as well. I also saw something online about a vim client server? Would that be something that would allow me to keep a vim instance running and send files to be converted to that server? Would that speed things up? If you only ever convert 1 file at a time, this will probably speed things up if done correctly, because Vim will not need to spend time starting up. You can use the --startuptime option to Vim a few times to see if the savings will be worth it. If you could potentially be converting multiple files at once (you say you get multiple requests at once), then a single Vim instance will become a bottleneck. Since Vim is not multi-threaded, it must process each file one at a time. You could set up multipe always running Vims to work around this, but then you'd need some logic outside of Vim to select the correct one or spawn a new one if needed. -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Thanks for all the help so far. See below for my responses. On Jun 15, 2011, at 11:53 AM, Ben Fritz wrote: On Jun 14, 3:23 pm, Benjamin Koltai bkolta...@gmail.com wrote: I am using vim to syntax highlight and convert plain text files to html. What sort of syntax highlighting are you using for your plain text files? By default plain text receives no syntax highlighting. TOhtml's primary purpose is to put the syntax-highlighting colors into the generated HTML. I am using vim to syntax highlight various file type extensions. My classification of a file as plain text is a misnomer, I meant files that are just plain text such as .c, .h, .rb... TOhtml has been doing what I want. Does the file have to load in order for it to get highlighted? Yes, a file is only syntax highlighted if Vim has loaded it. Perhaps if you know files will always be of a certain type, you could speed things along by turning off filetype/syntax detection and sourcing the correct scripts yourself (maybe in file-extension based autocmds even) to avoid the filetype detection logic. This would make it easier to implement the turn off syntax highlighting for HTML I suggested in my previous response as well. One of the main reasons I am using vim is so that I can let it auto-detect the type of file that it is opening and apply the correct syntax highlighting. If this is the part that is taking a while (which is understandable) than I will just have to deal with it, because it is the main reason why I am using vim to do this conversion. I also saw something online about a vim client server? Would that be something that would allow me to keep a vim instance running and send files to be converted to that server? Would that speed things up? If you only ever convert 1 file at a time, this will probably speed things up if done correctly, because Vim will not need to spend time starting up. You can use the --startuptime option to Vim a few times to see if the savings will be worth it. I would like to see this metric but could not get the --startuptime option to be recognized: vim --startuptime file I also looked in the help for startuptime and in the man page, and couldn't find such a thing? What am I missing? If you could potentially be converting multiple files at once (you say you get multiple requests at once), then a single Vim instance will become a bottleneck. Since Vim is not multi-threaded, it must process each file one at a time. You could set up multipe always running Vims to work around this, but then you'd need some logic outside of Vim to select the correct one or spawn a new one if needed. Ok. That is good to know that Vim is not multi-threaded. I am still a little unclear on what the benefits of using a client server are? Is it so that you don't have to start up a single vim instance every time and can just leave vim running? Also, is there any limit on how many vim instances I can have running at one time? I believe my current system just start a new vim instance for every file that it gets. The instance only lasts as long as it takes to open the file, highlight it, and close it. It is starting to sound like the bottleneck might be the syntax detection and actual coloring of the document. If this is the case than I can deal with it because it is the main feature I am after. I guess I was just wondering if there might be other plugins/initializers that might be running that I could squash to save time. I have tried running vim with -V, but the output is a little cryptic. I do see that it sources files such as /usr/share/vim/... and I am not sure if those files are required for what I am trying to do or not? Thanks again for all the suggestions! Benji -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Wed, Jun 15, 2011 at 10:37 AM, Ben Fritz fritzophre...@gmail.com wrote: You can also speed it up a little but by using: let g:html_no_progress = 1 Other options which might speed things up slightly (if their effects are acceptable): g:html_ignore_folding = 1 or g:html_dynamic_folds = 0 (otherwise Vim scans every line for fold information, twice, before starting the conversion) g:html_use_encoding = UTF-8 (or whatever encoding you desire) to avoid Vim needing to determine the encoding to use (probably VERY minor savings, but it can also avoid a conversion on write if the specified encoding matches your 'encoding' setting) g:html_expand_tabs = 0 (probably very minor savings, but this will prevent Vim from doing processing on tab characters to replace them with spaces; mostly this is useful if you do NOT use line numbers or a fold column in the generated output) One more to try, if you don't care about conceal support: g:html_ignore_conceal = 1 This will eliminate at least one check per syntax item, and I suspect (but have not verified) that the inner loop for non-concealed items (always used with ignore_conceal set) is faster than for concealed items. But again, this is probably minor savings. The biggest savings will probably be from the html_no_progress option, and from disabling syntax highlighting on the generated HTML output. I'm interested to learn what you end up with. I thought a few times about a use case like this, but decided it was unlikely because TOhtml is pretty slow for big files. Also note, Zyx has a plugin very similar to 2html, with a slightly expanded feature set, several possible output formats, and probably a very different implementation. If you do go with a Vim method, you can try it out. I don't think there's been very much comparison done between the two, it could be faster (or it could be slower, I really have no idea). http://www.vim.org/scripts/script.php?script_id=3113 Also, I have an item on my TODO list to allow the HTML output to use an external CSS file. My goal is that this external file can be generated just once and then used over and over, and that by not processing the CSS every time additional speed increases will be possible. I do not have an ETA on this feature, however; it has mostly stalled in the design stage until I find time to work on it again. If this is interesting enough I can probably put some other features on hold which I had planned on getting to first (automatic font detection, and uncopyable regions for things like line number and the fold column). -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Wed, Jun 15, 2011 at 11:12 AM, Benjamin Koltai bkolta...@gmail.com wrote:
I would like to see this metric but could not get the --startuptime option to
be recognized:
vim --startuptime file
I also looked in the help for startuptime and in the man page, and couldn't
find such a thing? What am I missing?
Possibly your Vim was not compiled with this option. It's part of a
normal build thogh, and it's missing from your runtime help files,
so more likely you're just using an old version of Vim in general.
What are the first few lines of :version output?
Ok. That is good to know that Vim is not multi-threaded. I am still a little
unclear on what the benefits of using a client server are?
Is it so that you don't have to start up a single vim instance every time and
can just leave vim running?
The main benefit is so you can use the same Vim for everything. It's
very useful if you're actually using Vim for editing and you want to
just work in one instance instead of finding the right one for every
file you're working on. It lets you launch new files in the same Vim
using the shell. It also allows for some interesting inter-process
communication, including using the shell to do background tasks in
Vim, even though Vim by itself is single-threaded.
Also, is there any limit on how many vim instances I can have running at one
time?
I don't think so, just the extra processor and memory it would incur.
Your method of closing each Vim instance when done also has the
benefit that you don't need to worry about resources Vim keeps around
for each buffer (even when that buffer is no longer shown). It will
take a little extra logic (probably just a :bwipeout command at the
end) to work with a single Vim instance.
It is starting to sound like the bottleneck might be the syntax detection and
actual coloring of the document. If this is the case than I can deal with it
because it is the main feature I am after. I guess I was just wondering if
there might be other plugins/initializers that might be running that I could
squash to save time. I have tried running vim with -V, but the output is a
little cryptic. I do see that it sources files such as /usr/share/vim/... and
I am not sure if those files are required for what I am trying to do or not?
When converting from the command line, this works for me:
vim -N -u NONE -i NONE --cmd filetype on --cmd syntax on +runtime
plugin/TOhtml.vim +runtime syntax/2html.vim {file name}
This disables ALL plugins and customizations, then specifically
enables syntax highlight and TOhtml.
I didn't realize I needed to source the plugin file before running
2html.vim...that is probably something for me to fix. I think it's
just used for a meta tag, though.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Hi Ben,
Thank you so much for all the help with this. I really appreciate it.
I ran the following command:
HOME=/path/to//directory/with/vimrc/ vim -N -u NONE -i NONE --cmd filetype on
--cmd syntax on +runtime plugin/TOhtml.vim +runtime syntax/2html.vim
file
This opened a terminal vim window with file and the html genreated from that
file. In the vim window that opened, my ~/.vimrc seemed to be applied, but when
I did :xa and saved both files, and opened the new file.html, it seems that
the colorscheme set in the vimrc in the path set to home (not my ~/.vimrc) were
applied, which is great!!
I feel that I am very close to what I want to achieve, I just have a few more
questions (because I am still very new to Vim and do not fully understand
everything yet, so please excuse my noobing):
1. I have set number in the vimrc in the specificed HOME path but the
resulting HTML did not have line number. What am I doing wrong?
2. Is there any way to include a command into the above command line to tell
vim to save and quite so that it doesn't open the actual vim session but just
saves the generated html?
Here is the vimrc I have in that directory. Am I missing any of the options you
suggested I add to get things moving faster? Do I have any wrong? Any others
you might suggest?
Also, the result of :version is:
VIM - Vi IMproved 7.2 (2008 Aug 9, compiled Jul 5 2010 16:33:43)
Included patches: 1-127, 257
Compiled by 'http://www.opensuse.org/'
Is this out of date? What is the easiest way to update?
BTW, this plugin is AWESOME!! Thanks a ton for doing it and being so helpful in
using it. It is really great stuff!
Benji
On Jun 15, 2011, at 12:25 PM, Benjamin Fritz wrote:
On Wed, Jun 15, 2011 at 11:12 AM, Benjamin Koltai bkolta...@gmail.com wrote:
I would like to see this metric but could not get the --startuptime option
to be recognized:
vim --startuptime file
I also looked in the help for startuptime and in the man page, and couldn't
find such a thing? What am I missing?
Possibly your Vim was not compiled with this option. It's part of a
normal build thogh, and it's missing from your runtime help files,
so more likely you're just using an old version of Vim in general.
What are the first few lines of :version output?
Ok. That is good to know that Vim is not multi-threaded. I am still a little
unclear on what the benefits of using a client server are?
Is it so that you don't have to start up a single vim instance every time
and can just leave vim running?
The main benefit is so you can use the same Vim for everything. It's
very useful if you're actually using Vim for editing and you want to
just work in one instance instead of finding the right one for every
file you're working on. It lets you launch new files in the same Vim
using the shell. It also allows for some interesting inter-process
communication, including using the shell to do background tasks in
Vim, even though Vim by itself is single-threaded.
Also, is there any limit on how many vim instances I can have running at one
time?
I don't think so, just the extra processor and memory it would incur.
Your method of closing each Vim instance when done also has the
benefit that you don't need to worry about resources Vim keeps around
for each buffer (even when that buffer is no longer shown). It will
take a little extra logic (probably just a :bwipeout command at the
end) to work with a single Vim instance.
It is starting to sound like the bottleneck might be the syntax detection
and actual coloring of the document. If this is the case than I can deal
with it because it is the main feature I am after. I guess I was just
wondering if there might be other plugins/initializers that might be running
that I could squash to save time. I have tried running vim with -V, but the
output is a little cryptic. I do see that it sources files such as
/usr/share/vim/... and I am not sure if those files are required for what I
am trying to do or not?
When converting from the command line, this works for me:
vim -N -u NONE -i NONE --cmd filetype on --cmd syntax on +runtime
plugin/TOhtml.vim +runtime syntax/2html.vim {file name}
This disables ALL plugins and customizations, then specifically
enables syntax highlight and TOhtml.
I didn't realize I needed to source the plugin file before running
2html.vim...that is probably something for me to fix. I think it's
just used for a meta tag, though.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Reply to message «Re: 2html optimization», sent 20:14:08 15 June 2011, Wednesday by Benjamin Fritz: Also, I have an item on my TODO list to allow the HTML output to use an external CSS file. My goal is that this external file can be generated just once and then used over and over, and that by not processing the CSS every time additional speed increases will be possible. I do not have an ETA on this feature, however; it has mostly stalled in the design stage until I find time to work on it again. If this is interesting enough I can probably put some other features on hold which I had planned on getting to first (automatic font detection, and uncopyable regions for things like line number and the fold column). If I am not mistaking, CSS is generated only once for each highlighting in both plugins, so I guess it will save almost no time. One thing you can do is to make a cache that persists across subsequent TOhtml runs like I do (though I end with file containing CSS items for all unique highlightings of all previous files, no matter whether they are used in current file or not. It is almost nothing compared to body, so I don't care), this will save more time for this case. By the way, I noticed that TOhtml creates links out of things like `http://www.rsyslog.com'. My plugin does not (and won't, it will only create links for tags). Can this be turned off so that comparison will be more fair? Original message: On Wed, Jun 15, 2011 at 10:37 AM, Ben Fritz fritzophre...@gmail.com wrote: You can also speed it up a little but by using: let g:html_no_progress = 1 Other options which might speed things up slightly (if their effects are acceptable): g:html_ignore_folding = 1 or g:html_dynamic_folds = 0 (otherwise Vim scans every line for fold information, twice, before starting the conversion) g:html_use_encoding = UTF-8 (or whatever encoding you desire) to avoid Vim needing to determine the encoding to use (probably VERY minor savings, but it can also avoid a conversion on write if the specified encoding matches your 'encoding' setting) g:html_expand_tabs = 0 (probably very minor savings, but this will prevent Vim from doing processing on tab characters to replace them with spaces; mostly this is useful if you do NOT use line numbers or a fold column in the generated output) One more to try, if you don't care about conceal support: g:html_ignore_conceal = 1 This will eliminate at least one check per syntax item, and I suspect (but have not verified) that the inner loop for non-concealed items (always used with ignore_conceal set) is faster than for concealed items. But again, this is probably minor savings. The biggest savings will probably be from the html_no_progress option, and from disabling syntax highlighting on the generated HTML output. I'm interested to learn what you end up with. I thought a few times about a use case like this, but decided it was unlikely because TOhtml is pretty slow for big files. Also note, Zyx has a plugin very similar to 2html, with a slightly expanded feature set, several possible output formats, and probably a very different implementation. If you do go with a Vim method, you can try it out. I don't think there's been very much comparison done between the two, it could be faster (or it could be slower, I really have no idea). http://www.vim.org/scripts/script.php?script_id=3113 Also, I have an item on my TODO list to allow the HTML output to use an external CSS file. My goal is that this external file can be generated just once and then used over and over, and that by not processing the CSS every time additional speed increases will be possible. I do not have an ETA on this feature, however; it has mostly stalled in the design stage until I find time to work on it again. If this is interesting enough I can probably put some other features on hold which I had planned on getting to first (automatic font detection, and uncopyable regions for things like line number and the fold column). signature.asc Description: This is a digitally signed message part.
Re: 2html optimization
On Jun 15, 1:35 pm, ZyX zyx@gmail.com wrote: I suggest you to use formatvim as output can be altered easier. How do you mean? Are you referring only to the different output formats (HTML/bbcode/etc.)? The reason for speed improvements is that I first generate a function that does formatting and then launch it, so it does not contain any if's which conditions are known to be false during generating html. That's a VERY interesting idea. Are you saying you look at all the options ahead of parsing, then generate a function on the fly which will be different each time according to which options were set, so that no option checking is needed in the acutal parsing? Or do I misunderstand? Sometime I should look at enhancing the performance of TOhtml, apparently. -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Reply to message «Re: 2html optimization»,
sent 22:59:48 15 June 2011, Wednesday
by Ben Fritz:
I suggest you to use formatvim as output can be altered easier.
How do you mean? Are you referring only to the different output
formats (HTML/bbcode/etc.)?
Everything is written in documentation. These formats are just dictionaries
(mostly {key : string}), you can write your own format easily by copying one of
them, replacing all script-local variables (there are not much,
bbcode-unixforum
contains none, for example) with global ones and, altering what you want to
alter and then using
execute frawor#Setup('0.0', {'plugin/format': '3.0'})
...
call s:_f.format.add('your_format_name', s:format_dictionary)
After then you can use
Format format your_format_name
. It is untested though: all existing defined formats are defined in s:formats
dictionary directly, not added by format.add feature.
When compiling some keys are added to the formatter function as plain strings,
some should be added as expressions (it is in TODO), others are function calls
(so there is more then one function compiled). (Really now all are compiled to
functions, but it may appear that resulting functions are not used.)
That's a VERY interesting idea. Are you saying you look at all the
options ahead of parsing, then generate a function on the fly which
will be different each time according to which options were set, so
that no option checking is needed in the acutal parsing? Or do I
misunderstand?
That's it. I have a large (1700 lines) function that mostly only compiles
another large function: historically I took (rewritten with some changes) large
parts of code from TOhtml (without splitting into functions) then partly put it
into `extend(formatfunction, [{code}])' constructs, so now it needs huge
refactoring and I have no idea where to start.
Note that in functions compiled from format dictionary option checking still
retains, but it is not much needed really.
Sometime I should look at enhancing the performance of TOhtml,
apparently.
I know only two approaches here that may significantly improve perfomance:
1. construct function as I do
2. use language with faster implementation (python, for example). Don't know
though what how large is the drawback for things like vim.eval(). Will be a
problem if vim-100500.0 will migrate to LLVM-jit implementation of vimscript.
It is good if you find the third. If you do I hope it will be combineable with
the first.
Original message:
On Jun 15, 1:35 pm, ZyX zyx@gmail.com wrote:
I suggest you to use formatvim as output can be altered easier.
How do you mean? Are you referring only to the different output
formats (HTML/bbcode/etc.)?
The reason for
speed improvements is that I first generate a function that does
formatting and then launch it, so it does not contain any if's which
conditions are known to be false during generating html.
That's a VERY interesting idea. Are you saying you look at all the
options ahead of parsing, then generate a function on the fly which
will be different each time according to which options were set, so
that no option checking is needed in the acutal parsing? Or do I
misunderstand?
Sometime I should look at enhancing the performance of TOhtml,
apparently.
signature.asc
Description: This is a digitally signed message part.
Re: 2html optimization
On Wed, Jun 15, 2011 at 1:47 PM, ZyX zyx@gmail.com wrote: By the way, I noticed that TOhtml creates links out of things like `http://www.rsyslog.com'. My plugin does not (and won't, it will only create links for tags). Can this be turned off so that comparison will be more fair? I had forgotten that (and keep forgetting it again and again; I'm always surprised when I notice it in the output). I should probably add an option to turn it off. But I doubt it takes THAT much processing time. -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Wed, Jun 15, 2011 at 1:22 PM, Benjamin Koltai bkolta...@gmail.com wrote: HOME=/path/to//directory/with/vimrc/ vim -N -u NONE -i NONE --cmd filetype on --cmd syntax on +runtime plugin/TOhtml.vim +runtime syntax/2html.vim file This opened a terminal vim window with file and the html genreated from that file. In the vim window that opened, my ~/.vimrc seemed to be applied, The -u NONE means don't read any configuration files, including the .vimrc so your .vimrc should not have been applied. Are you sure? Check the output of :scriptnames. You can add any additional commands with --cmd to take place before the file loads, and any more with -c which take place after. These can include sourcing scripts if needed (e.g. -c runtime colors/pablo.vim). but when I did :xa and saved both files, and opened the new file.html, it seems that the colorscheme set in the vimrc in the path set to home (not my ~/.vimrc) were applied, which is great!! I feel that I am very close to what I want to achieve, I just have a few more questions (because I am still very new to Vim and do not fully understand everything yet, so please excuse my noobing): 1. I have set number in the vimrc in the specificed HOME path but the resulting HTML did not have line number. What am I doing wrong? It's probably not being set, because probably your .vimrc should not actually be sourced. 2. Is there any way to include a command into the above command line to tell vim to save and quite so that it doesn't open the actual vim session but just saves the generated html? Sure. -c wqa or -c xa or similar, AFTER the running of the HTML conversion. -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Got it. The commands now make much more sense. Thanks for all the help! On Jun 15, 2011, at 4:35 PM, Benjamin Fritz wrote: On Wed, Jun 15, 2011 at 1:22 PM, Benjamin Koltai bkolta...@gmail.com wrote: HOME=/path/to//directory/with/vimrc/ vim -N -u NONE -i NONE --cmd filetype on --cmd syntax on +runtime plugin/TOhtml.vim +runtime syntax/2html.vim file This opened a terminal vim window with file and the html genreated from that file. In the vim window that opened, my ~/.vimrc seemed to be applied, The -u NONE means don't read any configuration files, including the .vimrc so your .vimrc should not have been applied. Are you sure? Check the output of :scriptnames. You can add any additional commands with --cmd to take place before the file loads, and any more with -c which take place after. These can include sourcing scripts if needed (e.g. -c runtime colors/pablo.vim). but when I did :xa and saved both files, and opened the new file.html, it seems that the colorscheme set in the vimrc in the path set to home (not my ~/.vimrc) were applied, which is great!! I feel that I am very close to what I want to achieve, I just have a few more questions (because I am still very new to Vim and do not fully understand everything yet, so please excuse my noobing): 1. I have set number in the vimrc in the specificed HOME path but the resulting HTML did not have line number. What am I doing wrong? It's probably not being set, because probably your .vimrc should not actually be sourced. 2. Is there any way to include a command into the above command line to tell vim to save and quite so that it doesn't open the actual vim session but just saves the generated html? Sure. -c wqa or -c xa or similar, AFTER the running of the HTML conversion. -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Reply to message «2html optimization»,
sent 00:23:50 16 June 2011, Thursday
by Benjamin Koltai:
I also saw something online about a vim client server? Would that be
something that would allow me to keep a vim instance running and send
files to be converted to that server? Would that speed things up?
It requires X support. If you don't have it, you have to use something like
screen instead:
screen -d -m -S vim-server-1 vim -u vimrc
screen -S vim-server-1 -X stuff $':view file\n'
screen -S vim-server-1 -X stuff $':Format format html to file.html\n'
screen -S vim-server-1 -X stuff $':bw!\n'
(this is example for my formatvim). It would speed up things a bit; more for
formatvim, less for tohtml. You could view what vim is doing by attaching
screen:
screen -r -S vim-server-1
I am using vim to syntax highlight and convert plain text files to html. My
server usually has multiple requests coming in at once, so I would love to
optimize vim to work as quickly as possible. I would love to get a better
understanding on what the minimum requirements for doing the conversion
are and how I can set them properly. I am currently using the command:
You need colorscheme, the whole syntax directory (and ftdetect if present) and
files
autoload/tohtml.vim
plugin/tohtml.vim
filetype.vim (for filetype detection)
In vimrc there must be
set nocompatible viminfo= noswapfile undolevels=0 nobackup
filetype on
syntax on
See how I did tohtml testing.
Original message:
Hi vim users,
I am using vim to syntax highlight and convert plain text files to html. My
server usually has multiple requests coming in at once, so I would love to
optimize vim to work as quickly as possible. I would love to get a better
understanding on what the minimum requirements for doing the conversion
are and how I can set them properly. I am currently using the command:
HOME=#{ CONFIG.paths.lib_dir }/vimfiles/ vi -n -X '+runtime!
syntax/2html.vim' '+xa' filename
I have a colorscheme in .../vimfiles/.vim/colors as well as a .vimrc. I
only need syntax highlighting and 2html. The above command works but takes
way to long. Are there other options I should add to speed up the process?
I tried -e but I do not get the syntax highlighting with that option.
Does the file have to load in order for it to get highlighted?
I also saw something online about a vim client server? Would that be
something that would allow me to keep a vim instance running and send
files to be converted to that server? Would that speed things up?
Thanks much for any help or advice,
Benji
signature.asc
Description: This is a digitally signed message part.
Re: 2html optimization
On Wed, Jun 15, 2011 at 3:35 PM, Benjamin Fritz fritzophre...@gmail.com wrote:
The -u NONE means don't read any configuration files, including the
.vimrc so your .vimrc should not have been applied. Are you sure?
Check the output of :scriptnames.
You can add any additional commands with --cmd to take place before
the file loads, and any more with -c which take place after. These can
include sourcing scripts if needed (e.g. -c runtime
colors/pablo.vim).
Thinking about this some more, a better way to start would probably be:
vim -N -u config.vim -i NONE --noplugin {file}
This way you can put all the sourcing of scripts, turning on filetype
detection, config options, etc. into the config.vim file, and not
worry about any other plugins which might be loading.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
Ok that does sound like a cleaner command.
I have:
HOME=/path/to/vimfiles/ vim -N -u /path/to/config.vim -i NONE --noplugin {file}
And this is my config.vim has:
filetype on
runtime colors/simplewhite.vim
colorscheme simplewhite
set number
syntax on
runtime plugin/TOhtml.vim
runtime syntax/2html.vim
It seems to be working without including the files suggested by ZyX, but might
I lose some syntax highlighting for certain file types?
I am also getting prompted:
file.html file.html 11L, 407C
Press ENTER or type command to continue
Is there some way to suppress this because I need the command to run without
additional input.
Thanks
On Jun 15, 2011, at 5:26 PM, Benjamin Fritz wrote:
On Wed, Jun 15, 2011 at 3:35 PM, Benjamin Fritz fritzophre...@gmail.com
wrote:
The -u NONE means don't read any configuration files, including the
.vimrc so your .vimrc should not have been applied. Are you sure?
Check the output of :scriptnames.
You can add any additional commands with --cmd to take place before
the file loads, and any more with -c which take place after. These can
include sourcing scripts if needed (e.g. -c runtime
colors/pablo.vim).
Thinking about this some more, a better way to start would probably be:
vim -N -u config.vim -i NONE --noplugin {file}
This way you can put all the sourcing of scripts, turning on filetype
detection, config options, etc. into the config.vim file, and not
worry about any other plugins which might be loading.
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On Wed, Jun 15, 2011 at 5:06 PM, Benjamin Koltai bkolta...@gmail.com wrote: And this is my config.vim has: filetype on runtime colors/simplewhite.vim colorscheme simplewhite set number syntax on runtime plugin/TOhtml.vim Good. I'm not 100% sure on the order, but it looks like you have a nice minimal config. runtime syntax/2html.vim This one probably ought to go on the command line to make sure it happens after all initializations are complete and your file(s) loaded. It seems to be working without including the files suggested by ZyX, but might I lose some syntax highlighting for certain file types? In addition to what you already have, ZyX suggests: * the whole syntax directory (and ftdetect if present) and files -- this does not need to be sourced directly, but the filetype on and syntax on which you have relies on their presence * autoload/tohtml.vim -- this does not need to be sourced directly, it is automatically loaded when needed by 2html.vim * filetype.vim (for filetype detection) -- again, this does not need to be sourced directly, it is used by the filetype on command. He also suggests: set nocompatible -- not needed due to your -N flag you pass into Vim, but you could dispense with the -N if you use this. viminfo= noswapfile undolevels=0 nobackup -- these options simply serve to speed Vim up in general, due to not storing any undo information, not making a backup when saving, and doing everything in memory instead of writing to a swap file from time to time. They are not needed for your purposes but may help performance. Remember my suggestion about disabling syntax for the output HTML file? Here's how you could do that: autocmd BufNewFile * setl ft= This assumes that you are creating HTML files which do not already exist, you will need to use a different event/matching pattern if the file could exist already. -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On 2011-06-15 08:36, Ben Fritz wrote: On my system, I have a ~/.vim/syntax/2html.vim, and also the default $VIMRUNTIME/syntax/2html.vim. When I use +runtime! syntax/2html.vim I see what you describe, as I would expect, because the '!' in the runtime command says source all files found. When I omit the '!' it works as intended, and only does a single conversion. Ah, that makes sense, I was unaware that runtime! would source all the files it finds. I have not tested 2html with Vim in ex mode. I would not be surprised if there are some problems with this. If it works, that's pretty cool! Do let me know if you see problems running 2html in ex mode, I imagine that might speed things up some. It works for me, I think i had to specify the TERM to get syntax highlighting to work. Without it and screen I sometimes get very simple highlighting but not the 256 color highlighting I want. For reference, the full command I use and the vimrc are posted here: -http://vpaste.net/index.cgi?ft=sh -http://vpaste.net/vimrc?ft=vim I notice that these references use the dynamic_folds option, but there are no folds in the document. See my note above, you could probably make this faster by setting the ignore_folding option! Folds aren't used by default, but you can add them. Same thing for some of the other settings, so I have to leave a lot of the 2html features turned on even if they're not used most of the time. For example: - http://vpaste.net/index.cgi?ft=sh,fdm=indent,number,expandtab I guess I could check the value of 'foldmethod' and if it's set to 'manual' I could set 'ignore_folding'. Some of these options I suggest above were added/augmented in later versions of the TOhtml plugin than you are using. I think that v9 is currently in the Hg repository, and I've given v10 to Bram, but you're still using v6 according to the meta tags in the generated output. Ok, I'll have to try updating sometime. Also note the html_use_encoding option takes an IANA name (i.e. UTF-8 rather than utf8). Perhaps I should add an auto-correction for that...I'll think about it. You could maybe add it to the help document. I missed the part about the equivalent html charset name. Using a real example such as UTF-8 instead of foobar might make it a little more obvious. pgpd8JHtYx4oC.pgp Description: PGP signature
Re: 2html optimization
Reply to message «Re: 2html optimization», sent 02:35:38 16 June 2011, Thursday by Benjamin Fritz: Just look at my configuration for tohtml and formatvim testing. It is minimal. You may need to add ftdetect directory and do something to collect syntax/*.vim from different runtime directories, but that's all. Original message: On Wed, Jun 15, 2011 at 5:06 PM, Benjamin Koltai bkolta...@gmail.com wrote: And this is my config.vim has: filetype on runtime colors/simplewhite.vim colorscheme simplewhite set number syntax on runtime plugin/TOhtml.vim Good. I'm not 100% sure on the order, but it looks like you have a nice minimal config. runtime syntax/2html.vim This one probably ought to go on the command line to make sure it happens after all initializations are complete and your file(s) loaded. It seems to be working without including the files suggested by ZyX, but might I lose some syntax highlighting for certain file types? In addition to what you already have, ZyX suggests: * the whole syntax directory (and ftdetect if present) and files -- this does not need to be sourced directly, but the filetype on and syntax on which you have relies on their presence * autoload/tohtml.vim -- this does not need to be sourced directly, it is automatically loaded when needed by 2html.vim * filetype.vim (for filetype detection) -- again, this does not need to be sourced directly, it is used by the filetype on command. He also suggests: set nocompatible -- not needed due to your -N flag you pass into Vim, but you could dispense with the -N if you use this. viminfo= noswapfile undolevels=0 nobackup -- these options simply serve to speed Vim up in general, due to not storing any undo information, not making a backup when saving, and doing everything in memory instead of writing to a swap file from time to time. They are not needed for your purposes but may help performance. Remember my suggestion about disabling syntax for the output HTML file? Here's how you could do that: autocmd BufNewFile * setl ft= This assumes that you are creating HTML files which do not already exist, you will need to use a different event/matching pattern if the file could exist already. signature.asc Description: This is a digitally signed message part.
2html optimization
Hi vim users,
I am using vim to syntax highlight and convert plain text files to html. My
server usually has multiple requests coming in at once, so I would love to
optimize vim to work as quickly as possible. I would love to get a better
understanding on what the minimum requirements for doing the conversion are and
how I can set them properly. I am currently using the command:
HOME=#{ CONFIG.paths.lib_dir }/vimfiles/ vi -n -X '+runtime! syntax/2html.vim'
'+xa' filename
I have a colorscheme in .../vimfiles/.vim/colors as well as a .vimrc. I only
need syntax highlighting and 2html. The above command works but takes way to
long. Are there other options I should add to speed up the process? I tried -e
but I do not get the syntax highlighting with that option.
Does the file have to load in order for it to get highlighted?
I also saw something online about a vim client server? Would that be something
that would allow me to keep a vim instance running and send files to be
converted to that server? Would that speed things up?
Thanks much for any help or advice,
Benji
--
You received this message from the vim_use maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: 2html optimization
On 2011-06-14 16:23, Benjamin Koltai wrote:
I am currently using the command:
HOME=#{ CONFIG.paths.lib_dir }/vimfiles/ vi -n -X '+runtime!
syntax/2html.vim' '+xa' filename
Using '+runtime! syntax/2html.vim' is acting funny on my computer.
I haven't looked into it, but it's generating html for filename, and
then generating another html for filename.html and calling that one
filename.html.html.. Using '+runtime' instead of '+runtime!' or just
using +TOhtml seems to fix it..
I tried -e but I do not get the syntax highlighting with that option.
You can force ex to use syntax highlighting with a few tricks. You'll probably
need to run it in screen if it's being run by a daemon, and you'll need to
force the value of 'term' in vimrc. The gist of the command I use is:
HOME=/home/andy screen -D -m ex -nXZ -i NONE -u vimrc '+sil! TOhtml' ...
You can also speed it up a little but by using:
let g:html_no_progress = 1
For reference, the full command I use and the vimrc are posted here:
- http://vpaste.net/index.cgi?ft=sh
- http://vpaste.net/vimrc?ft=vim
If you find anything else that helps, let me know!
pgpXyEyxL2w1S.pgp
Description: PGP signature
Re: 2html optimization
On Tue, Jun 14, 2011 at 1:23 PM, Benjamin Koltai bkolta...@gmail.com wrote: I am using vim to syntax highlight and convert plain text files to html. My server usually has multiple requests coming in at once, so I would love to optimize vim to work as quickly as possible. I would love to get a better understanding on what the minimum requirements for doing the conversion are and how I can set them properly. This may be heretical for this list, but is Vim the right tool for the job? There are many other syntax highlighting engines out there. I've used the Pygments http://pygments.org/ library with some success and restview http://mg.pov.lt/restview/ to render it to a browser. -- /George V. Reilly geo...@reilly.org Twitter: @georgevreilly http://www.georgevreilly.com/blog http://blogs.cozi.com/tech -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
