Re: can't diffsplit
Eric Leenman wrote: I'm having the same problem, and the three solutions don't work for me. In all the three solutions (and even the other thread I started with your reply windo diffthis) doesn't work. What happens is that one window folds the complete file to one line. The other window keeps the file open as it was. Let's start from scratch. Say you have two files, old.txt and new.txt, where they are similar but different. You run: gvim -d old.txt new.txt You should see the two files side-by-side. Groups of lines that are identical are folded. Differences are highlighted. Each vertical split has a fold column (grey by default) down the left side. Does the above work? When you follow Tony's advice, do you see the same fold column in each split? If not, you need to switch to the split missing the fold column and type ':diffthis'. I was expecting somekind of Winmerge behaviour. Winmerge can do some great stuff, but Vim differencing of two file is better because Vim folds lines that are the same. John
Re: VimWiki - Page Titles
Sebastian Menge wrote: Put the list of 1500 tip titles in one file, one title per line. Then edit that file to clean up the titles. Then run a script to rename each tip to match the cleaned-up title. One idea was that the editing can be done on the wiki. Just edit the Errornames page :-) Neat, but please give explicit directions if that's what you want. There's not much point in my editing the titles if you meanwhile are planning to use some other scheme. Also, we (actually, you, because it looks like you're doing all the work:), need to resolve the issue of exactly what is allowed in a title, and we should agree on some general guidelines. I think the Wikipedia style of prominently saying something like this page should be titled xxx but due to technical restrictions we can't do that is too ponderous (although reasonable in their context). Maybe we could have something more informal (if scriptable). For example: tip 249 in your errornames might be: Title = C - Quickly insert precompiler directives [I'm not very happy with this wording] But first line of the tip might say: C/C++: Quickly insert #if 0 - #endif around block of code stable regexes for 1500 pages are not easy to do I'm glad it's you and not me! It's hardly reasonable to come up with one script that correctly formats all of the existing pages. I imagine a fair bit of manual tweaking will be needed. If you gave me a couple of days over a weekend when it's quiet here, I might be able to do a fair bit (I sent over 260 typos to Bram which he incorporated in the 7.1 release, so I can occassionally cope with tediousness). OTOH we can all do that after the initial import. I can download all 1500 tips from wikia, and determine if any still have html (what will wikia do to html tags??), and fix them then. John
Re: VimWiki - Page Titles
Sebastian Menge wrote: Find the list (95 entries) here: http://scratchpad.wikia.com/wiki/Errornames Thanks for the good start. FYI there are a couple of lines with broken links: 157 160: 171: Do you know the g/ and g? commands? Above gives: Vim Online Error Couldn't find tip 160. Are you sure it exists? Im not sure howto proceed here. Should we a) find better titles before the import Yes! In option (b), you have to change every '/' to '__OR__', so you may as well change the titles to something good now. Can you readily do something like this: Put each tip in a separate file on your disk. Name them tip0001, tip0002, etc. Put the list of 1500 tip titles in one file, one title per line. Then edit that file to clean up the titles. Then run a script to rename each tip to match the cleaned-up title. or b) replace '/' by sth like '__OR__' and fix the whole title later? Whatever works, but wouldn't this create a whole bunch of problems? I don't understand the internals of wikis but I think your suggestion would create 95 tips with URLs that will later need to be manually edited. Not so easy, and probably involves copying the content from the wiki to a new page, then deleting the old page (I guess). BTW: For the import I will now use WikipediaFS. Wow - amazing. How do you get the wiki format files from the VimTips web site? If you're going to do the work, I don't need this answered, but I'm thinking that you're going to need one of the scripts to convert the existing html to wiki format. I noticed that Charles Campbell's script does appropriate things with common html codes like nonbreaking space. Probably all that processing should be done before the files are uploaded? With your scheme, you're going to get 1500 tip files on your disk. It would be great if you could clean them as much as possible before uploading. It would be pretty easy to find all html markups and '' codes when the files are still on your disk. Easy, but time consuming. Let us know if you want some help. John
Re: A performance question (utility included)
Charles E Campbell Jr wrote:
I've attached a patch to vim 7.1 which extends getfsize()
As I've mentioned, I think further testing will be needed before
patching Vim for 64-bit file lengths.
Here is a possible interim workaround to allow Dr.Chip's
LargeFile.vim script to accurately detect large files on many
platforms.
Attached is a tiny C program to build a tool called filemeg.
Example usage: filemeg /path/to/file
gives output: 42
which means that the specified file is 42 megabytes (actually,
any value from 42 to nearly 43).
I was going to work out how to adapt the LargeFile script to use
this tool, if the user sets an option to invoke it. But it's
taking too long because I don't know enough about Vim, so I'm
just presenting the tool at this stage.
People may like to check how filemeg works on various systems
and report back. I have tried it on files over 4GB on Fedora
Core 6 and Windows XP (x86-32 platform).
Putting something like this inside Vim would be a bit of a
nightmare IMHO because of the extraordinary range of supported
compilers, operating systems and hardware.
Adapting LargeFile.vim to work with filemeg:
- Compile the source and test running at command line.
- Put the executable in your path (better: in a Vim
directory which the script could invoke somehow).
- Set a new script option to use filemeg.
- The script BufReadPre would call a new script function.
- That function would check the file size with:
let result = system('filemeg /path/to/file')
- If result is a number, it is the file size in megabytes.
- Otherwise, result is Error... and the script should
treat the file as large (or maybe not...).
I've attached the C source, and included it below for those who
don't mind a little wrapping.
John
/* Output length of specified file in megabytes.
* John Beckett 2007/05/25
* For Linux with LFS (large file support), and Win32.
*
* Output is suitable for reading by a script.
* Output is always one line.
* If any problem occurs, line starts with Error.
* Otherwise, line is the size of the specified file in megabytes.
* Size is a truncated integer (file of 3.9MB would give result 3).
* The size won't overflow a 32-bit signed integer (Error if it does).
* If argument is a directory, result is 0 (done by stat64()).
*/
#if defined(__linux)
# define _LARGEFILE64_SOURCE
#elif defined(_WIN32)
# define off64_t__int64
# define stat64 _stati64
#endif
#include stdio.h
#include sys/types.h
#include sys/stat.h
int main(int argc, char *argv[])
{
off64_t size, overflowmask;
struct stat64 sb;
if ( argc != 2 ) {
puts(Error: Need path of file to report its size in megabytes.);
return 1;
}
if ( stat64(argv[1], sb) != 0 ) {
puts(Error: Could not get file information.);
return 1;
}
size = sb.st_size 20; /* 2^20 = 1 meg */
overflowmask = 0x7fff;/* ensure 64-bit calculation */
if ( (size ~overflowmask) != 0 ) {
puts(Error: File size in megabytes overflows 32-bit signed
integer.);
return 1;
}
printf(%d\n, (int)size);
return 0;
}
filemeg.c
Description: Binary data
Re: A performance question (patch included)
Charles E Campbell Jr wrote: I'm also under the impression that ls itself uses fstat(), so its not likely to be any more informative. That's likely on some systems, but 'ls -l' gives correct results for files over 4GB on Fedora Core 6 using x86-32. John
Re: A performance question
Yakov Lerner wrote: 9-digit number can still be larger than 2^32-1, or than 2^31-1. Just for the record: 2^30 = 1,073,741,824 So 999,999,999 (largest 9-digit number) won't overflow a 32-bit signed integer. John
Re: A performance question
Charles E Campbell Jr wrote:
Sounds like the filesize is getting stored in a 32bit signed
number, and overflowing.
Yes, definitely.
Please let me know what getfsize() is actually returning
The return value is the bit pattern for the low 32 bits of the
true 64-bit file size:
3,146,839,490 file actual size
-1,148,127,806 returned by getfsize()
The sum of the absolute values is exactly 4G. I confirmed the
above with a file exactly 8G + 512 bytes (getfsize() said it was
512 bytes).
I was going to suggest that you treat a negative getfsize()
value as a large file (may as well do that even if the value is
-1 for an error indication).
I suppose that would be useful until some better solution is
implemented. I didn't propose it because it would be quite ugly
for the script to give results like this:
3GB file is large
5GB file is not large
7GB file is large
9GB file is not large
Another ugly (but accurate) workaround would be:
- Provide the source for a small executable called islargefile
to do the calculation.
- Provide an option in the script to use the executable.
- Have the script execute system('islargefile xxx 123456').
Return value 0 means no, 1 means yes, 2 means error
(return is first character of system() string).
Need to work out how to pass arguments:
xxx = path of file to be tested
123456 = limit at which file is large
John
Re: A performance question
Yongwei Wu wrote: Even FAT32 supports files much larger than 4GB. Not true. FAT32 supports files up to 4 GB. Sorry I shot my mouth off there - I realised my blunder about ten minutes after sending. I haven't actually used a FAT32 partition for over ten years, and was confusing the maximum size of a FAT32 partition with its maximum file size. On NTFS of course, as you mentioned, the sky is the limit. I have made files larger than 4GB, and have written a couple of simple programs to work with such files, so my basic point is valid. The Win32 API supports files much larger than 4GB. John
Re: A performance question
panshizhu wrote:
Yes, but on all systems, vim script could not take 64-bit
integers
I know that. My proposal is for a new Vim script function:
islargefile({fname}, {limit})
which would return nonzero if the size of the file is greater
than the 32-bit signed {limit} argument.
Vim could easily handle the 64-bit arithmetic that is available
on several systems, so the proposed islargefile() would
accurately indicate whether the file size was greater than the
specified limit. The limit would be up to 2G - I don't think
there's any need to get cute and allow the caller to pass a
negative value which would then be treated as unsigned.
John
Re: A performance question
A.J.Mechelynck wrote: What about a different function to return, say, the number of 1K blocks (or the number of times 2^n bytes, with a parameter passed to the function) that a file uses? Yes, that's a much more general and better idea. Since there's probably not much need for this, I think that simplicity would be good. That is, have the function work in a fixed way with no options. Re Dr.Chip's LargeFile script: It occurs to me that another workaround would be to use system() to capture the output of 'ls -l file' or 'dir file' (need an option for which). Then do some funky editing to calculate the number of digits in the file length. If more than 9, treat file as large. I'm playing with a tiny utility to help the LargeFile script. Bluesky: Its code (64-bit file size) could potentially be incorporated in Vim. I'll post results in vim-dev. John
Re: A performance question
Peter Palm wrote: http://www.vim.org/scripts/script.php?script_id=1506. Indeed, among other things, this disables the swap file for 'large' files, which should really speed up things. I was going to report the following issue to vim-dev after I got a chance to investigate it a little further, but it seems appropriate to mention it now. I did some work with a 3GB text file on Win32, using Vim 7.0 and Dr.Chip's LargeFile.vim script from Tip 1506 above. The result was really ugly. The script failed to notice that 3GB was large because the Vim function getfsize(f) returned a negative number. Vim eventually opened the file and was able to search, etc. So Vim doesn't rely on the code behind getfsize(). I started looking at what could be done to fix the bug, but have had to leave it for another day. I was starting to think that it wouldn't be too hard to use the extended functions available in recent Win32 and Linux to get a 64-bit file size. Then maybe provide a function to compare a 64-bit file size with a specified 32-bit limit, so LargeFile.vim could work reliably. I haven't checked getfsize() on 32-bit Linux yet, nor am I sufficiently patient to try opening the 3GB file with Vim 7.1. John
Re: A performance question
panshizhu wrote:
As far as I know, Windows does not support files larger than
4GB. So its okay to use unsigned 32-bit for filesize in
windows.
It's not as bad as that! Even FAT32 supports files much larger
than 4GB.
The Win32 API includes function _stati64() to get a 64-bit file
size (the API really wants you to use GetFileSize(); _stati64()
is for old timers).
I was envisaging some new Vim script function like:
islargefile({fname}, {limit})
which would return nonzero if the size of the file is greater
than the 32-bit signed {limit} argument.
On many systems, the calculation could use 64-bit integers.
John
Re: Vim Wiki - Tip Page Formatting Deadline
Sebastian Menge wrote: [...very helpful summary for Vim Wiki...] Thanks for keeping this moving. I'm happy with what you said, but here are some comments. Please disregard all this and just do it, if you can't stand my detail at the moment! everything should fit on one page, no scrolling That would be an aim, but clearly it can't be enforced by the initial import (although it does mean that any information box should be at the end of the tip). 2.) Drop the id for new tips, use old ids just to redirect. Good, but let's have a specific example. How about: ---Current example--- URL:http://scratchpad.wikia.com/wiki/VimTip1_v2 Title: Tip: #1 - the super star ---What we want--- URL:http://vim.wikia.com/wiki/the_super_star Title: The super star The import script should change the tip title as above. -Or- Do we want to keep the (obsolete!) tip number? ---Also have a redirect page--- URL:http://scratchpad.wikia.com/wiki/vimtip1 Page content (I'm not sure if syntax is correct): #REDIRECT [[the_super_star]] ---Example of new tip--- URL:http://vim.wikia.com/wiki/my_great_new_tip Title: My great new tip (there would be no tip number and no redirect page) 3.) Use some scripting to keep ids, e.g. via a cron job that tidies up each night or a javascript that uses a timestamp. I think you are saying that this is not a good plan. I agree that we cannot realistically have an automated process run on someone's computer to edit wiki pages. We'll have to plan so there is no need to automate adjustments. Tony mentioned the possibility of a wiki robot, but I guess that we would not be able to use such a feature, unless we knew a friendly administrator (hi Martin!). Layout issues of the template can be discussed in the respective discussion page. Let's not do that yet. I suggest we keep all discussion here until the wiki is imported. After the first import, let's discuss it here and agree yes. Once that is done, those who are willing can discuss details on the wiki discussion pages. Conversion of HTML in Tip-Text. This should be converted to wiki markup in the long run. Please - let's do it in the import script! Anything else would be a heap of unrewarding and unending work. The script could convert simple cases, and maybe flag awkward cases for manual attention. We have had three fantastic scripts (I think it's three) that were produced very quickly. Someone needs to pick one and ask the author if they are willing to make adjustments. A problem are tips that contain URLs to external sites (e.g. other tips) Fix this in the import script as well: - Links to tips should be adjusted to wiki format. - Links to external sites should be converted to plain text for later review. I like Martin Krischik's suggestion that we use the vi wikibooks where he is an administrator. However, we would get really ugly URLs - I think something like (one line!): http://en.wikibooks.org/wiki/Learning_the_vi_editor/Vim/Tips_and_Tricks/the_super_star Maybe Martin has a suggestion to improve that. I also saw a comment that wikia would provide additional benefits over wikibooks (I don't quite understand this): a complete wiki, not just a bunch of pages To do: - Discuss here for, say, three more days. - Import ten tips. - Discuss for another three days. - Import all tips. John
Re: Vim Wiki - Tip Page Formatting Deadline
Martin Krischik wrote: But do we need an id? - each tip will have a page name which will be unique. IMHO you are correct - we should NOT have a tip id. I'm not sure what is feasible, but here is my ideal: - Import current tip_nr into the template. - Display tip_nr near the top of the tip, very modestly. - When we're ready to abandon ids, stop displaying tip_nr. Content first is makes Wikis successfull. The Vim Wiki project is talking to much about ID numbers, templates, organization etc. pp. without producing any content. You're talking a lot of sense, Martin. However, someone is going to import 1500 tips, so I can put up with a bit more planning. But if there's no action before June, I think it would be reasonable to despairg. John
Re: Vim Wiki - Wiki Template Proposal
Martin Krischik wrote: Well, on most Wikibooks comments are seldom cleaned up. They are just left to rot. And since there are not on the main page it does not matter. I see what you mean. If that were to happen with the Vim wiki then it would be really ugly (i.e. if junk comments were left indefinitely on the main tip page). Let's agree that in November (say), we could have a clean up. Someone could post a message to vim@vim.org with a list of tips that contain extraneous comments. Give everyone 14 days notice that the listed tips will be deleted if no one cares enough about them to clean up the comments. This suggestion is not for difficult cases where the tip is worthwhile, but for various reasons it's hard to integrate the comments. However, even in those cases, if we care enough we should edit the comments for spelling and accuracy, and should delete unhelpful comments. John
Vim Wiki - Tip id and URL
It is important for us to have an easy way to refer to tips,
and the URL of each tip should be simple (and not wrap in an
email).
Below is some detail for a proposal to achieve this.
What does everyone think?
Have a script scrape tip titles from the current Vim Tips.
Create a plain text file with fields (say tab-separated):
tip_nrtip_idtip_name
tip_id would initially be empty.
Example:
Tip #1504: External commands on Windows
http://www.vim.org/tips/tip.php?tip_id=1504
would give line (':' represents '\t'):
1504::External commands on Windows
I would edit this file (or maybe split it over three
volunteers). We would manually insert a short tip_id by
abbreviating tip_name. For example:
1504:win_external_cmd:External commands on Windows
We would manually confirm (with a script) that the 1500
tip_id fields are unique.
The script to import each tip would read the above file:
- Use tip_nr to read the current tip.
- Use tip_id to make the URL of the new tip.
- Ignore tip_name.
The example URL might be:
http://vim.wikia.com/wiki/tips/win_external_cmd
Perhaps a contents page could list all tips:
win_external_cmdExternal commands on Windows
Where we might once have said see tip 1504, we would instead
say see tip win_external_cmd.
John
Re: vim 7.1 and cr/lf interpretation
Thomas Michael Engelke wrote: In the programming language I currently write, using the seperator string somewhere in the string I'm parsing gives a new entry. So a string containing only a line seperator char/pattern would have 2 entries, which in this case would mean 2 lines. As I see now, this is handled differently in vim. Sorry to beat this really dead horse, but it's not Vim which handles this differently! End-of-line has always been a terminator, not a separator (as Tony explained earlier). I've had a closer look at the file you attached: - All lines end with CR LF, except for three lines. - Two lines end with CR only. - One line (the last) ends with LF only. John
Re: Vim Wiki - Tip Page Formatting Deadline
Sebastian Menge wrote: Therefore, I would vote for using tip-title == page-title and let the categorization be done by [[Category:VimTip]]. Using Wikipedia standards (CamelCaseIsUgly) we would get pages like The Super Star and we could reuse the title in the template. We would lose the tip-id for new tips. Thinking about how a wiki works shows that keeping tip numbers is doomed. First, there is no auto-increment id, and as you point out, there is no reasonable way to automate fixes. But more than that. I hope that the wiki tips really will be edited and improved. Currently, there are lots of tips that are of very marginal value, and there are several related tips. I hope that unhelpful tips will be brutally deleted, and related tips will be merged (if that doesn't make the new tip too big). In other words, we should end up with many missing tip numbers. If all tips are retained for nostalgic reasons, the usefulness of the whole site will be reduced due to the growing amount of cruft that would obscure the helpful tips. Re the tip URL, what would you recommend for this: List of text editors http://en.wikipedia.org/wiki/List_of_text_editors Will it be possible to edit the URL of a link, in order to simplify it? For example, I don't like: Optimization (mathematics) http://en.wikipedia.org/wiki/Optimization_%28mathematics%29 I've seen much worse URLs, but can't find one now. John
Re: Vim Wiki - Wiki Template Proposal
Martin Krischik wrote: We should not include comments on the content page! That's what the discussion page is for. You are very keen on that point, so I'm going into a bit of detail about why I don't agree. A wiki discussion page (as you know!) is intended for people to discuss the future of the page. Does an error need fixing? Are there points which need to be expanded? Is the content or style inconsistent with overall guidelines? Or, on the discussion page, I might ask why you reverted my edits, and we could debate whether my wording was better than yours. We'll still need the above in a Vim wiki. However, the Comments in Vim Tips are a different animal. Most comments are fluff, and need to be deleted ASAP. Many comments are very helpful, and their content needs to be merged into the body of the tip. On some tips, a reader would need a lot of persistence to work out what to do, because the tip says X, some comment says Y, and another comment says Z. I think I recall seeing cases where a comment points out that the tip is hopeless because there's a better way of handling the situation. We wouldn't want that comment hidden on the discussion page (where a casual reader won't see it). As I understand it, the whole point of moving Vim Tips to a wiki is so that we can fix each tip so that there is one consistent story on each page. You are correct that having the comments on the main page will be ugly. However, we hope that will be temporary. Perhaps I should say that *I* hope it will be temporary because I see that the proposed sample has a section for Comments. I imagine editing the wiki will go like this: - Import all tips with comments on main page. - Edit important tips and clean them up completely. - Edit nearly all tips to remove junk comments. - Leave difficult cases for later. I imagine there will be lots of difficult cases where considerable effort would be needed to merge the comments. In those cases, we would just leave the useful comments, perhaps editing them where helpful. Later (say in six months) we would discuss what to do with those tips that still have unmerged comments. In some cases, it might be very reasonable to leave comments on the main page. For example, a tip might describe a scenario and its solution. Then a comment might say that if you are running on a certain platform, then a better approach would be something else. It may never be worth fixing all tips to eliminate such comments, yet you wouldn't want to hide that useful info on the discussion page. I think that following the above strategy would be much easier for people editing a tip (easier than editing the main page and the discussion page, because once a comment is dealt with, it would have to be removed from the discussion page). Also, seeing the old comments on the main page would be an immediate reminder that the tip needs cleaning up. Imagine the mess if comments were on the discussion page, then someone edited the main page to include a few useful comments from the discussion page, but failed to remove those comments. It would then take herculean efforts to properly fix the tip, and the discussion pages would have so much junk in them that their function as a tip discussion would fail. John
Re: vim 7.1 and cr/lf interpretation
Thomas Michael Engelke wrote: Assuming that fileformat unix means 0x10 means open a new line which seems to be the case (0x13 stays as ^M, 0x10 is interpreted as a new line), shouldn't this mean that in the last line there is a line break missing? It's not important here, but for the record, 0x10 means hex 10, which is decimal 16. That's not what you have. The correct story is: ^M = Ctrl-M = 13 = 0x0d = CR = carriage return ^J = Ctrl-J = 10 = 0x0a = LF = line feed A Unix format file has LF at the end of each line. A DOS format file has CR LF at the end of each line. Some stupid systems put nothing at the end of the last line. That is what we see in your sample file (your last line has no terminator). John
Re: Stable Vim version 7.1 has been released
Bram Moolenaar wrote: Announcing: Vim (Vi IMproved) version 7.1 Excellent - thanks. Is someone going to post to a security list like full disclosure that Vim 7.1 is available, and it overcomes a serious security problem? The issue has received a brief mention in relation to patches for Vim 7.0, but now that a full release is available, perhaps an announcement should be made. John
Re: VimWiki - again - but with a brand new option
Tom Purl wrote: Here's what I propose we do: 1. Finalize a tip formatting standard. 2. Use the best available script that supports this standard. 3. Update the best available script if necessary. 4. Revise the standard if necessary. 5. Convert a tips sample. 6. Review the sample and revise the script if necessary. Good. But to keep our discussion focussed, please do what you did last time: Put a sample tip on a wiki page so we can agree on its features. Please take Gene's advice and manually edit the page to how you think it should look. Once the format is agreed, we can ask for a script. I recommend: - Propose a format for the URL of each tip, as well as the format of the page. - Omit the info box with author, date, tip rating, Vim version. It's too hard to maintain, and too intrusive. - Keep the comments on the tip page, with a very simple comment heading in front of each, something like: -By UserName on March 8, 2001 14:51- To make it easy to edit the page, the comment heading should be a single line in the wiki source. -Or- Put all the comments on the talk page, with the format as above. However, that seems unnecessarily tricky to do in practice (it doubles the number of pages we have to work with). I favour putting the comments in the main page to make it easier to clean up the tip. When we see a tip with old-style comments, we would know that it needed an overhaul. 1. Let's use this mailing list to coordinate the project. All comments regarding wiki page format, however, should be written to the talk section of the affected wiki page. Please be more explicit. Will we use the vim or the vim-dev list? How can we comment on the wiki page format on the talk section? I think we should use the vim mailing list for all discussions until a decision (your decision!) is made to finalise the wiki site, format, and script. Final suggestion: Please start a new thread (new subject) which we will follow until everything is finalised, rather than replying to this. It would be great if you would consider what I and others have written, then make a proposal with what you think. John
Re: feedkeys() allowed in sandbox
Ciaran McCreesh wrote: 100 bytes is more than enough room to download and execute a file that contains the real malicious code. I actually agree that it is extremely unlikely that a length check would make modelines more secure, but I'm being argumentative because it's irritating to be authoritatively assured that a length check would have no benefit in the future. We just don't know whether some future vulnerability (perhaps using a currently-unknown new feature) might be avoided with a modeline length check. John
Re: search pattern on sed or grep
Tim Chase wrote: C:\Temp find /? No patterns other than fixed text Since Windows 2000, there is also the findstr command. It is not as clever as grep, but it supports patterns and is able to efficiently handle big files. In Vim: :helpgrep findstr At Windows command prompt: findstr /? John
Re: feedkeys() allowed in sandbox
Matthew Winn wrote: My objection to your idea is that it won't improve security by even the tiniest bit. It's not defence in depth at all. We've probably slugged this out enough, but I'm glad to have another opportunity to promote the safe modelines message. Bram has made the point that despite repeated modeline vulnerabilities over several years, there are no known cases of a malicious attack. We have only seen PoC and jokes. I see the sense of that position - why put in a bunch of ugly checking which is going to reduce features and upset some users? Why do it if there are no known benefits? My answer is essentially an appeal to a higher moral purpose. There may never be in-the-wild exploits based on modelines, but that would make it all the easier to direct a specific attack against a targeted victim. The attacker would have a list of 10 or 20 slight security flaws in the victim's network. One of those would be the fact that the victim uses Vim. An attacker may use a Vim modeline as the coup de grace to fully own the victim's network. I find that situation offensive, and believe that modelines should be REALLY fixed. My claim is: 1. A modeline can execute untrusted code. 2. That is incredibly dangerous. 3. Any bugs in modeline handling should be fixed. 4. In addition, modelines should be sanity checked. I think we agree on points 1-3. I mentioned that the first step for point 4 should (IMHO) be rejecting any modeline beyond some fairly small maximum size. However, that was just the first part of my hoped-for sanity check. After that, I would like the modeline to be examined to determine whether there are any constructs that look dangerous. I would reject any modeline with more than ten backslashes, and would reject anything looking like an expression or 'call'. What I'd really like would be a separate sanity check that verifies that the syntax in the modeline is boringly standard 'set' options for a declared whitelist of things that a modeline is allowed to do. Note that this checking should NOT be done only in the code that executes the modeline. The checking should be an independent, prior step. That redundancy is likely to save someone's foot in future years, when extra features are added. My objection to your idea [to limit modeline length] is that it won't improve security by even the tiniest bit. You may be right. But if I were to accidentally execute malware, I would prefer that the malware was short, rather than of an essentially unlimited length. I agree that 100 bytes of malware could do more damage than I could bear, but I would still prefer that situation. For example, 100 bytes of malware might be able to erase my files, but perhaps it couldn't do something more sophisticated like launching a hidden infiltration of my network. I don't really know why I want to limit the modeline length. That's the whole point of proper security measures. Just because I can't think of a way that a long modeline might be bad, does not mean that some attacker won't find a way, particularly in five years after a bunch more stuff has been added to Vim. That means that my modelines are quite long. I'm a reasonable guyg. Let's take your longest modeline and double it. That length should be the maximum allowed for a modeline unless some new anything goes option is enabled. Re Perl tainting: I think we essentially agree on this, although I don't think Vim needs to mark an executable expression read from a modeline as tainted. Vim should immediately reject any modeline that might execute code (unless some new anything goes option is enabled). John
Re: feedkeys() allowed in sandbox
Bram Moolenaar wrote: N times as safe still isn't 100% safe. I am not claiming that sanity-checking a modeline before execution would make it 100% safe. But there have been many examples in other software where minor bugs have turned into security disasters because some simple point that could have been checked, wasn't. While code is working correctly, a simple check is redundant, and indeed is offensive because it lengthens and obscures the code. But a few simple checks may prevent disaster at some future time, when Vim is further developed. The Google test (searching for past instances of trouble with Vim's modeline) proves the case that future problems are likely. Modelines are default off when you are root. The mail filetype plugin also switches it off. Good grief - I didn't know that. So you *have* got sanity checks built in! I'll go and sit in the corner now, but thanks for confirming that multiple layers of defence are desirable. John
Re: feedkeys() allowed in sandbox
Matthew Winn wrote: If there was a security problem in Vim do you really think it couldn't be exploited in 100 characters? That's a pretty shaky foundation on which to build your security. I am quite surprised at the lack of appreciation for the merits of defense in depth here. I am not claiming that a length limit would preclude damage, just that a modeline should be sanity checked before execution, and a reasonable length would be the first check. It's certainly true that a modeline of 100 bytes could wreak havoc on an unpatched Vim. But it is quite possible that a future vulnerability might allow code to be injected from a modeline, and limiting its length *might* make the attacker's job harder. It's sensational that Vim can process files with very long lines, for the occasions we need that. But it would be absurd for Vim to process a multi-megabyte modeline. By all means abuse me for my cheeky suggestion to limit modelines to 100 bytes, but while doing that you might agree that some limit under 1MB should be enforced. A web browser should be able to handle anything thrown at it in a way that doesn't compromise security. _Every_ application should be able to handle anything thrown at it in a way that doesn't compromise security. Even if a program is perfect now, a later change can introduce a bug. Any program which can automatically execute untrusted code should sanity-check the input as a separate step from sandboxing. That is standard Security 101 stuff - not my idea. What you're suggesting isn't much different from security through obscurity. Jargon is great, but not when it's misused. Perl and Vim have exactly the same requirements: the need to safely handle code taken from an untrustworthy source. It makes no difference whether it comes directly from a network or from a disk. (If, like me, you use Vim as your source viewer for web pages, the need for the same level of security is obvious.) It doesn't matter, but for the record, Perl's tainting system is not related to the scenario you describe. Perl wants to make sure that untrusted input is not later used as the basis for some expression that could do harm, such as executing SQL code. John
Re: feedkeys() allowed in sandbox
A.J.Mechelynck wrote: Is folding really needed in a default modeline? Folding may be useful in a modeline. (Don't know what you call a default modeline.) By default modeline I mean I would like Vim to be changed so that its default behaviour is aggressively safe. If wanted, there could be a new option to enable clever features, and a user could choose to allow modelines with folding or expression evaluation, etc. But the only long-term safe procedure is to have Vim *default* to work with only very restricted modelines (set tab and other options - no way to even get near executing code). I am wondering what the lack of comment on this topic indicates. Do you understand that another modeline vulnerability could allow the next file you open to overwrite all files under your home folder? Or it might overwrite all sectors on your disk, if you have sufficient privilege. How about if you go to another computer that you rarely use. Would you be happy using Vim on that computer? Network admins in secure environments should be prohibited from using Vim. If I am overlooking something, or am overly alarmist, please tell me. For anyone new to this, enter following in Google: vim vulnerability modeline I just noticed that the fourth hit features Ciaran McCreesh who discovered a vulnerability in January 2005. John
Re: feedkeys() allowed in sandbox
Matthew Winn wrote: I don't like the idea of preventing modelines over 100 bytes. I imagine (haven't looked) that a modeline has no hard limit to its length. So multi-megabyte modelines are probably handled by Vim. That's potentially offering attackers extraordinary power. Would someone who wants a modeline longer than 100 bytes please show us an example. How about a 200-byte limit? Modelines are enabled by default, and are very useful for things like setting tabs. So most people, and all new installs, will have modelines enabled. It's very poor security practice to offer a rich auto-execution environment with a single line of defence (the sandbox). This discussion reminds me of the days of the Code Red vulnerability in IIS (Microsoft web server), and of the years of repeated vulnerabilities in Internet Explorer. The IIS and IE developers just couldn't bring themselves to build in limits to what their wonderful software could do. But a web site might need a 100KB URL with hundreds of '../' paths!. Furthermore, what am I supposed to do if I want a long, complicated but legitimate modeline? I would like a default high security setting for handling modelines. If people want modelines that do complex stuff, I would recommend setting a new anything goes option. I like Perl's approach to untrustworthy data. It's flagged as tainted at the point it is read, and anything derived from it is also flagged as tainted. Perl has to have that system because part of its intended usage is to handle data entered into web pages. It's pretty complex and has taken years to get right. Vim is a text editor - it should not automatically execute code in any old file that I might accidentally open. John
Re: feedkeys() allowed in sandbox
Bram Moolenaar wrote: That's pretty nasty. I'll make a patch right away. Thanks. However, perhaps the modeline concept needs more safety - defence in depth. Perhaps modelines should only allow a VERY limited set of operations by default (even more restricted than now). Googling for 'vim feedkeys joke' shows April 1 joke with the following (I've replaced vim with vvv): vvv: foldmethod=expr:foldexpr=feedkeys( \\esc\\x3a%!cat\\x20-n\\CR\\esc\\x3a%s/./\:)/g\\CR \\esc\\x3aq!\\CR): I'm too lazy to unobfuscate this, but one glance tells you that modelines should not be fixed - going down that path is likely to give a new vulnerability every year. Instead, modelines should be SEVERELY limited by default. Examples: Total length 100 bytes. No expressions; no function calls; no execution. Treat a double-quoted string as if in single quotes. Is folding really needed in a default modeline? John
Re: feedkeys() allowed in sandbox
Tomas Golembiovsky wrote: today somebody came to #vim, and pasted some modeline (containig joke or such). Thanks for raising that issue. I found the April 1 joke with Google. I actually noticed that posting (seeing Vim while browsing a security list caught my attention), but there's so much waffle out there that I didn't pay any attention to the point of the message. John
Re: feedkeys() allowed in sandbox
Bram Moolenaar wrote: Perhaps modelines should only allow a VERY limited set of operations by default (even more restricted than now). Sure, simply use :set nomodeline. I'm suggesting defence in depth. My vimrc might have ':set nomodeline', but what if I make a mistake? What if I'm using some other machine where I'm not sure what's in vimrc? What if I use the -u option? And there are probably other scenarios where a simple oversight could cause me to execute a modeline in an untrusted file. What if I want modelines (but never to do more than set a few options like tabs). Even setting 'textwidth' to 2 may already be considered harmful, or at least annoying. I don't mind being occasionally irritated with a stupid modeline. But I really don't want an editor to execute code when I open a file. Change volatile settings - no problem. Execute - no thanks. Perhaps we can add yet-another-option to completely disable setting some options from the modeline. Modelines should default to be safe (safe by design, AND safe because of defence in depth). If another option were added, it should be to allow expressions or other features that are potentially unsafe (not disallow unsafe features). Potentially unsafe means we're pretty sure it IS safe, but (for example), it's simply not worthwhile allowing a modeline longer than 100 bytes because if another vulnerability were ever found, we don't want to make it easy for the attacker. It's better to make sure the sandbox works as it should. Of course, but bitter experience shows that defence in depth is the best strategy. Vim has a lot of very clever features that are too complex to ever be secure, when executing a modeline from an untrusted source. John
Re: Vim freezes system ?!
Tim Chase wrote:
Or even a quality :s command instead of a function:
:%s/.\{72}/r/g
Thanks - I occasionally need to split a line into equal-length blocks
and your command is excellent. Your posts are very valuable.
However, for anyone trying this, 'r' should be '\r'.
:%s/.\{72}/\r/g
John
Re: VimTips Wiki: New Direction
Denis Perelyubskiy wrote: why don't we discuss that here: http://en.wikibooks.org/wiki/Talk:Learning_the_vi_editor/Vim/TipsSandbox/Tip_1:_the_super_star :-) You and I started talking on that discussion page. I just wanted to say (to everyone) that now I think all comments would be better made here. I totally agree with you on the need to omit most comments. One change: if it matters, we could reserve a line (or two) for a list of contributors. This needs a decision. I favour the simple approach of omitting all names, but if that is too brutal, I like your idea of adding a list of contributors - it could be several lines, at the END of the tip. John
Re: VimTips Wiki: New Direction
Brian McKee wrote: One of the first things I was thinking about mirrors the above comments. http://en.wikibooks.org/wiki/Talk:Learning_the_vi_editor/Vim/ TipsSandbox/Tip_1:_the_super_star has a bunch of thanks for the great tip! type comments with more useage info interspersed. I think those 'great tip' comments go to a separate page, while the 'use # or % instead' kind of comments need to be edited into the actual tip itself. But why keep the 'great tip' comments? The friendly atmosphere of the current Vim Tips web site is rather nice, but why put all the work of moving it to a wiki unless you make the tips more helpful? Unhelpful tips and unhelpful comments should be omitted. John
Re: VimTips Wiki: New Direction
A few fundamental decisions need to be made so the conversion script
can produce a result that minimises future effort.
How will tips be found? Can there be a contents page? If so, the
script should generate a suitable list, perhaps for hand-tweaking.
Currently, the URL of the sample tip has form:
http:.../Tip_1:_the_super_star
One of the following would be better:
http:.../the_super_star
http:.../Tip_1
http:.../Tip_1_the_super_star
In thinking about the tip URL, consider how tips should evolve.
Would tips simply be added, so potentially there could be fifty
independent tips on searching? Or (particularly with important
topics like searching), would we hope that one day there would be
a single search tips page with links to tips on different aspects
of searching.
The sample tip starts with an info box:
author
created
complexity
as of Vim (version)
rating
As has been mentioned, the info box is pretty intrusive. The page
should start with the tip. Maintaining the data in the box seems
unecessarily difficult to me - can we omit it? We can't maintain the
rating. If retained, move the box to the end of the tip.
What will be done with the comments? As several people have noted,
many of the comments are not helpful for a potential user of a tip.
I think that eventually some brutal editing should omit most
comments. Ideally, we would incorporate useful comments into the
tip. Meanwhile, we could delete unhelpful comments while retaining
and rewording and reordering helpful comments.
If people agree that comments should be severely edited, I suggest
that the script should make the job easier by omitting the By...
On... box and replacing it with a simple tip separator. Using 'edit
this page' shows that currently each By... On... box occupies
eight source lines.
It might be helpful to temporarily retain the author and date by
replacing the By... On... box with a single source line (normal
text), something like this:
-{By sample user on March 8, 2001 14:51}-
I don't see any benefit from moving the comments to the discussion
page. Why set up the wiki in the first place unless there is an
intention to improve the tips? The discussion page should be
restricted to serious pondering of what editors should do with this
tip, and with related tips. The Great tip comments are just noise.
John
Re: VimTips Wiki: New Direction
Paul Irofti wrote: Back-ups are better made on the server side. For mediawiki a regular sqldump (maybe added in crontab) is all that is necessary. Good. But can we actually do that? I thought we were planning to use a system where we have no access to the server except via a wiki interface. John
Re: VimTips Wiki: New Direction
A.J.Mechelynck wrote: Moving the tip body to a wiki page and the comments to its talk page can (IIUC) be automated. /Then/ the tip author (or maintainer, etc.) can archive the talk page, remove empty comments (of the great tip kind), refactor useful comments into the tip body, etc. My feeling is that it would be more helpful to the editor if all the comments were put underneath the tip, as at present. That seems much less effort when refactoring to me. Someone needs to try this to compare the two approaches. John
Re: VimTips Wiki: New Direction
Tom Purl wrote: I looked into the anti-spam features of Wikibooks, and they basically do the basics: blacklists for abusers and easy rollbacks. So the top 2% of spammers/vandalizers will be blocked, and it will be easy for the admins to roll back the problems created by the outher 98%. Wikibooks sounds good. I wonder if it would be worth looking for a volunteer to occasionally take a snapshot of the wiki as an emergency backup. Presumably wget or similar would be adequate, or maybe there is an easy way to get the mediawiki source. Such a backup might be useful in a few weeks, and then perhaps monthly. John
Re: VimTips - Google Wiki Usefulness
Bram Moolenaar wrote: I think this puts too much burdon the volunteers that become an admin. And it defeats the easy of use of a wiki. I was suggesting that people who have a tip, or a change, would email it to a Vim mailing list, where it would be massaged by the community, then posted to the wiki by an admin. Yes, that would burden the admins, and is against the spirit of a wiki. However, as I understand the Google wiki, a person wanting to post a tip would need to have a Google ID, and would have to mail an admin, and the admin would have to add the new person as an admin, then reply to the mail. That's also a pain for an admin, and is not really easy use of a wiki either. And which admin would be emailed? John
Re: VimTips - Google Wiki Usefulness
I like the concept of using the Google wiki. In addition to the Google is Good factor, there is the likelihood of very high reliability and zero cost. Also, it seems appropriate due to Bram's work. As has been pointed out, spam is a really big threat, and will get worse (more automated) every year. I don't see why we should ask anyone here to take on the job of manually removing junk. The problems of using the Google wiki may be a benefit in the future. How about this concept: Use the Google wiki with a dozen admins. Those admins are likely to be here, and either have a Google ID or would be willing to get one. Create a new mailing list ([EMAIL PROTECTED]), or use an existing list with a convention that the subject should start with vimtip. On the wiki, explain how to post a tip by emailing the tip to the agreed mail account. That would require the user to join the list. Alternative: Have an automated account that forwards any mail including vimtip to the agreed mailing list. A discussion here would then massage the tip and decide its fate. Perhaps it is too similar to an existing tip, or perhaps it should be incorporated into an existing tip. After a week, a kind wiki admin would post the final tip to the wiki. ---Benefits--- No spam on wiki. Each posted tip will be massaged and worthwhile. Tips on similar topics could be arranged together. Someone with a tip doesn't have to worry if it's good (it will only be posted if the community likes it). That might encourage tippers. ---Problems--- No WYSIWYG editing (might be a benefit - uniform style). Burden on community to edit tip within a few days (and likelihood of unresolved arguments). Burden on admins to post tip. Some tippers will be offended when their work is edited or rejected. That might discourage tippers. John
Re: tips project
Tom Purl wrote: I just checked the Google Code site (http://code.google.com/hosting/), and couldn't find a Vim or VimTips project. I'd try creating it, but I feel like someone more official to the project should do so. I think someone should do it before someone else takes the name (like some malicious Emacs user :), even if the tips conversion script isn't done yet. Otherwise, I'd be happy to do it and brag to my dorky friends about how integral my efforts are to the Vim project ;) Tom: Were you thinking of creating the conversion script? And copying the web site tips into the wiki? Does anyone know of some docs that might assist in this process? Does a wiki have a bulk import, or is it a matter of emulating what a person would type? John
Re: tips project
Tom Purl wrote: I just checked the Google Code site (http://code.google.com/hosting/), and couldn't find a Vim or VimTips project. I'd try creating it, but I feel like someone more official to the project should do so. I think someone should do it before someone else takes the name (like some malicious Emacs user :), even if the tips conversion script isn't done yet. Otherwise, I'd be happy to do it and brag to my dorky friends about how integral my efforts are to the Vim project ;) Tom: Were you thinking of creating the conversion script? And copying the web site tips into the wiki? Does anyone know of some docs that might assist in this process? Does a wiki have a bulk import, or is it a matter of emulating what a person would type? John
Re: tips project
Bram Moolenaar wrote: Using the project name VimTips would be good. Everyone here is used to the name Vim Tips so VimTips sounds good, but on a wiki I suggest that the name should be just Vim. It seems logical to me that a Vim Tips wiki would start with a (brief) page outlining what Vim is, then have links to other information. Links would include www.vim.org for core information, and a contents page for the wiki tips. To be useful, the existing Vim Tips should be structured into some logical order (tips on navigation, search, status line, fonts, tags, etc). Perhaps the existing structure of Vim Help could be used. OTOH the perfect is the enemy of the good, so perhaps you just want to move all tips as is, then have people slowly massage them into a coherent set of useful tips. The wiki tips project would only be helpful after some severe editing IMHO. Some tips should simply be omitted (or held under a section with a name suggesting deprecation, like Old Tips). Other tips should be combined. Unfortunately these steps would involve author angst. One good feature of the current Tips web site is that each tip is clearly written by a specific person who is prominently credited. That gives the author a good reason to correct errors and make updates. Traditionaly that would not be done in a wiki, and there may be some loss. John
Re: Tips which are spam
Andy Wokula wrote: Currently there is much spam in recent comments: http://vim.sourceforge.net/tips/recent_notes.php Last time this was discussed I got the impression that there is a feeling that if no one reads the spam, then it is not a problem. But I think the situation is worse than that. The spammers don't care if anyone reads the tips. They want the optimisation to their search ranking in Google et al from having links to their site. Helping these leaches is no longer acceptable IMHO. Lots of places on the Internet have had to implement a simple logon or at least a captcha - anonymous posting can't be allowed. John
Re: BOF Vim 8 - EncryptLine
Robert Lee wrote: What is wrong with SCP/SFTP? If these are not available, externally encrypt the file. You can even make a vim macro to do the job IIRC. Text editors don't do encryption and never should. I promise to stop posting when everyone else does! SCP etc just encrypts the traffic when copying a sensitive file to another machine. After backups are taken and stored, and various other stuff happens, your sensitive file can end up in quite a lot of places. Defence-in-depth means you have to encrypt the actual secret (the file, or the secret parts of the file). As Tony pointed out, Vim can encrypt a file. That is good because if you rely on scripts and what-have-you, there can be temp files, and files left over when something crashes. It's much better if Vim handles the decryption and encryption for you - much more convenient, and reliable, and a lot more secure. I can see that my EncryptLine proposal doesn't have any support. Fair enough - I accept the judgement. But the idea is not stupid. When you document tricky stuff on a network, you end up with a file where normal file-access security is good enough to protect 95% of the content. What about the passwords in the document? You really need to encrypt the file to protect them, but that means you have to decrypt the file (enter its key) everytime you need to extract some fact (like an IP address). I have actually implemented my EncryptLine on an obsolete editor and it works really well - but only because I have a need for it. John
Re: BOF Vim 8 - Suggestions
A.J.Mechelynck wrote:
IMHO it is important that function keys (with the exception of F1 = :help)
should by default _not_ have preset functions in Vim, in order that they
be safely available for whatever mappings any user would want to assign to
them, without competing with existing functions. There aren't so many keys
that users can safely use as the {lhs} of a mapping with no fear that it
will collide with an existing function.
For the record, I don't want function keys to be mapped by default. I am
suggesting that there should be a really simple (preferably built-in) way to
invoke a sample set of mappings for a C programmer.
For example, I could then send someone an email telling them to download
Vim and enter command :set template=cprogram (or whatever). I could
then advise the new user (a C programmer, not a dummy) that they can do
(whatever) to grep stuff, and (whatever) to use ctags. You would then have
one more converted Vim user!
As a final comment, I would recommend that any such template should
include mappings:
- Clear search highlights and the message line.
- Search for highlighted text.
John
Re: BOF Vim 8 - EncryptLine
Marc Weber wrote: Make an easy way to encrypt a secret within a line. Then you can have a simple text file to document stuff, with embedded secrets. On reading, you only need to enter a key if you want to see a secret. I don't think this should be a general vim feature either. Yet another idea to solve this: Why not use syntax and set forgground/background color to the same value to hide the text? Then you don't even notice that there is text. OK - I'm convinced that EncryptLine is not wanted! However, I want the secret to really be encrypted because I use a system to copy my data files from machine to machine for backups. The encryption is to protect the secret if someone somehow gets access to the file. John
Re: Odp: BOF Vim 8 - Suggestions
A.J.Mechelynck wrote: I personally recommend to create the following as $HOME/_vimrc (or $HOME/.vimrc) immediately after first installation, and to add tweaks as one gets going: ... Good advice, as always, Tony. But I am trying to crack a different nut. In the BOF talk, Bram really was asking for ideas on what would make new users flock to Vim. These days there are a lot of really good editors. No one is going to spend a couple of hours methodically working through vimtutor (sorry Bram!). A new user is just going to try Vim as an editor. If the default Vim behaviour confuses or upsets the user, Vim will never be used again. I sense an attitude here that it's just the luser's loss if they don't learn how to use Vim. Fair enough, but there should be a way for a non-vi user to enter a command telling Vim I'm one of those 95% of people who use a modern PC - please switch to a useful mode. I had the experience of advising a very competent programmer who got a job working on virtual memory in the Linux kernel (previously he had only dabbled in Linux, and had developed under Windows). At the time, we couldn't meet, and I was restricted to providing advice via email. Let me tell you, it is pretty well impossible to convince someone to use Vim via email. The guy wanted to do stuff right now, and I would send some 100-line email with instructions on how he could do this and that to vimrc, and then everything would be simple. One problem was how search highlighting is persistent (which is great), but it is very distracting to some people when you want to turn your attention to another issue. Telling him how to map a key to do ':nohl' is just unnecessary mumbo jumbo. John
Re: BOF Vim 8 - EncryptLine
Nikolai Weibull wrote: Make an easy way to encrypt a secret within a line. [Really complex scheme to implement this.] Why is it not enough to simply implement a function that encrypts/decrypts a range of text, much like g? ROT13s a range of text? Because the scheme needs to be simple to use. Once you have that, you have the danger that you will accidentally encrypt or decrypt twice ... then you can lose data. You need some safety checks from a carefully-written program ... although I suppose you could incorporate all that in your hypothetical function. John
Re: BOF Vim 8 - EncryptLine
Bram Moolenaar wwrote: Suggested new feature: Make an easy way to encrypt a secret within a line. This is very a specific feature. You should implement this in a script, this doesn't sound like something Vim should support internally. OK. I just thought I would mention the concept because I find it useful, and AFAIK it's a novel idea that might have appealed. John
Re: BOF Vim 8 - EncryptLine
Nikolai Weibull wrote: 2. Don't write down passwords at all - use phrases that you remember instead 3. Don't write down passwords where other people might walk by and see what you're typing Let's not start a religious war, but FWIW many authorities have changed their mind and no longer advocate the don't write down a password advice. In particular, any network admin simply has to record passwords and other sensitive information - you can't reliably remember more than two or three passwords, particularly when you're not using them often. There are many password safe utilities for this, but I like a simple text file with the secrets encrypted, yet easily viewable (without changing the file). John
Re: BOF Vim 8 - Suggestions
Bram Moolenaar wrote: Mostly PageUp and PageDown do the reverse of each other. If you mean that the cursor has moved, that is a completely different thing. I'm not sure what completely different thing adds. I'm just trying to respond to your call for suggestions on how to make Vim more attractive to new users. I know a couple of people who don't like the fact that, in Vim, PageUp does not reverse PageDown. Naturally I am talking about the cursor moving. Switching off search highlighting is part of the tutor. People who skip the vimtutor are going to run into trouble anyway. OK - but you could make Vim more attractive if (when enabled by some new option) pressing Escape in Normal mode cleared search highlights and cleared the message line. I do agree that good defaults are important. But backwards compatibility is also important. It's not always easy to make a choice. Suggestion: Provide a simple way for a user to invoke a standard set of predefined mappings. In fact, there could be various predefined themes that insert useful settings into vimrc. Then, I could write an email to a friend saying Run gvim and do stuff to select a theme. Then you can press F11 to do clever thing. For example, perhaps F11 = :cn, Shift-F11 = :cp. Naturally there would be a lot of different opinion on what should be included. Anything would be better than the current situation where I have to convince a prospective new user to become a guru before they can use some of Vim's brilliant features. John
Re: Odp: BOF Vim 8 - Suggestions
Nikolai Weibull wrote: Perhaps a better way is to leave 'hlsearch' off and provide a binding that toggles it on and off. That way you don't get the distracting highlighting until you actually request it. OK but I imagine most people would like hlsearch on while they are searching (I certainly do). I am trying to make a slightly different point: 1. Bram has asked for ideas on how to popularise Vim. 2. I think better default behaviour would be best. IMHO it is quite idiotic that Vim has the really great feature of globally highlighting searches, but the user has to learn how to map keys to make it work in a sensible way. I suppose there are people who don't mind the distracting highlights once they've finished looking at them, but I'm pretty confident that most people here have a mapping to deal with the situation. All of what I am saying it totally irrelevant if there is no desire to popularise Vim. John
Re: Odp: BOF Vim 8 - Suggestions
A.J.Mechelynck wrote: I sense an attitude here that it's just the luser's loss if they don't learn how to use Vim. Fair enough, but there should be a way for a non-vi user to enter a command telling Vim I'm one of those 95% of people who use a modern PC - please switch to a useful mode. Easy Vim and mswin.vim were written for exactly that purpose, and IMHO they cripple Vim to almost complete _un_usefulness. Vim is a very powerful editor, but to be able to use it, you need at the very least to understand the difference between Normal mode and Insert mode. To use it like the fine tool it is, you need to study, just like one needs to study to become a concert pianist. If you're one of those 95% of users who need to be led by the hand by their OS and editor, and are too dumb for a modal editor, go use Notepad and stop bugging us. I accidentally said mode with the unintended suggestion that I was whining about Normal mode etc. Sorry. I totally agree that a Vim user has to work with Vim, and not try to pretend it is something different. Normal and Insert modes are fine by me. By a useful mode I meant a more accessible implementation of core features. Take searching. Incremental search and global search highlighting are sensational. But new users are going to want to turn the highlighting off (gedit has a menu item for this). It's obvious that most people are going to want a key mapping. So why not provide a standard mapping? OK, I can understand that you may not want the standard mappings enabled by default. So provide some new command to enable them. Why not just type :noh at the keyboard then, if you can't use a mapping? Are you suggesting this in the context of my proposal to improve the experience for new Vim users, in order to popularise Vim? There has to be an EASY way to turn off search highlights! John
Re: BOF Vim 8 - EncryptLine
Nikolai Weibull wrote: In particular, any network admin simply has to record passwords and other sensitive information - you can't reliably remember more than two or three passwords, particularly when you're not using them often. I don't understand what you're trying to say in the first part of your sentence. I promise the list that I won't post about this again, but FWIW I'm not telling you how I work, I'm describing how most people in the business say they work (from talking to people, and following lists etc). Most networks have lots of devices which require accounts. A router, a firewall, a mail server, ten other servers, etc. Then there are your various email accounts - some important, some junk. Single sign-on can integrate many, or even most of these. But not all. So, most network admins need to record all the account details, possibly with notes, e.g. just how do you log on to that wireless access point that you last configured six months ago. There are many password safe utilities for this, but I like a simple text file with the secrets encrypted, yet easily viewable (without changing the file). Which defeats the whole point of having multiple passwords, as if someone figures out the master password then the other passwords will also be available. Given that you're going to reveal the password to anyone with a knife, there doesn't seem to be much point in having bullet proof security. I'm sure many people do as you suggest, but take it from me that many other people do not use the same password on their firewall and their mail server etc. So it's better to use one good password/passphrase and stick with it. Diceware is a really nice system: http://www.diceware.com/ John
Re: Odp: BOF Vim 8 - Suggestions
Nikolai Weibull wrote: Well, hlsearch only kicks in /after/ you've completed your search, whether you're using 'incsearch' or not. I once thought this was a nice feature, but I've realized that I rarely need to have other matches highlighted. I mean, either I've found what I want using 'incsearch', or I have failed to find the text location that I wanted, and highlighting the other matches won't really help much. Narrowing your search using 'incsearch' is often a lot quicker than doing a quick search and pressing 'n' a bunch of times to get to the right match. What you describe works well in many circumstances. But I find that the global highlighting is invaluable when programming. Built-in to Vim is the fact that pressing * searches for the current word. Programmers love this. IMHO they would appreciate a simple way to clear the highlights _and_ the message line when turning attention to something else. Searching for non-existent junk is just too distracting to my mental process. John
Re: BOF Vim 8 - Suggestions
Ilya Bobir wrote: If you think that you can provide a better defaults for novice users you can just write a script that will adjust vim the way you see it and ask Bram to add it into the distribution along with a note in the tutor OK. Perhaps that would handle the issue. I don't care how it's done, my aim is just to make the suggestion that to popularise Vim, it would be helpful to provide easily-accessible and well-thought-out defaults for some of Vim's more clever features. John
Re: BOF Vim 8 - Suggestions
Bram Moolenaar wrote: No, I don't want that. Pressing ESC is to get back to Normal mode, it should not have side effects like this. OK. But my suggestion was not that ESC would go to Normal mode _and_ clear highlighting. My proposal was that if I start in Insert mode, then press ESC I would be in Normal mode (as normal :), but if I press ESC again then highlighting and message text would be cleared. I'm happy so long as you've heard my idea. A couple of people here seem to think that _I_ want these suggestions. Not at all. I'm fluent in Vim and don't need any of my proposals for myself. But my recent experience of trying to make Vim attractive to a programmer moving into Linux showed me that some simple changes to Vim might make it a lot more attractive to new users. You appear to assume that what you want is what everybody wants. No! For posterity let me record that I am not one of those people. I'm only making these suggestions because I know you want to promote Vim usage, and some way to easily invoke a pre-defined set of behaviour for a modern PC would help IMHO. Imagine how many exceptions we need to handle in the documentation: if you use theme X then this doesn't work and you need to type XYZ. I take your point and agree. But I will make one final suggestion: Do not hide how Vim works. Tell users about vimrc, and what a mapping is, etc. People who would use Vim are smart, and can instantly understand that there is a config file, and that keys can be mapped. But a new user will probably not want to take the time to work out the details and their optimum settings right now. Take the wonderful quickfix window (which I use mainly for vimgrep). Using quickfix with ':copen' etc just doesn't work for me. I don't mind typing a few commands, but in this situation, the commands interfere too much with my thoughts. Once I mapped keys for :copen, :cn and :cp, quickfix was a magnificent feature. My vague concept about a theme is that it would insert text into vimrc. The user would be told this, and they could modify the text to taste. Right now, it is pretty easy to get BufExplorer working, and then \be is the default key sequence to start it. That's like what I am proposing for other killer features of Vim. You can always tell someone to download your script and use it. There are too many tips and scripts already. I was hoping (*not* for me!) to integrate some of the best work procedures into one or two pre-defined behaviours. John
BOF Vim 8 - Suggestions
Sorry I'm late, but I just listened to the Vim BOF session that Bram mentioned three months ago. In the talk, Bram sounded quite evangelical with regard to promoting Vim usage, and he asked for suggestions on how he should best spend his limited time in working towards a new version (Vim 8). I am posting this introductory message with my response, plus a second message describing a feature I would like (EncryptLine). The best way to expand Vim usage IMHO would be to work out better default settings to improve the first hour of contact. In addition, perform necessary fixups, but resist new features. If I were starting Vim usage now, I would probably abandon it as soon as I discovered that pressing PageUp did NOT reverse the effect of PageDown. I still hate that! Vim has many wonderous features, but I imagine that many people don't stay to find them because of irritations like PgUp/PgDn. Highlighting all matches when searching is excellent, but there needs to be a way for a new user to turn it off. I map Space for this, however, it would be better to press Escape to clear highlights and message text (if Escape is pressed while in Normal mode, remove search highlights and clear any message). Maybe also have a way for this feature to be turned off. I won't say more now. If Bram feels that improved defaults would be worth investigating, a discussion here would probably be best. OTOH people who dream in Vim script may not be the best source of ideas on how Vim should be configured to win new converts. John
BOF Vim 8 - EncryptLine
Suggested new feature:
Make an easy way to encrypt a secret within a line.
Then you can have a simple text file to document stuff, with
embedded secrets. On reading, you only need to enter a key if you
want to see a secret.
Example lines before encryption:
server12 { admin topsecret } any text
mybank { account 123456789 pin 1234 }
Example lines after encryption:
server12 {~8vP09fb3+Pn6+/z9/v8AAwocSE9cDYPAYJUThgE} any text
mybank {~afSDKoy9saGMCZ91x6F7pHkwdzEcMBoGCSqGSIb3DQEJ}
When viewing a file with encrypted secrets, it doesn't matter if
others are shoulder surfing. You only need to get rid of onlookers
for the short time it would take to enter a key and view a secret in
the message line (which would not change the file).
I implemented this scheme in an obsolete editor many years ago,
and offer the suggestion in case it appeals to Bram. However, as
noted in my Suggestions message, I think new features should be
resisted in favour of fixups, so I won't be offended if this is
ignored.
A more detailed description of the proposal follows.
A secret is entered between { and } on a single line.
There is a space after the opening brace.
The encrypted result is stored as base64 text, with ~ inserted as
the first character. The space (plaintext) and tilde (ciphertext)
are safety checks so text is not encrypted or decrypted twice.
These commands would be required:
EnterKey - Prompt user to enter a key for encryption/decryption.
EncryptLine - Encrypt text inside braces on the current line.
DecryptLine - Reverse EncryptLine.
ShowSecret - Show decrypted secret in the message line.
EnterKey prompts the user and allows them to enter a key (no echo).
The key is hashed, and the hash is retained in memory for this
session. It can be cleared by using EnterKey to enter a blank key.
The hashed key is used for any subsequent encryption and decryption.
EncryptLine checks that the current line contains { (with space),
followed by }. It then uses the hashed key to encrypt the text
between the braces, then replaces that text in the current line with
a base64 encoded form of the ciphertext.
EncryptLine inserts a tilde (~) after the first brace. This is a
safety mechanism so you won't accidentally encrypt a line twice.
EncryptLine inserts a small amount of random padding (salt). The
padding is of variable length so the length of the secret is not
known to intruders. However, there is only a small amount of padding
so the result is fairly compact.
ShowSecret decrypts the secret in the current line, and displays the
plaintext in the message line. The file is not changed. There should
be an easy way to put the plaintext in the clipboard, and an easy
way to blank the displayed secret.
DecryptLine reverses EncryptLine, changing the current line. It does
nothing (apart from display an error) if the result is not
reasonable (the ciphertext must be a tilde followed by base64, and
the decryption should satisfy certain sanity checks, and should
yield printable text starting with a space). This is a safety check
to avoid losing data if the wrong key is used to decrypt.
John
