Re: Opening files with Unicode names under Windows?
On Sat, Sep 13, 2008 at 16:09, Bram Moolenaar wrote: I wonder why this has not been a problem before. It is not surprising that this does not come up often. Those people who use localized versions of Windows are less likely to run into such problems (for example on a Hungarian Windows system the default encoding does include ő and ű). The rest of us learn quickly that it is best to avoid file names with characters that are not present in the system encoding. Even today there are many programs that don't understand Unicode, and these file names can cause all kinds of weird problems (like a picture viewer skipping them in a slide show, or even worse: a CD burner program silently omitting them from the CD ...) Also the names of those files that are most commonly edited with Vim (like source code) are unlikely to contain non-ascii characters. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On Sep 11, 1:08 am, Tony Mechelynck [EMAIL PROTECTED] wrote: On 10/09/08 22:59, Szabolcs Horvát wrote: 2008/9/10 Szabolcs[EMAIL PROTECTED]: It turns out that I was wrong. Vim *can* open these files if I use the :e command, or I browse to the file with Vim as you suggested, or I just drag it onto the Vim window. The problem is only present when I use the Edit with Vim entry of the right click menu. So I suppose that the gvimext shell extension is the culprit. It seems that gvimext does not support Unicode file names, so I attempted to fix it ... However, I am not a programmer, and I know absolutely nothing about Windows programming (I just Googled for documentation) ... I would appreciate it very much if someone could take a look at my primitive effort to add Unicode support, and tell me if I did something stupid. (It does appear to work when tested with the őű.txt file.) Well, I don't know anything about programming Windows for Unicode but you might test the following names (I'm adding quotes because some of them have spaces) Э-эй ухнем.txt Ĉu.ĉi-tie.ĉiuaŭtune.ŝiriĝas.folioj.htlm 爲無爲、則無不治。.txt أَلسَّلامُ عَلَيْكَمْ.txt Paste them, don't hand-copy them ;-). I don't think the content of the files matters. I expect the last two to be particularly difficult because of the high CJK codepoints and the RTL Arabic words with composing characters in them. I don't guarantee that all values are permitted as filenames but they are meaningful text (well, more or less; let's say they're linguistically OK) and I intentionally didn't use question marks and asterisks (which I think are Yes, all of these names are working. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On 11/09/08 11:18, Szabolcs wrote: On Sep 11, 1:08 am, Tony Mechelynck[EMAIL PROTECTED] wrote: On 10/09/08 22:59, Szabolcs Horvát wrote: 2008/9/10 Szabolcs[EMAIL PROTECTED]: It turns out that I was wrong. Vim *can* open these files if I use the :e command, or I browse to the file with Vim as you suggested, or I just drag it onto the Vim window. The problem is only present when I use the Edit with Vim entry of the right click menu. So I suppose that the gvimext shell extension is the culprit. It seems that gvimext does not support Unicode file names, so I attempted to fix it ... However, I am not a programmer, and I know absolutely nothing about Windows programming (I just Googled for documentation) ... I would appreciate it very much if someone could take a look at my primitive effort to add Unicode support, and tell me if I did something stupid. (It does appear to work when tested with the őű.txt file.) Well, I don't know anything about programming Windows for Unicode but you might test the following names (I'm adding quotes because some of them have spaces) Э-эй ухнем.txt Ĉu.ĉi-tie.ĉiuaŭtune.ŝiriĝas.folioj.htlm 爲無爲、則無不治。.txt أَلسَّلامُ عَلَيْكَمْ.txt Paste them, don't hand-copy them ;-). I don't think the content of the files matters. I expect the last two to be particularly difficult because of the high CJK codepoints and the RTL Arabic words with composing characters in them. I don't guarantee that all values are permitted as filenames but they are meaningful text (well, more or less; let's say they're linguistically OK) and I intentionally didn't use question marks and asterisks (which I think are Yes, all of these names are working. Well, Bram, have you seen the patch to gvimext earlier in this thread? What do you think of it? Best regards, Tony. -- Schnuffel, n.: A dog's practice of continuously nuzzling in your crotch in mixed company. -- Rich Hall, Sniglets --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On 10/09/08 04:15, John Beckett wrote:
Vadim Zeitlin wrote:
So, for example, opening a file Аня.vcf opens the file
???.vcf on my English Windows system.
Tony's advice about encoding might fix the situation. I have the following in
my
vimrc:
set encoding=utf-8
and can open a file with the name you mentioned on Windows. The font I'm using
displays squares for the file name characters, but the window title is
correct.
John
Beware that if your keyboard includes some characters which aren't
strict 7-bit US-ASCII, the above :set line (used alone) will probably
make them unusable, especially in Console Vim but also in some GUI
versions (including, I think, Windows), unless you take special care of
those characters. This is what I use:
if has('multi_byte')multibyte features compiled-in
if encoding !~? '^u' the OS locale is not Unicode
if termencoding == '' empty means 'use enc'
let termencoding = encoding avoid clobbering keyboard codes
endif
set encoding=utf-8 we can do it, now that the kb is taken care of
endif
set fileencodings=ucs-bom,utf-8,latin1 heuristics for existing files
setglobal bomb fileencoding=latin1defaults for new files
'bomb' doesn't apply to latin1
it applies when 'fenc' is manually set to Unicode
endif
The :setglobal line is optional, it is a question of personal
preference. In the 'fileencodings' (plural) setting, ucs-bom should be
first, and an 8-bit value should be last (because anything after the
first 8-bit value is never used).
Best regards,
Tony.
--
Pure drivel tends to drive ordinary drivel off the TV screen.
--~--~-~--~~~---~--~~
You received this message from the vim_dev maillist.
For more information, visit http://www.vim.org/maillist.php
-~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
Hello Tony, On Sep 10, 3:30 am, Tony Mechelynck [EMAIL PROTECTED] wrote: Assuming I am not missing some trivial option/workaround which would allow to open these files in Vim [obviously I did... sorry for the noise and thanks a lot for your answer!] 2. What is your 'encoding' option set to? Vim cannot, for any purpose, use any character which has no representation in your current 'encoding'. This was the problem, adding set encoding=utf-8 did fix the problem. Sorry for not thinking to try it, I somehow thought that encoding only applied to the file contents and not the file names. Thanks a lot for your helpful answers Tony and John! VZ --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On 10/09/08 16:12, Vadim Zeitlin wrote: Hello Tony, On Sep 10, 3:30 am, Tony Mechelynck[EMAIL PROTECTED] wrote: Assuming I am not missing some trivial option/workaround which would allow to open these files in Vim [obviously I did... sorry for the noise and thanks a lot for your answer!] 2. What is your 'encoding' option set to? Vim cannot, for any purpose, use any character which has no representation in your current 'encoding'. This was the problem, adding set encoding=utf-8 did fix the problem. Sorry for not thinking to try it, I somehow thought that encoding only applied to the file contents and not the file names. Thanks a lot for your helpful answers Tony and John! VZ 'encoding' applies to file contents (after conversion by 'fileencoding' as set by 'fileencodings' or ++enc when reading), but it also applies (after 'termencoding') to what you type at the command-line and (after :scriptencoding) to what is read from a sourced script. It's always better to set 'encoding' to a superset of everything you might be using in your Vim session. Best regards, Tony. -- Queensboro president Donald Mannis, charged with receiving bribes in exchange for city contracts, resigned on Tuesday. Mannis feels he must devote more time to impending litigation, some of which might emanate from a recent statement he made comparing New York Mayor Ed Koch to Nazi Martin Bormann. A spokesman from the Bormann estate said they are weighing the odds of a slander suit. Mayor Koch could naturally be reached for comment, but we chose not to listen. -- Dennis Miller, Saturday Night Live --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On 10/09/08 21:00, Szabolcs wrote: I do have set enc=utf-8 in my vimrc, but Windows gVim (on English WinXP) still cannot open files with non-latin1 characters in the name. For example, opening őű.txt will try to open ou.txt. I use version 7.2 downloaded from www.vim.org. What might be the difference between my and Vadim Zeitlin's setup? I don't know. Vadim is using Cyrillic characters in his filenames and you're using accented Latin, but should that make a difference? Can you browse a directory containing that kind of files, and are the filenames displayed correctly? Let's say you try :cd magyar :view ./ (or similar) how are the filenames displayed? If you put the cursor on a Hungarian double-acute-accented vowel and hit ga -- what does Vim answer? Best regards, Tony. -- Sex is like a bridge game -- If you have a good hand no partner is needed. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On Sep 10, 9:22 pm, Tony Mechelynck [EMAIL PROTECTED] wrote: On 10/09/08 21:00, Szabolcs wrote: I do have set enc=utf-8 in my vimrc, but Windows gVim (on English WinXP) still cannot open files with non-latin1 characters in the name. For example, opening őű.txt will try to open ou.txt. I use version 7.2 downloaded from www.vim.org. What might be the difference between my and Vadim Zeitlin's setup? I don't know. Vadim is using Cyrillic characters in his filenames and you're using accented Latin, but should that make a difference? Can you browse a directory containing that kind of files, and are the filenames displayed correctly? Let's say you try :cd magyar :view ./ (or similar) how are the filenames displayed? If you put the cursor on a Hungarian double-acute-accented vowel and hit ga -- what does Vim answer? Hello Tony, It turns out that I was wrong. Vim *can* open these files if I use the :e command, or I browse to the file with Vim as you suggested, or I just drag it onto the Vim window. The problem is only present when I use the Edit with Vim entry of the right click menu. So I suppose that the gvimext shell extension is the culprit. There are easy workarounds, so this is not a serious problem. I just got so used to using the right-click menu for opening files with Vim that I didn't even try other methods ... Szabolcs --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
2008/9/10 Szabolcs [EMAIL PROTECTED]: It turns out that I was wrong. Vim *can* open these files if I use the :e command, or I browse to the file with Vim as you suggested, or I just drag it onto the Vim window. The problem is only present when I use the Edit with Vim entry of the right click menu. So I suppose that the gvimext shell extension is the culprit. It seems that gvimext does not support Unicode file names, so I attempted to fix it ... However, I am not a programmer, and I know absolutely nothing about Windows programming (I just Googled for documentation) ... I would appreciate it very much if someone could take a look at my primitive effort to add Unicode support, and tell me if I did something stupid. (It does appear to work when tested with the őű.txt file.) --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~--- gvimext.patch Description: Binary data
Re: Opening files with Unicode names under Windows?
On 10/09/08 22:59, Szabolcs Horvát wrote: 2008/9/10 Szabolcs[EMAIL PROTECTED]: It turns out that I was wrong. Vim *can* open these files if I use the :e command, or I browse to the file with Vim as you suggested, or I just drag it onto the Vim window. The problem is only present when I use the Edit with Vim entry of the right click menu. So I suppose that the gvimext shell extension is the culprit. It seems that gvimext does not support Unicode file names, so I attempted to fix it ... However, I am not a programmer, and I know absolutely nothing about Windows programming (I just Googled for documentation) ... I would appreciate it very much if someone could take a look at my primitive effort to add Unicode support, and tell me if I did something stupid. (It does appear to work when tested with the őű.txt file.) Well, I don't know anything about programming Windows for Unicode but you might test the following names (I'm adding quotes because some of them have spaces) Э-эй ухнем.txt Ĉu.ĉi-tie.ĉiuaŭtune.ŝiriĝas.folioj.htlm 爲無爲、則無不治。.txt أَلسَّلامُ عَلَيْكَمْ.txt Paste them, don't hand-copy them ;-). I don't think the content of the files matters. I expect the last two to be particularly difficult because of the high CJK codepoints and the RTL Arabic words with composing characters in them. I don't guarantee that all values are permitted as filenames but they are meaningful text (well, more or less; let's say they're linguistically OK) and I intentionally didn't use question marks and asterisks (which I think are forbidden). Best regards, Tony. -- It was a Roman who said it was sweet to die for one's country. The Greeks never said it was sweet to die for anything. They had no vital lies. -- Edith Hamilton, The Greek Way --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: Opening files with Unicode names under Windows?
On 10/09/08 01:45, Vadim Zeitlin wrote: Hello, Sorry in advance if I'm simply missing something but it looks like Vim can't open files with Unicode file names which are not convertible to the current locale. I'm still using 7.0 (standard Win32 version) but looking at http://vim.svn.sourceforge.net/viewvc/vim/vim7/src/os_win32.c?view=markup it seems to be still the same in the latest code as get_cmd_argsW() function calls WideCharToMultiByte(GetACP()) and this fails (returns a string full of question marks) if the string can't be represented in the current user encoding. So, for example, opening a file Аня.vcf opens the file ???.vcf on my English Windows system. The annoying thing is not only Vim fails to open it but that notepad (sic) opens it just fine so this looks like the first example of something notepad does better than Vim I found since a dozen years of using Vim ;-) Assuming I am not missing some trivial option/workaround which would allow to open these files in Vim and also that nobody is already working on fixing this, is it realistic to someone reasonably competent with Win32 but not familiar with Vim code base to fix this? Looking at the code it seems to be rather difficult to replace the narrow strings representing the file names with the wide character ones so I thought about simply replacing CP_ACP in os_win32.c with CP_UTF8 and then converting this UTF-8 string to wchar_t before passing it to any Windows functions. Does this look like a reasonable approach? Thanks in advance for any help! VZ 1. Which version of Windows are you using? Which compiler was used to compile your version of Vim (as seen at the bottom of the output of :version)? (Steve Hall uses Cygwin gcc and Bram uses MS Visual C, I think boith are OK, as is the MinGW version of gcc; but I had problems with Unicode in general when I was on W98 with the Borland C compiler) Steve's builds are at https://sourceforge.net/project/showfiles.php?group_id=43866package_id=39721 ; if you want to compile Vim yourself, I recommend the procedure based on Cygwin gcc, described at http://users.skynet.be/antoine.mechelynck/vim/compile.htm 2. What is your 'encoding' option set to? Vim cannot, for any purpose, use any character which has no representation in your current 'encoding'. I'm going to bed now, but in the meantime you can check http://vim.wikia.com/wiki/Working_with_Unicode and/or http://vim.sourceforge.net/scripts/script.php?script_id=789 (Switching to Unicode in an orderly manner). If it doesn't work for you, give specifics, and if no one answers while I sleep, I will after I wake up. Best regards, Tony. -- I don't believe in astrology. But then I'm an Aquarius, and Aquarians don't believe in astrology. -- James R. F. Quirk --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
RE: Opening files with Unicode names under Windows?
Vadim Zeitlin wrote: So, for example, opening a file Аня.vcf opens the file ???.vcf on my English Windows system. Tony's advice about encoding might fix the situation. I have the following in my vimrc: set encoding=utf-8 and can open a file with the name you mentioned on Windows. The font I'm using displays squares for the file name characters, but the window title is correct. John --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
