Re: non-latin characters aren't displayed correctly in Windows menu
On 6/13/06, A.J.Mechelynck wrote:
Mojca Miklavec wrote:
On 6/12/06, A.J.Mechelynck wrote:
You must make sure that you have:
- an 'encoding' which includes the non-Latin characters you want to use
- (in console Vim) a terminal code page which includes them
What is that? And console vim if you mean the one without GUI menus
is not always there (I have it one one computer, but not on the
other).
Console Vim is the one without GUI. On Windows a different executable
must be used, usually named vim.exe (or possibly something starting in
vim as in vimd.exe for a debug version etc.); it displays in a Dos
Box. On Unix a single executable can be used as a GUI (through X11) or
as a console version (displaying on /dev/tty or on xterm, konsole, etc.)
depending on how it is invoked.
OK, when I installed the new vim 7, it's there. It wasn't present in
the old 6.4 version, but that's not really important: I (would/will)
never use it anyway.
'guifont' empty means some default system font, such as Fixed. IMHO it
is not he prettiest but if you're satisfied with it you may stay with it.
I don't say that it's the prettiest, but I tried to search for some
better fonts using some suggestions in vim recipes. The problem is
that most fonts mentioned don't support Central European characters.
(And some fonts that I have need to big sizes to be readable. Lucida
Consile is the only other acceptable font that I know/have.) I would
really like to use something else, but I didn't find anything more
suitable yet.
There is a -menufont {font} command-line option, but from where it is
described in the help I fear it is only applicable to X11 (all
Unix/Linux versions and possibly some MacOsX versions). AFAIK the font
used for menus in Windows is common to all applications and thus outside
the reach of gvim. Try the following:
gvim -N -u NONE
:language messages
:set encoding?
Great! Thanks a lot. That did it! [grrr ... I'm so stupid that I
didn't try to remove my _vimrc file before]
I get:
Current messages language: SL
encoding=cp1250
The problem was that I had the following two lines in _vimrc:
set encoding=utf-8 termencoding=cp1250
set fileencodings=ucs-bom,utf-8,cp1250
I don't know how exactly each one influences the behaviour (the second
line was suggested to me on this mailing list some time ago), but I
had to comment out the first one and the problem has gone. I had
numerous problems with utf-8 before adding those lines. For some
strange reason Windows doesn't recognize vim as being able to do
Unicode, so it communicates in an old cp1250 fashion with it. cp1250
is indeed set in the regional settings, but I didn't manage to explain
to Windows that vim can handle utf-8 as well.
OK, I only left
set fileencodings=ucs-bom,utf-8,cp1250
in my _vimrc and the menues seem to be OK now.
But now my second question: if I open some utf-8 file, it opens as
utf-8, which is OK. But when I open a new file and save it, it's saved
in cp1250 unless I type :set encoding=utf-8 before. So, after saying
set encoding=utf-8 the menu stays OK, but if I put that into my
_vimrc, the encoding in menu is broken. Any remedy for it?
Thanks a lot,
Mojca Miklavec
Re: non-latin characters aren't displayed correctly in Windows menu
Mojca Miklavec wrote:
On 6/13/06, A.J.Mechelynck wrote:
Mojca Miklavec wrote:
On 6/12/06, A.J.Mechelynck wrote:
You must make sure that you have:
- an 'encoding' which includes the non-Latin characters you want
to use
- (in console Vim) a terminal code page which includes them
What is that? And console vim if you mean the one without GUI menus
is not always there (I have it one one computer, but not on the
other).
Console Vim is the one without GUI. On Windows a different executable
must be used, usually named vim.exe (or possibly something starting in
vim as in vimd.exe for a debug version etc.); it displays in a Dos
Box. On Unix a single executable can be used as a GUI (through X11) or
as a console version (displaying on /dev/tty or on xterm, konsole, etc.)
depending on how it is invoked.
OK, when I installed the new vim 7, it's there. It wasn't present in
the old 6.4 version, but that's not really important: I (would/will)
never use it anyway.
'guifont' empty means some default system font, such as Fixed. IMHO it
is not he prettiest but if you're satisfied with it you may stay with
it.
I don't say that it's the prettiest, but I tried to search for some
better fonts using some suggestions in vim recipes. The problem is
that most fonts mentioned don't support Central European characters.
(And some fonts that I have need to big sizes to be readable. Lucida
Consile is the only other acceptable font that I know/have.) I would
really like to use something else, but I didn't find anything more
suitable yet.
Courier_New has a larger variety of glyphs than most other fixed-width
encodings. If Lucida_Console hasn't got the glyphs you need, I recommend
Courier_New.
There is a -menufont {font} command-line option, but from where it is
described in the help I fear it is only applicable to X11 (all
Unix/Linux versions and possibly some MacOsX versions). AFAIK the font
used for menus in Windows is common to all applications and thus outside
the reach of gvim. Try the following:
gvim -N -u NONE
:language messages
:set encoding?
Great! Thanks a lot. That did it! [grrr ... I'm so stupid that I
didn't try to remove my _vimrc file before]
I get:
Current messages language: SL
encoding=cp1250
The problem was that I had the following two lines in _vimrc:
set encoding=utf-8 termencoding=cp1250
This tells gvim to represent the data internally using UTF-8 but to go
on interpreting the data sent by your keyboard according to cp1250 which
is what Windows sends.
set fileencodings=ucs-bom,utf-8,cp1250
This defines the heuristics used by gvim to detect a file's charset when
opening it for editing. They are used from left to right:
- ucs-bom: if the file starts with a BOM, the Unicode encoding
defined by the BOM is used. (The BOM is a prefix of two to four bytes,
different for all Unicode encodings, and the BOM for each encoding is
illegal in all others.) If there is no BOM, then:
- utf-8: if the data in the file is valid for UTF-8, then Vim does
the equivalent of setlocal fileencoding=utf-8. If it is no valid
UTF-8, then:
- cp1250: This is an 8-bit encoding, it cannot return an invalid
signal. If none of the above is recognised, then the file is read
according to the Windows-1250 charset (Windows proprietary charset for
Central Europe).
I don't know how exactly each one influences the behaviour (the second
line was suggested to me on this mailing list some time ago), but I
had to comment out the first one and the problem has gone. I had
numerous problems with utf-8 before adding those lines. For some
strange reason Windows doesn't recognize vim as being able to do
Unicode, so it communicates in an old cp1250 fashion with it. cp1250
is indeed set in the regional settings, but I didn't manage to explain
to Windows that vim can handle utf-8 as well.
OK, I only left
set fileencodings=ucs-bom,utf-8,cp1250
in my _vimrc and the menues seem to be OK now.
With these settings, you will be able to edit cp1250 files; but UTF-8
files containing codepoints not pesent in cp1250 will appear garbled
(Unicode caters for more than 10 million characters, cp1250 has only 256).
But now my second question: if I open some utf-8 file, it opens as
utf-8, which is OK. But when I open a new file and save it, it's saved
in cp1250 unless I type :set encoding=utf-8 before. So, after saying
set encoding=utf-8 the menu stays OK, but if I put that into my
_vimrc, the encoding in menu is broken. Any remedy for it?
Thanks a lot,
Mojca Miklavec
Since your Windows encoding is Windows-1250, I believe it would be more
prudent to write your menufile in cp1250 (and, I suppose, include
cp1250 in its name where there was utf-8).
See :help ++opt about how to set the file's charset manually when
reading or writing.
IMHO you may change 'encoding' after the menus have been set up. For
instance:
source $VIMRUNTIME/vimrc_example.vim
if termencoding ==
set termencoding =
Re: non-latin characters aren't displayed correctly in Windows menu
Mojca Miklavec wrote: On 6/13/06, A.J.Mechelynck wrote: Mojca Miklavec wrote: Courier_New has a larger variety of glyphs than most other fixed-width encodings. If Lucida_Console hasn't got the glyphs you need, I recommend Courier_New. No, Lucide_Console is OK (and Courier_New as well), but on high-resolution they are a bit too thin for my eyes. ... (but that's not the main point of my question anyway) set encoding=utf-8 termencoding=cp1250 This tells gvim to represent the data internally using UTF-8 but to go on interpreting the data sent by your keyboard according to cp1250 which is what Windows sends. set fileencodings=ucs-bom,utf-8,cp1250 This defines the heuristics used by gvim to detect a file's charset when opening it for editing. They are used from left to right: - ucs-bom: if the file starts with a BOM, the Unicode encoding defined by the BOM is used. (The BOM is a prefix of two to four bytes, different for all Unicode encodings, and the BOM for each encoding is illegal in all others.) If there is no BOM, then: - utf-8: if the data in the file is valid for UTF-8, then Vim does the equivalent of setlocal fileencoding=utf-8. If it is no valid UTF-8, then: - cp1250: This is an 8-bit encoding, it cannot return an invalid signal. If none of the above is recognised, then the file is read according to the Windows-1250 charset (Windows proprietary charset for Central Europe). Thanks! OK, I only left set fileencodings=ucs-bom,utf-8,cp1250 in my _vimrc and the menues seem to be OK now. With these settings, you will be able to edit cp1250 files; but UTF-8 files containing codepoints not pesent in cp1250 will appear garbled (Unicode caters for more than 10 million characters, cp1250 has only 256). UTF-8 files are opened properly (which is perfect). It's only for the new files that I have to set encoding manually. That would be pretty cumbersome, since I try to switch to UTF-8 completely anyway and I would most probably keep forgetting to set the proper encoding each time. Since your Windows encoding is Windows-1250, I believe it would be more prudent to write your menufile in cp1250 (and, I suppose, include cp1250 in its name where there was utf-8). I prepared all three variants (cp1250, utf-8 and latin2), but it really doesn't seem to matter to gvim which one I use. If one works, all of them do and if one doesn't, all three fail. See :help ++opt about how to set the file's charset manually when reading or writing. Thank you. That one should be part of the save as and open menu as well. (I try to learn new commands every now and then or when I need to do some repetitive task, but being able to use some standard Windows shortcuts or menu really simplifies everything for non-experts.) It would be great if the user could set the encoding visually. No matter how strange it may sound - I often use Mozilla to check the encoding of arbitrary files. It might be due to the lack of my skills with vim (and due to too many encodings everywhere: one for keyboard, one for file, one for font, one for menu, one for mapping, one for cummunication with operating system ...). (Working with encodings in Mozilla is much more intuitive, but I may not compare it with Vim, which is much more powerful if you know how to use it.) IMHO you may change 'encoding' after the menus have been set up. For instance: source $VIMRUNTIME/vimrc_example.vim if termencoding == set termencoding = encoding endif set encoding=utf-8 fileencodings=ucs-bom,utf-8,cp1250 I believe the following will set 'fileencoding' to cp1250 by default for newly-created files, and set a BOM at the start of newly-created Unicode files setglobal fileencoding=cp1250 bomb The example above didn't work. I was able to say if termencoding == set termencoding = encoding endif (without the '' sign) in which case I get a message window each time when I start gvim, saying that encoding=cp1250. Oops! typo! I should have written let termencoding = encoding But set encoding=utf-8 after that influences the menu and the menu is wrong again, no matter where in _vimrc I set it. Also, the following recipe described under :help menutrans doesn't work: :source $VIMRUNTIME/delmenu.vim :source c:\soft\vim\vim70\lang\menu_sl_si.cp1250.vim :source $VIMRUNTIME/menu.vim I get the English menu instead, even if I additionally set :language messages sl_SI.cp1250 which you suggested (Menus are set up, in your Windows default language and encoding which are SL.cp1250, by the vimrc_example. Do you want to say that vimrc_example sets anything relevat? Because I can't find any font/language/encoding-related commands there. It issues commands (see below) which result in loading the menu structure; the :language messages setting defines in which language the menus will be loaded. If the language looked for isn't found, then English (United States) is used as
Re: non-latin characters aren't displayed correctly in Windows menu
On 6/14/06, A.J.Mechelynck wrote: In $VIMRUNTIME/vimrc_example.vim for Vim 7.0, last change 2002 Sep 19, at line 56 filetype plugin indent on ... Thanks for all the hints. Using the extensive explanation and after playing with all possible combinations: the most crutial part was setting set termencoding=cp1250 encoding=utf-8 before both syntax on filetype plugin indent on in vimrc_example. I'll play with it and inspect a bit more (looks pretty weired and I would never have come to the clue that termencoding might be important for syntax and highlighting issues) to understand it better. Again: thanks a lot for taking your time, it works like a charm now! Mojca
Re: non-latin characters aren't displayed correctly in Windows menu
Mojca Miklavec wrote: On 6/14/06, A.J.Mechelynck wrote: In $VIMRUNTIME/vimrc_example.vim for Vim 7.0, last change 2002 Sep 19, at line 56 filetype plugin indent on ... Thanks for all the hints. Using the extensive explanation and after playing with all possible combinations: the most crutial part was setting set termencoding=cp1250 encoding=utf-8 before both syntax on filetype plugin indent on in vimrc_example. I'll play with it and inspect a bit more (looks pretty weired and I would never have come to the clue that termencoding might be important for syntax and highlighting issues) to understand it better. Again: thanks a lot for taking your time, it works like a charm now! Mojca Well, so much the better if it woks like a charm. Once you understand how best to do it, maybe you should write a tip and publish it at vim-online http://vim.sourceforge.net/. Don't forget to mention pitfalls and alternative methods (such as using 'langmenu'). I don't recommend modifying the vimrc_example (except by copying it to another pathfilename) because: a) if a bug is found in it, a patchlevel upgrade may modify it silently; b) when you upgrade to a Vim version higher than 7.0, the whole $VIMRUNTIME structure will be re-created from scratch under a different name (such as $VIM/vim71 instead of $VIM/vim70). In both cases, any modifications you made into the vimrc_example will be lost (maybe after you've forgotten about them). What I recommend is sourcing the vimrc_example from your own vimrc, with most customizations after it, and maybe a few lines before it if they influence what the vimrc_example does. For instance, I prefer English menus and messages, and I have noticed that to get them even on multilingual Vim, and that on both Windows and Linux, I need the following before sourcing the vimrc_example: if has(unix) language messages C else language messages en endif Anything which doesn't have to go before the vimrc_example should go after it, and many settings must go after it to override it. For instance: filetype indent off overrides the indent part of :filetype plugin indent on set nobackup writebackupoverrides set backup in the vimrc_example etc. 'termencoding' defines how Vim interprets keyboard input in all cases. In console vim but not in the GUI it also tells Vim how the console interface will interpret screen output. In the GUI, screen output is defined by 'encoding' and by the glyphs in the font(s) defined by 'guifont' and, when used (which is seldom necessary), 'guifontset' and 'guifontwide'. Your further experiments may help us all understand how Vim writes to the menu bar. I would expect that when 'termencoding' is empty Vim should fall back to the value of 'encoding' (see :help 'termencoding then scroll to /^\s*When empty/). That would be right until or unless the 'encoding' setting is altered (normally by the vimrc). I notice that the help says 'termencoding' is forcibly set to UTF-8 at GUI startup in the GTK+2 gvim. I wonder why. Does the GTK+2 keyboard interface always use UTF-8? Best regards, Tony.
Re: non-latin characters aren't displayed correctly in Windows menu
Mojca Miklavec wrote:
On 6/12/06, A.J.Mechelynck wrote:
Mojca Miklavec wrote:
Hello,
I translated the menus for gvim and wanted to use them under windows,
but the non-latin characters simply don't show in the menu (other
characters are shown instead: squares š and ž and some other character
for č).
All other programs under Windows work fine, the encoding should also
be OK (If I rename the Russian file to menu_sl_si.utf-8.vim, I only
get question marks in menu, but I saw other cyrillic programs running
on the same computer without any problems as well).
I suspect that Vim might be asking Windows for Western European font
for the menu. I'm not sure about it, but in any case it's a bit weird.
Is there any remedy for it?
Any hints would be appreciated,
Mojca Miklavec
You must make sure that you have:
- an 'encoding' which includes the non-Latin characters you want to use
- (in console Vim) a terminal code page which includes them
What is that? And console vim if you mean the one without GUI menus
is not always there (I have it one one computer, but not on the
other).
Console Vim is the one without GUI. On Windows a different executable
must be used, usually named vim.exe (or possibly something starting in
vim as in vimd.exe for a debug version etc.); it displays in a Dos
Box. On Unix a single executable can be used as a GUI (through X11) or
as a console version (displaying on /dev/tty or on xterm, konsole, etc.)
depending on how it is invoked.
- (in gvim) a 'guifont' which has the glyphs for them.
I have that (the default font is OK), but in Menu another font is used
(some default Windows font which is the same in all the applications).
Check 'encoding' and 'guifont' by means of
:verbose set encoding? guifont?
encoding=utf-8
Last set from D:\soft\_vimrc
guifont=
[some parts deleted]
'guifont' empty means some default system font, such as Fixed. IMHO it
is not he prettiest but if you're satisfied with it you may stay with it.
3. check that there is a line scriptencoding utf-8 near the top of the
file, before the first non-Latin character
It is.
4. save the file with :saveas ++enc=utf-8
~/.vim/lang/menu_sl_si.utf-8.vim (on Unix) or :saveas ++enc=utf-8
~/vimfiles/lang/menu_sl_si.utf-8.vim (on Windows) (without hte quotes
in either case).
I didn't use that, but it seems to be in UTF-8.
I guess that the problem is not related to the vim itself, but rather
to the Windows GUI, so the file might work properly on Mac or Linux.
I've seen no options to modify the menu font (I did, but I had the
impression that that works only in Linux).
There is a -menufont {font} command-line option, but from where it is
described in the help I fear it is only applicable to X11 (all
Unix/Linux versions and possibly some MacOsX versions). AFAIK the font
used for menus in Windows is common to all applications and thus outside
the reach of gvim. Try the following:
gvim -N -u NONE
:language messages
:set encoding?
This will tell you which language and charset settings are passed by
Windows to gvim before any vimrc or other script changes them. If they
are not compatible with what you want to display in the menus, you will
need to change the country-specific settings before starting Windows.
I think the settings are to be found in the international keyboard
widget, probably configured under Control Panel - Keyboard, then choose
a language and keyboard layout from the system tray; but I might be wrong.
See
http://vim.sourceforge.net/tips/tip.php?tip_id=246
:help ++opt
:help :scriptencoding
Can please anybody check the attached file? It seems to me that it IS
in utf-8, but I might be wrong. I'm only sending the Tools menu.
Instead of Skoči nazaj I see Skoèi nazaj and instead of Pokaži
napake I get Poka[]i napake, where [] stands for a box. Can anyone
get the content right?
Thanks a lot,
Mojca
I don't know the language it's in, but, when I view it in gvim, it sets
'fileencoding' to utf-8 and the contents look like some mixture of
English and some Slavic-family language using Latin alphabet with
diacritics (mostly c, s and z, all three with caron).
If you _see_ it as something else when you _edit_ it, then either you
don't have 'encoding' and 'fileencoding' set both to utf-8, or your
'guifont' lacks some necessary glyphs. If you see it OK when _editing_
but not in the _menus_ then see above about the Windows settings.
Best regards,
Tony.
