> This is not true. In fact, if the file contains "señor" instead of > "ññ", Vim does resort to Latin1. This said, Vim's failure here does > sound like a bug. But I would like to hear from Bram first.
Well spotted, Yongwei. So there is something more subtle about this bug, and I believe it is this: Vim doesn't recognise a file as invalid utf8 if, when you get to the first invalid sequence, there are less bytes in the file than would be required to read a valid sequence beginning with the unicode leader character read. I.e. if the last byte in the file is C2-DF, or one of the last two bytes is E0-EF or one of the last three bytes is F0-F4. As these sequences would take 2, 3 and 4 bytes respectively to read a valid character, and there are not that many bytes in the file, Vim finishes its analysis thinking 'valid' as it hasn't read a 'whole invalid character'. :-) This is a very specific scenario, though. Question for Dervish: was it just with this small test case that you noticed the problem, or does it occur elsewhere?! > As I stated in another message, it looks to me when Vim reads from > stdin, the content is already interpreted in termencoding. I have not > yet found other results. This isn't true. I can set termencoding to e.g. big5 but Vim will read the input as latin1 or utf8 and thus display question marks as the ñ cannot be represented. On the other hand, with tenc=utf8 I can set fencs to big5 on the commandline (vim --cmd 'set fencs=big5' -) and have the <f1> interpreted and displayed as Chinese. So I don't know about your Vim, but mine behaves exactly the same way whether something is pumped into stdin or opened as a regular file from disk, using fencs. I wonder if this behaviour could be platform-specific or depend on which libraries are available/compiled in. Because we both seem to have solutions, but neither of them works for the other person. Hmmmm. Ben. Send instant messages to your online friends http://au.messenger.yahoo.com --~--~---------~--~----~------------~-------~--~----~ You received this message from the "vim_dev" maillist. For more information, visit http://www.vim.org/maillist.php -~----------~----~----~----~------~----~------~--~---
