On Sat, Mar 23, 2024 at 07:41:45PM +0800, Sadeep Madurange wrote: > Initially, LANG was unset and LC_CTYPE="C". The character encoding was > US-ASCII. I changed these variables (i.e., LANG, LC_CTYPE and locale > settings) to en_US.UTF-8. Then the ? changed to ?. So, looks like you > are on to something. I will check this with OpenBSD community as well. > > In Xdefaults, I have set XTerm*utf-8 setting to true as well.
Your problem is that these settings are not consistent (and you still
have this problem, because the "solution" proposed by Sirius is
incorrect--even if it appears to have solved your issue). By having
LANG unset, you've told your shell (and therefore everything started
by it) to use ASCII, but you've explicitly told xterm to use Unicode.
That's wrong.
The TL;DR of this is:
1. You should NEVER need to set Mutt's charset explicitly. [*]
2. Your shell, Mutt, and X should all inherit what they need from your
LANG environment variable, assuming it is set properly for your
system and environment (it definitely isn't in your case).
3. Setting Mutt's charset may appear to "work" but it's not the
correct solution, because your shell and terminal settings are
still inconsistent. You'll have trouble with other things later if
you don't fix this.
________________________________________________________________
[*] Except in extremely rare and completely esoteric cases that apply
only to experts... and by now should really apply to no one.
The unfortunately lengthy details:
----------------------------------
Displaying characters properly is actually tricky business on modern
computers, because of the legacy methods by which we tried to
accommodate different languages, and the (relatively) recent advent of
Unicode to unify that mess. All of the following must be set
consistently: Your shell, your terminal program (or your operating
system's console), your font, all of your application programs, and
when appropriate, the X window system. If any of these are not
consistently set, you can, and eventually WILL, have trouble. Most
modern systems have the concept of a default locale, which is
typically set for you at install time, and which every process you
start inherits, unless you configure your user environment
differently.
Fortunately, there is a very simple mechanism by which this happens,
which is the LANG environment variable. There are additional
ancillary environment variables which start with "LC_*" but you
usually should not have to set any of these, because they inherit
their value from LANG if they are not explicitly set. When you run
the locale command, values enclosed in quotes are inherited from LANG,
and values NOT enclosed in quotes have been set explicitly:
$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=C
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
Here you can see that I explicitly set LC_COLLATE=C. The rest are
inherited from LANG. Typically most users will want to leave all of
the LC_* variables unset, and inherit from LANG.
I haven't tried a *BSD in a really long while, but if it doesn't ask
you for your default locale during install, or if you made a mistake
setting it up, then you should add the settings manually to your login
shell environment. If you're using UTF-8 (which you should be--by now
every modern OS uses it by default), the value of LANG should reflect
that. Pretty much no one should be using ASCII anymore (i.e. LANG
should NEVER be unset). The most portable way to do that would be to
include the following in BOTH .profile and .kshrc (or whatever file
you've set ENV to):
LANG=en_US.UTF-8
export LANG
[See the Invocation section of the ksh man page for exact details of
which files you should put this in, but in general it's the ones I
said.]
Of course if you are not using English, change en_US to whatever your
default language is, but you'll want to retain the ".UTF-8" portion.
That *should* be sufficient to handle everything... however, there may
be additional places you'll need to add it for your X applications,
depending on exactly what OpenBSD does to initialize users' X
sessions. In general, the X startup stuff is supposed to make sure
that it sources the user's environment so that you don't need to
figure out which of the 17 different files you actually need to put
this stuff in... but over the years most vendors have bastardized how
X sessions start up, so you may have to look up or trace out how your
system does it to make sure everything works correctly. So you might
have to also add it to .xinitrc or .xsession or similar. Or, if it
does not already, you could simply have your X init thingy source your
.profile or .kshrc or whatever--and probably should.
As for this:
On Sun, Mar 24, 2024 at 08:11:29AM +0800, Sadeep Madurange wrote:
> On 2024-03-23 12:52:40, Sirius Rayner-Karlsson via Mutt-users wrote:
> > It may be that you just need to pop in the "set charset="utf-8"" in
> > your mutt config and you are good to go.
As I said, you should NEVER need to do this... if your environment is
set up properly, Mutt will CORRECTLY inherit its charset and language
settings from that. The only time you should ever need to mess with
Mutt's idea of your charset is if you have some very esoteric edge
case use that requires things to be different from your default
language. Since the proliferation of Unicode, this is now EXTREMELY
rare, and if you don't know what you're doing, you almost certainly
shouldn't be doing this.
> Thanks for sharing your config. You're right. I needed to add "set
> charset=utf-8" to muttrc and set LC_CTYPE. The problem was I had set the
> latter in my .kshrc when I needed to set it in .xsession. It's all good
> now.
Your settings may "work" but they are not correct. You might not have
trouble with Mutt displaying ??? where it shouldn't anymore, but if
you don't take care to set up your locale correctly, you will
eventually continue to experience related issues elsewhere, inside and
outside Mutt.
You should really remove that charset setting in Mutt, to confirm that
your other settings are now correct. You don't need it, and you
shouldn't use it. It's just more likely to cause problems later.
--
Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
signature.asc
Description: PGP signature
