Bug#320078: whiptail is unable to handle special characters not present in current locale

[Agustin Martin]

On Wed, Jul 27, 2005 at 09:11:21AM -0400, Thomas Dickey wrote:
> On Wed, Jul 27, 2005 at 02:50:06PM +0200, Micha Lenk wrote:
> > Is it even not possible to prevent the user from entering non-locale
> > characters? As a user I would prefer to get multiple other but legal
> > characters when I enter a non-locale character, wich I am able to delete
> > just after. In cases where I enter such a non-locale character by
> > accident I would at least be able to correct my mistake.
> 
> There are two cases: displaying data that it reads from some other location
> and data entered from the keyboard.  whiptail should of course distinguish
> between the two.  If it cannot at least let you delete the keyboard entry,
> that's a defect in whiptail.

Hi, Alastair

Regarding this I would like to point to a closely related problem that
should probably fit into this bugreport (please let me know if you prefer me
to open a new one).

When some utf-8 text is injected into whiptail, and seen from a single byte
environment, utf8 strings are shown as a concatenation of the single byte
chars, not funny, but really the expected behavior. However, when 
those single byte chars contain any char that is not displayable in the
local encoding, whiptail seems to be confused in menu items, and anything
after that char is skipped on display for that item, although if selected
returns the complete string as expected. Surprisingly in text the
non-displayable char is shown as a code, and further text is properly
displayed, what seems acceptable.

I am attaching a sample script to reproduce that, to be run in a latin1
environment. It contains bulgarian in cyrillic followed by a 7bit string,
"$the_utf8_string means bulgarian" and displays it both as text and as
a menu item. The problem is that the utf8 string contains \212, which is
not displayable for latin1, as fourth char, and some similar chars
afterwards, so menu is shown as (what is in square brackets is what should
be the first menu item, after non-displayable->something)

-----------------------------------------------------
Here goes the text with                                       
[бÑ<8A>лгаÑ<80>Ñ<81>ки means bulgarian]

бÑ
normal text
-----------------------------------------------------

and in menu item everything after \212 is ignored, and only Ð±Ñ is
shown in that item.

dialog works as expected, but on display replaces non-displayable chars by ~

Cheers,

-- 
Agustin

#!/bin/sh
# -*- coding: utf-8 -*-
: ${DIALOG=whiptail}

tempfile=`tempfile 2>/dev/null` || tempfile=/tmp/test$$
trap "rm -f $tempfile" 0 1 2 5 15

function analyze {
    retval=$1
    option=$2
    
    case $retval in
        0)
            echo "[`cat $tempfile`] chosen fror $option";;
        1)
            echo "Cancel pressed.";;
        255)
            if test -s $tempfile ; then
                cat $tempfile
            else
                echo "ESC pressed."
            fi
            ;;
    esac
}

text="български means bulgarian"

#--msgbox text height width

$DIALOG --msgbox "$text" 30 50

# --menu text height width menu-height [ tag item ] ...

$DIALOG --menu "Here goes the text with\n[$text]" 30 50 20 "$text" "" "normal 
text" ""  2> $tempfile

analyze $? --menubox