Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Andrew Fish via groups.io]

> On Mar 22, 2021, at 11:31 AM, Nate DeSimone  
> wrote:
> 
> Hi Andrew,
> 
> I tested VT_UTF8 on the macOs Terminal software and I can confirm that 
> VT_UTF8 renders nicely. See the attached screenshot.
> 

Nate,

Yes it is close to VT_UTF8, but not quite the same. I was looking back through 
the driver and I’d forgotten how similar it really is. 

Symbols used in table below
===
  ESC = 0x1B
  CSI = 0x9B
  DEL = 0x7f
  ^   = CTRL

+=+==+===+==+==+=+
| | EFI  | EFI 1.10  |  |  | |
| | Scan |   |  VT100+  |  | |
|   KEY   | Code |  PC ANSI  |  VTUTF8  |   VT100  | xterm-color | 
+=+==+===+==+==+=+
| NULL| 0x00 |   |  |  | |
| UP  | 0x01 | ESC [ A   | ESC [ A  | ESC [ A  | ESC [ A |
| DOWN| 0x02 | ESC [ B   | ESC [ B  | ESC [ B  | ESC [ B |
| RIGHT   | 0x03 | ESC [ C   | ESC [ C  | ESC [ C  | ESC [ C |
| LEFT| 0x04 | ESC [ D   | ESC [ D  | ESC [ D  | ESC [ D |
| HOME| 0x05 | ESC [ H   | ESC h| ESC [ H  | ESC [ H |
| |  |   |  |  | ^A  |
| END | 0x06 | ESC [ F   | ESC k| ESC [ K  | ESC [ F |
| INSERT  | 0x07 | ESC [ @   | ESC +| ESC [ @  | ^E  |
| |  | ESC [ L   |  | ESC [ L  | |
| DELETE  | 0x08 | ESC [ X   | ESC -| ESC [ P  | 0x7f|
| PG UP   | 0x09 | ESC [ I   | ESC ?| ESC [ V  | ESC [ 5 ~   |
| |  |   |  | ESC [ ?  | ^P  |
| PG DOWN | 0x0A | ESC [ G   | ESC /| ESC [ U  | ESC [ 6 ~   |
| |  |   |  | ESC [ /  | ^N  |
| F1  | 0x0B | ESC [ M   | ESC 1| ESC O P  | ESC O P |
| F2  | 0x0C | ESC [ N   | ESC 2| ESC O Q  | ESC O Q |
| F3  | 0x0D | ESC [ O   | ESC 3| ESC O w  | ESC O R |
| F4  | 0x0E | ESC [ P   | ESC 4| ESC O x  | ESC O S |
| F5  | 0x0F | ESC [ Q   | ESC 5| ESC O t  | ESC [ 1 5 ~ |
| F6  | 0x10 | ESC [ R   | ESC 6| ESC O u  | ESC [ 1 7 ~ |
| F7  | 0x11 | ESC [ S   | ESC 7| ESC O q  | ESC [ 1 8 ~ |
| F8  | 0x12 | ESC [ T   | ESC 8| ESC O r  | ESC [ 1 9 ~ |
| F9  | 0x13 | ESC [ U   | ESC 9| ESC O p  | ESC [ 2 0 ~ |
| F10 | 0x14 | ESC [ V   | ESC 0| ESC O M  | ESC [ 2 1 ~ |
| Escape  | 0x17 | ESC   | ESC  | ESC  | ESC |
+=+==+===+==+==+=+
| F11 | 0x16 |   |  |  | ESC [ 2 3 ~ |
| F12 | 0x16 |   |  |  | ESC [ 2 4 ~ |
+=+==+===+==+==+=+


I’m wondering it we could pick the default terminal type based on the 
toolchain? 

Thanks,

Andrew Fish

> Thanks,
> Nate
> 
> On 3/17/21, 9:02 AM, "Andrew Fish"  > wrote:
> 
>If we are mentioning terminal types the default terminal type on a Mac is 
> xterm-256color. So that is going to be the default when people run OVMF on a 
> Mac. So it would be nice if we can add that. I can help out with anything 
> xterm-256color related. 
> 
>Thanks,
> 
>Andrew Fish
> 
>> On Mar 16, 2021, at 8:23 AM, Laszlo Ersek  wrote:
>> 
>> Hi Nate,
>> 
>> (adding Leif and Ard)
>> 
>> On 03/13/21 03:52, Desimone, Nathaniel L wrote:
>>> I've created a new wiki page for this task with all the information I
>>> have gathered thus far. I've done some more experimentation and found
>>> that there are several newer terminal emulators that don't support
>>> DEC Special Graphics so I've reduced the number of modes where DEC
>>> Special Graphics should be preferred. Laszlo, if you could take a
>>> look at the terminal type matrix I created that would be very
>>> helpful.
>>> 
>>> https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements
>> 
>> (
>> 
>> My background:
>> 
>> I settled on plain (non-UTF-8) xterm around 1998, and have been using it
>> ever since. Whenever something was off, I always tried to hammer the
>> application into conformance with my particular xterm setup, rather than
>> the other way around. I also have some quirky terminal settings -- for
>> me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
>> "delete" generates keycode 119, and there's no "rubout". I still don't
>> use UTF-8 (I use latin2).
>> 
>> )
>> 
>> * Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
>> setting (which means VT-100). Using that setting, I see the following
>> kind of "ASCII approximation" for box drawing:
>> 
>> /--\
>> |Boot Manager
>>   |
>> \--/
>> 
>> I'm really ha

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Laszlo Ersek]

On 03/17/21 00:25, Desimone, Nathaniel L wrote:

> The UEFI spec doesn't read on this at all, even though it describes
> VT100 and VT100+ as separate modes... it doesn't say how they differ.
> I agree with you that it seems reasonable for VT100 to keep character
> output to strict ASCII only... that way the "+" in VT100+ actually
> means something. I've updated the wiki accordingly.
> 
> I'd advocate for the default to be switched to VT_UTF8. I really
> don't think you will run into many terminal emulators that don't
> implement UTF-8 anymore, XTerm included. Those who want pure ASCII
> output can switch to VT100.

Hmmm, OK. As long as I can permanently switch my domains, one by one, to
VT100, I guess I'll be fine.

Thanks!
Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72975): https://edk2.groups.io/g/devel/message/72975
Mute This Topic: https://groups.io/mt/81273234/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Andrew Fish via groups.io]

If we are mentioning terminal types the default terminal type on a Mac is 
xterm-256color. So that is going to be the default when people run OVMF on a 
Mac. So it would be nice if we can add that. I can help out with anything 
xterm-256color related. 

Thanks,

Andrew Fish

> On Mar 16, 2021, at 8:23 AM, Laszlo Ersek  wrote:
> 
> Hi Nate,
> 
> (adding Leif and Ard)
> 
> On 03/13/21 03:52, Desimone, Nathaniel L wrote:
>> I've created a new wiki page for this task with all the information I
>> have gathered thus far. I've done some more experimentation and found
>> that there are several newer terminal emulators that don't support
>> DEC Special Graphics so I've reduced the number of modes where DEC
>> Special Graphics should be preferred. Laszlo, if you could take a
>> look at the terminal type matrix I created that would be very
>> helpful.
>> 
>> https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements
> 
> (
> 
> My background:
> 
> I settled on plain (non-UTF-8) xterm around 1998, and have been using it
> ever since. Whenever something was off, I always tried to hammer the
> application into conformance with my particular xterm setup, rather than
> the other way around. I also have some quirky terminal settings -- for
> me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
> "delete" generates keycode 119, and there's no "rubout". I still don't
> use UTF-8 (I use latin2).
> 
> )
> 
> * Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
>  setting (which means VT-100). Using that setting, I see the following
>  kind of "ASCII approximation" for box drawing:
> 
>  
> /--\
>  |Boot Manager
>   |
>  
> \--/
> 
>  I'm really happy with this, as I don't care much for nice-looking
>  boxes; instead I prefer portability.
> 
>  (NB: this seems to disagree with your "Current Behavior (Which is
>  wrong)" line for VT100, as it suggests CP437. That's not what I'm
>  seeing with VT100.)
> 
>  TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
>  far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
>  the default, in 2015:
> 
>  http://mid.mail-archive.com/555458DB.3090602@redhat.com
>  
> http://mid.mail-archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sqf...@mail.gmail.com
> 
>  (I'd like to CC Roy, but I can't tell if he's now working for Linaro,
>  Cavium, HPE, Marvell, or another company.)
> 
> * Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
>  looks like this:
> 
>  
> ÚÄÄż
>  łBoot Manager
>   ł
>  
> ŔÄÄŮ
> 
>  Obviously I'd much prefer if I got the simple ASCII approximation here
>  as well.
> 
> * Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially* supposed
>  to use DEC Special Graphics, I can't tell.
> 
>  I know what my preferences are:
> 
>  - the current BackSpace and Delete mappings (which work fine for me
>with both VT100 and PC_ANSI, but *not* with TTY_TERM),
> 
>  - and the most primitive ASCII mapping (no special graphics, no UTF-8
>sequences, etc). I really like a super dumb terminal, where taking
>simple "ASCII screenshots" (and pasting them into plaintext emails!)
>is *trivial*.
> 
>  ... Looking at your "Expected Behavior" table, there is only one line
>  left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
>  TTY_TERM breaks my BackSpace / Delete settings :(
> 
> * In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
>  ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
>  rather than PC_ANSI, by default.
> 
> Thanks!
> Laszlo
> 
> 
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72971): https://edk2.groups.io/g/devel/message/72971
Mute This Topic: https://groups.io/mt/81273234/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Leif Lindholm]

+Roy (now at Marvell)

/
Leif (now at Qualcomm)

On Tue, Mar 16, 2021 at 16:23:45 +0100, Laszlo Ersek wrote:
> Hi Nate,
> 
> (adding Leif and Ard)
> 
> On 03/13/21 03:52, Desimone, Nathaniel L wrote:
> > I've created a new wiki page for this task with all the information I
> > have gathered thus far. I've done some more experimentation and found
> > that there are several newer terminal emulators that don't support
> > DEC Special Graphics so I've reduced the number of modes where DEC
> > Special Graphics should be preferred. Laszlo, if you could take a
> > look at the terminal type matrix I created that would be very
> > helpful.
> >
> > https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements
> 
> (
> 
> My background:
> 
> I settled on plain (non-UTF-8) xterm around 1998, and have been using it
> ever since. Whenever something was off, I always tried to hammer the
> application into conformance with my particular xterm setup, rather than
> the other way around. I also have some quirky terminal settings -- for
> me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
> "delete" generates keycode 119, and there's no "rubout". I still don't
> use UTF-8 (I use latin2).
> 
> )
> 
> * Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
>   setting (which means VT-100). Using that setting, I see the following
>   kind of "ASCII approximation" for box drawing:
> 
>   
> /--\
>   |Boot Manager   
>|
>   
> \--/
> 
>   I'm really happy with this, as I don't care much for nice-looking
>   boxes; instead I prefer portability.
> 
>   (NB: this seems to disagree with your "Current Behavior (Which is
>   wrong)" line for VT100, as it suggests CP437. That's not what I'm
>   seeing with VT100.)
> 
>   TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
>   far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
>   the default, in 2015:
> 
>   http://mid.mail-archive.com/555458DB.3090602@redhat.com
>   
> http://mid.mail-archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sqf...@mail.gmail.com
> 
>   (I'd like to CC Roy, but I can't tell if he's now working for Linaro,
>   Cavium, HPE, Marvell, or another company.)
> 
> * Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
>   looks like this:
> 
>   
> ÚÄÄż
>   łBoot Manager   
>ł
>   
> ŔÄÄŮ
> 
>   Obviously I'd much prefer if I got the simple ASCII approximation here
>   as well.
> 
> * Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially* supposed
>   to use DEC Special Graphics, I can't tell.
> 
>   I know what my preferences are:
> 
>   - the current BackSpace and Delete mappings (which work fine for me
> with both VT100 and PC_ANSI, but *not* with TTY_TERM),
> 
>   - and the most primitive ASCII mapping (no special graphics, no UTF-8
> sequences, etc). I really like a super dumb terminal, where taking
> simple "ASCII screenshots" (and pasting them into plaintext emails!)
> is *trivial*.
> 
>   ... Looking at your "Expected Behavior" table, there is only one line
>   left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
>   TTY_TERM breaks my BackSpace / Delete settings :(
> 
> * In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
>   ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
>   rather than PC_ANSI, by default.
> 
> Thanks!
> Laszlo
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72965): https://edk2.groups.io/g/devel/message/72965
Mute This Topic: https://groups.io/mt/81273234/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Nate DeSimone]

Hi Laszlo,

> -Original Message-
> From: disc...@edk2.groups.io  On Behalf Of
> Laszlo Ersek
> Sent: Tuesday, March 16, 2021 8:24 AM
> To: Desimone, Nathaniel L 
> Cc: disc...@edk2.groups.io; cadenkli...@gmail.com; edk2-devel-groups-io
> ; Ard Biesheuvel (TianoCore)
> ; Leif Lindholm (Nuvia address)
> 
> Subject: Re: [edk2-discuss] Google Summer of Code Interested Student
> 
> Hi Nate,
> 
> (adding Leif and Ard)
> 
> On 03/13/21 03:52, Desimone, Nathaniel L wrote:
> > I've created a new wiki page for this task with all the information I
> > have gathered thus far. I've done some more experimentation and found
> > that there are several newer terminal emulators that don't support DEC
> > Special Graphics so I've reduced the number of modes where DEC Special
> > Graphics should be preferred. Laszlo, if you could take a look at the
> > terminal type matrix I created that would be very helpful.
> >
> > https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-d
> > river-improvements
> 
> (
> 
> My background:
> 
> I settled on plain (non-UTF-8) xterm around 1998, and have been using it
> ever since. Whenever something was off, I always tried to hammer the
> application into conformance with my particular xterm setup, rather than the
> other way around. I also have some quirky terminal settings -- for me,
> "backspace" generates ^H / keycode 22 (stty sets erase to ^H), "delete"
> generates keycode 119, and there's no "rubout". I still don't use UTF-8 (I use
> latin2).
> 
> )
> 
> * Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
>   setting (which means VT-100). Using that setting, I see the following
>   kind of "ASCII approximation" for box drawing:
> 
>   
> /--\
>   |Boot Manager   
>|
>   
> \--/
> 
>   I'm really happy with this, as I don't care much for nice-looking
>   boxes; instead I prefer portability.
> 
>   (NB: this seems to disagree with your "Current Behavior (Which is
>   wrong)" line for VT100, as it suggests CP437. That's not what I'm
>   seeing with VT100.)

I went back and looked at this is more detail, and I missed the following 
critical detail:

if (TerminalDevice->TerminalType != TerminalTypePcAnsi) {
  GraphicChar = AsciiChar;
}

Yes you are totally right! I've adjusted the table to reflect this behavior.

> 
>   TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
>   far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
>   the default, in 2015:
> 
>   http://mid.mail-archive.com/555458DB.3090602@redhat.com
>   http://mid.mail-
> archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sQf8w
> w...@mail.gmail.com
> 
>   (I'd like to CC Roy, but I can't tell if he's now working for Linaro,
>   Cavium, HPE, Marvell, or another company.)
> 
> * Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
>   looks like this:
> 
> 
> ÚÄÄÄ
> ÄÄÄż
>   łBoot Manager   
>ł
> 
> ŔÄÄÄ
> ÄÄÄŮ
> 
>   Obviously I'd much prefer if I got the simple ASCII approximation here
>   as well.
> 
> * Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially*
> supposed
>   to use DEC Special Graphics, I can't tell.

The UEFI spec doesn't read on this at all, even though it describes VT100 and 
VT100+ as separate modes... it doesn't say how they differ. I agree with you 
that it seems reasonable for VT100 to keep character output to strict ASCII 
only... that way the "+" in VT100+ actually means something. I've updated the 
wiki accordingly.

I'd advocate for the default to be switched to VT_UTF8. I really don't think 
you will run into many terminal emulators that don't implement UTF-8 anymore, 
XTerm included. Those who want pure ASCII output can switch to VT100.

> 
>   I know what my preferences are:
> 
>   - the current BackSpace and Delete mappings (which work fine for me
> with both VT100 and PC_ANSI, but *not* with TTY_TERM),
> 
>   - and the most primitive ASCII mapping (no special graphics, no UTF-8
> sequences, etc). I really like a super dumb terminal, where taking
> simple "ASCII screenshots" (and pasting them into plaintext emails!)
> is *trivial*.
> 
>   ... Looking at your "Expected Behavior" table, there is only one line
>   left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
>   TTY_TERM breaks my BackSpace / Delete settings :(
> 
> * In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
>   ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
>   rather than PC_ANSI, by default.
> 
> Thanks!
> Laszlo
> 
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Laszlo Ersek]

Hi Nate,

(adding Leif and Ard)

On 03/13/21 03:52, Desimone, Nathaniel L wrote:
> I've created a new wiki page for this task with all the information I
> have gathered thus far. I've done some more experimentation and found
> that there are several newer terminal emulators that don't support
> DEC Special Graphics so I've reduced the number of modes where DEC
> Special Graphics should be preferred. Laszlo, if you could take a
> look at the terminal type matrix I created that would be very
> helpful.
>
> https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements

(

My background:

I settled on plain (non-UTF-8) xterm around 1998, and have been using it
ever since. Whenever something was off, I always tried to hammer the
application into conformance with my particular xterm setup, rather than
the other way around. I also have some quirky terminal settings -- for
me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
"delete" generates keycode 119, and there's no "rubout". I still don't
use UTF-8 (I use latin2).

)

* Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
  setting (which means VT-100). Using that setting, I see the following
  kind of "ASCII approximation" for box drawing:

  
/--\
  |Boot Manager 
 |
  
\--/

  I'm really happy with this, as I don't care much for nice-looking
  boxes; instead I prefer portability.

  (NB: this seems to disagree with your "Current Behavior (Which is
  wrong)" line for VT100, as it suggests CP437. That's not what I'm
  seeing with VT100.)

  TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
  far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
  the default, in 2015:

  http://mid.mail-archive.com/555458DB.3090602@redhat.com
  
http://mid.mail-archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sqf...@mail.gmail.com

  (I'd like to CC Roy, but I can't tell if he's now working for Linaro,
  Cavium, HPE, Marvell, or another company.)

* Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
  looks like this:

  
ÚÄÄż
  łBoot Manager 
 ł
  
ŔÄÄŮ

  Obviously I'd much prefer if I got the simple ASCII approximation here
  as well.

* Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially* supposed
  to use DEC Special Graphics, I can't tell.

  I know what my preferences are:

  - the current BackSpace and Delete mappings (which work fine for me
with both VT100 and PC_ANSI, but *not* with TTY_TERM),

  - and the most primitive ASCII mapping (no special graphics, no UTF-8
sequences, etc). I really like a super dumb terminal, where taking
simple "ASCII screenshots" (and pasting them into plaintext emails!)
is *trivial*.

  ... Looking at your "Expected Behavior" table, there is only one line
  left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
  TTY_TERM breaks my BackSpace / Delete settings :(

* In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
  ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
  rather than PC_ANSI, by default.

Thanks!
Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72915): https://edk2.groups.io/g/devel/message/72915
Mute This Topic: https://groups.io/mt/81273234/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Nate DeSimone]

I've created a new wiki page for this task with all the information I have 
gathered thus far. I've done some more experimentation and found that there are 
several newer terminal emulators that don't support DEC Special Graphics so 
I've reduced the number of modes where DEC Special Graphics should be 
preferred. Laszlo, if you could take a look at the terminal type matrix I 
created that would be very helpful.

https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements

Thanks,
Nate

> -Original Message-
> From: disc...@edk2.groups.io  On Behalf Of Nate
> DeSimone
> Sent: Thursday, March 11, 2021 9:46 PM
> To: cadenkli...@gmail.com
> Cc: Laszlo Ersek ; disc...@edk2.groups.io;
> devel@edk2.groups.io
> Subject: Re: [edk2-discuss] Google Summer of Code Interested Student
> 
> Hi Caden,
> 
> Great to meet you and welcome to the TianoCore project! Glad you hear you
> are interested! Sorry it has taken me a while to get back to you, researching
> the topics you are interested in ended up being somewhat involved 😊.
> 
> I went back and did some investigation of current state of the terminal
> driver, and some of the work has already been done. However, there are
> some things missing and some odd bugs that need attention. To give you a
> little more detail, the Terminal driver is located at
> https://github.com/tianocore/edk2/tree/master/MdeModulePkg/Universal
> /Console/TerminalDxe
> 
> The most prevalent use case for the terminal driver is to display the BIOS
> setup menu on headless server systems using a PC style serial port
> connected to a laptop via null modem. This allows administrators to adjust
> BIOS settings on rack mounted systems without needing to connect a
> monitor and keyboard.
> 
> Historically, the BIOS setup menu would be rendered using the IBM PC VGA
> text mode, which encoded text using code page 437 (CP437). This was
> important for box-drawing characters, such as ┌ , ─ , and ┐ , which VGA text
> mode encodes as 0xDA, 0xC4, and 0xBF respectively. However, most
> terminal emulators assume text to be encoded in UTF-8. Unicode defines
> these box drawing characters as 0x250C, 0x2500, and 0x2510. In UTF-8
> encoding, these characters translate into 3 byte sequences of (0xE2, 0x94,
> 0x8C), (0xE2, 0x94, 0x80), and (0xE2, 0x94, 0x90) respectively. The VGA
> encodings of these box characters will end up generating errors if one
> attempts to decode them as strict UTF-8, though most terminals assume that
> the intended characters to be drawn are Ú, Ä, and ¿, which have the Unicode
> character codes 0xDA, 0xC4, and 0xBF. The end result is the BIOS setup menu
> looks like this:
> 
> ÚÄÄÄ
> ÄÄÄ¿
> ³   Device Manager
>  ³
> ÀÄÄÄ
> ÄÄÄÙ
> 
>Devices List  List all the Driver
>  > Driver Health Manager Health instances to
>  > RAM Disk Configurationmanage
>  > OVMF Platform Configuration
>  > iSCSI Configuration
>  > Network Device List
> 
> 
>Press ESC to exit.
> 
> 
> ÚÄÄÄ
> ÄÄÄ¿
> ³ 
>  ³
> ³ ^v=Move Highlight   =Select Entry  Esc=Exit  
>  ³
> ÀÄÄÄ
> ÄÄÄÙ
> 
> Instead of like this:
> 
> ┌─
> ─┐
> │   Device Manager
>  │
> └─
> ─┘
> 
>Devices List  List all the Driver
>  > Driver Health Manager Health instances to
>  > RAM Disk Configurationmanage
>  > OVMF Platform Configuration
>  > iSCSI Configuration
>  > Network Device List
> 
> 
>Press ESC to exit.
> 
> 
> ┌─
> ─┐
> │ 
>  │
> │ ^v=Move Highlight   =Select Entry  Esc=Exit  
>  │
> └─
> ─┘
> 
> The terminal driver has fully supported both the legacy CP437 encoding and
> the UTF-8 encoding for more than 10 years now. Which mode to use is part
> of the configuration settings given to the terminal driver at start up.
> However, those configuration settings need to come from somewhere. For
> example, OVMF has the following page in its setup menu for configuring the
> serial terminal:
> 
> ┌

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Nate DeSimone]

Hi Laszlo,

> -Original Message-
> From: disc...@edk2.groups.io  On Behalf Of
> Laszlo Ersek
> Sent: Friday, March 12, 2021 10:51 AM
> To: Desimone, Nathaniel L ;
> cadenkli...@gmail.com
> Cc: disc...@edk2.groups.io; devel@edk2.groups.io
> Subject: Re: [edk2-discuss] Google Summer of Code Interested Student
> 
> As far as I can remember, the terminal type has always been hard-coded in
> OVMF like this -- I believe it's not a regression (old or recent). I guess I
> haven't seen a good PlatformBootManagerLib example on this topic.

Yeah specifically on the topic of properly setting the terminal type, I don't 
think I've seen a completely correct implementation of PlatformBootManagerLib 
either. Agreed it is probably not a regression, but I still think it would be a 
great contribution to get this sorted out properly.

> 
> Thanks
> Laszlo
> 
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72738): https://edk2.groups.io/g/devel/message/72738
Mute This Topic: https://groups.io/mt/81273234/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Laszlo Ersek]

On 03/12/21 06:45, Desimone, Nathaniel L wrote:

> Now, here is the second bug. That BIOS setup menu page that OVMF has
> for configuring the serial port has a field for setting the terminal
> type. But, changing the value in that field doesn't actually change
> the configuration data that is sent to the terminal driver. So the
> terminal driver always ends up using PC_ANSI mode even if the user
> changes that setting. This isn’t a bug in the terminal driver really,
> it’s a bug in OVMF's setup menu implementation. But it does create
> the appearance of a problem in the terminal driver and should be
> fixed as part of this GSoC project. This should be fixed in both he
> OVMF implementation and the MinPlatform implementation.

It's a shortcoming of OVMF's PlatformBootManagerLib.

A solution would be nice where, if a (non-volatile) terminal type
setting existed, that would take effect, but if no such setting existed,
then we'd still automatically add the serial port(s) -- with some
default terminal type -- to the console I/O variables.

ArmVirtQemu's PlatformBootManagerLib works somewhat differently (see the
build-time feature test macro TTY_TERMINAL). I'm not up-to-date on
whether that PlatformBootManagerLib instance handles the Setup TUI-based
terminal type setting correctly.

As far as I can remember, the terminal type has always been hard-coded
in OVMF like this -- I believe it's not a regression (old or recent). I
guess I haven't seen a good PlatformBootManagerLib example on this topic.

Thanks
Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#72721): https://edk2.groups.io/g/devel/message/72721
Mute This Topic: https://groups.io/mt/81273234/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

[Nate DeSimone]

Hi Caden,

Great to meet you and welcome to the TianoCore project! Glad you hear you are 
interested! Sorry it has taken me a while to get back to you, researching the 
topics you are interested in ended up being somewhat involved 😊.

I went back and did some investigation of current state of the terminal driver, 
and some of the work has already been done. However, there are some things 
missing and some odd bugs that need attention. To give you a little more 
detail, the Terminal driver is located at 
https://github.com/tianocore/edk2/tree/master/MdeModulePkg/Universal/Console/TerminalDxe

The most prevalent use case for the terminal driver is to display the BIOS 
setup menu on headless server systems using a PC style serial port connected to 
a laptop via null modem. This allows administrators to adjust BIOS settings on 
rack mounted systems without needing to connect a monitor and keyboard.

Historically, the BIOS setup menu would be rendered using the IBM PC VGA text 
mode, which encoded text using code page 437 (CP437). This was important for 
box-drawing characters, such as ┌ , ─ , and ┐ , which VGA text mode encodes as 
0xDA, 0xC4, and 0xBF respectively. However, most terminal emulators assume text 
to be encoded in UTF-8. Unicode defines these box drawing characters as 0x250C, 
0x2500, and 0x2510. In UTF-8 encoding, these characters translate into 3 byte 
sequences of (0xE2, 0x94, 0x8C), (0xE2, 0x94, 0x80), and (0xE2, 0x94, 0x90) 
respectively. The VGA encodings of these box characters will end up generating 
errors if one attempts to decode them as strict UTF-8, though most terminals 
assume that the intended characters to be drawn are Ú, Ä, and ¿, which have the 
Unicode character codes 0xDA, 0xC4, and 0xBF. The end result is the BIOS setup 
menu looks like this:

ÚÄÄ¿
³   Device Manager ³
ÀÄÄÙ

   Devices List  List all the Driver
 > Driver Health Manager Health instances to
 > RAM Disk Configurationmanage
 > OVMF Platform Configuration
 > iSCSI Configuration
 > Network Device List


   Press ESC to exit.


ÚÄÄ¿
³  ³
³ ^v=Move Highlight   =Select Entry  Esc=Exit   ³
ÀÄÄÙ

Instead of like this:

┌──┐
│   Device Manager │
└──┘

   Devices List  List all the Driver
 > Driver Health Manager Health instances to
 > RAM Disk Configurationmanage
 > OVMF Platform Configuration
 > iSCSI Configuration
 > Network Device List


   Press ESC to exit.


┌──┐
│  │
│ ^v=Move Highlight   =Select Entry  Esc=Exit   │
└──┘

The terminal driver has fully supported both the legacy CP437 encoding and the 
UTF-8 encoding for more than 10 years now. Which mode to use is part of the 
configuration settings given to the terminal driver at start up. However, those 
configuration settings need to come from somewhere. For example, OVMF has the 
following page in its setup menu for configuring the serial terminal:

┌──┐
│ Set COM Attributes   │
└──┘

   Set COM Baud Rate  <115200>   Set COM Baud Rate
   Set COM Data Bits  <8>
   Set COM Parity 
   Set COM Stop Bits  
   Set COM Terminal Type  
   Set COM Flow Control   

   Commit Changes and Exit
   Discard Changes and Exit


┌──┐
│ F9=Reset to Defaults  F10=Save   │
│ ^v=Move Highlight   =Select Entry  Esc=Exit   │
└──┘

The default terminal type is PC_ANSI, which uses CP437. In this day and age 
that is probably not the right default anymore, though one could argue whether 
PC_ANS