Template Version: @(#)sac_nextcase %I% %G% SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Virtual Keyboard Input Method
1.2. Name of Document Author/Supplier:
Author: Naoyuki Ishimura
1.3 Date of This Document:
21 January, 2009
4. Technical Description
OVERVIEW
As an extension to the current IIIMF keyboard layout emulation feature [1],
this project proposes to add GUI-based virtual keyboard on our desktop.
Virtual keyboard is a graphical representation of computer keyboard on
GUI desktop that can be summoned on as needed based on the current input
mode and can be used to input characters by clicking on the graphically
represented keys on the virtual keyboard.
The virtual keyboard that will be delivered by this project will not only
supply an additional way of inputting characters (by clicking on the keys on
the virtual keyboard), but also allow individual users to create a new
keyboard layout as a new emulation based input mode or customize the current
and existing keyboard layout as needed so that they can re-map keys, assign
a different character (or a set of characters) on a key, or do both on
the current keyboard layout and keys.
Once customized layout is saved with your specified name, it can be used
persistently. User can create as many as custom layout they wish; and they
will be usable for both at the virtual keyboard and, if chosen so, also
with the physical keyboard.
This feature has been asked by US government and EMEA customers specifically
for Unicode/UTF-8 locales to aid multiscript input and also to allow user
customization of the keyboard layouts. As we do not add any new features to
CDE, in the same manner and context, we do not plan to back-port this to
legacy codeset locales at this point unless we have a requirement with
some specific revenue number attached. We also consider that the existing
input modes and methods quite good enough and sufficient for such legacy
codeset locales.
Once approved by PSARC, we will also go through xDesign/HCI review so that
the resulting GUI will meet Sun standard and be intuitive as much as possible.
TECHNICAL DETAILS
(1) User Interface:
The virtual keyboard [9] which can be invoked from terminals or GUI desktop |
menu has keyboard window and control panel as its main user interface |
components: |
- Keyboard window:
This will be the main interface that users will use to see and interact
with to input characters and also to customize with.
Users can have and use multiple keyboard windows on their desktop.
In terms of how to group and organize such multiple keyboard windows,
there will be three different styles that one can choose from: stand-alone,
internal frame, and tab styles. (For more details, please see the sub-section
on the control panel at below.)
Each keyboard window will show a keyboard layout with usual character and
modifier keys based on the XKB symbol and geometry data of the current
input mode. Users can select a specific XKB symbol, an XKB geometry, or
both, such as "US/English" symbol, "Generic 101" geometry, and so on.
[10, 11, 12]
A character key will act like a push button and a modifier key will act
like a toggle button as specified in the XKB data. Each key will show the
currently effective character(s) based on the combined state of the key
with the modifier keys. As an example, if Caps Lock key is in effect,
the key for 'a' will show 'A'. (Please see "Appendix A: Visual effect with
modifier keys" at pages 4 and 5 of [10].)
The character(s) shown on the keys will also be customizable.
When the keyboard window's context menu is brought up by clicking on |
the right mouse button at the window, one can also choose to show or hide |
the control panel, select symbol data, edit keys, and do some other |
convenience operations directly. [11] |
- Control panel:
The control panel can be used to control keyboard window style and mode
among other things as shown in [12].
The style defines how keyboard windows are organized. As noted at the above
sub-section, the following three different styles will be supported:
-- Stand-alone frame style: Each keyboard will be displayed as a separate
top-level window.
-- Internal frame style: One or more keyboards will be displayed within
a top-level container window as internal frame windows.
-- Tab frame style: Each keyboard will be displayed in a tab pane of
a top-level container window.
Please see "Appendix B: Multiple view with different styles" at pages 5 and
6 at [10] for a couple of concept designs.
The mode defines whether keyboard windows are in input mode or in
design mode (i.e., edit or customization mode). When the current mode is in
input mode, if one clicks on a key, associated character(s) will go to
an application that had the input focus the last. While it is in design
mode, users can edit the keyboard layout and keys for a customization.
When "New Keyboard" button is pressed along with selections on symbol and
geometry, a new keyboard layout (and thus a new input mode) can be created
and saved.
There will be also a check box for "Synchronize with physical keyboard"
which can be used to instruct the current keyboard layout emulation input
mode to follow the customized keyboard layout and keys on the virtual
keyboard window.
As noted earlier, some of the operations on the control panel can also be
done via the context menu of the keyboard window.
(2) Keyboard customization and keyboard data handling:
When it is in design mode, the keyboard window will be used as the keyboard
layout editor. For each key, users can specify a character or a set of
characters by using an associated text field or by doing copy or swap |
operations via drag-and-drop. (Using the same drag-and-drop, the copy |
operation is done between two keyboards and the swap is done within |
a single keyboard.) |
When a user creates his own keyboard layout (symbol) data, it will be saved in
the user's own directory ($HOME/.iiim/vkb/) as an XML file. It will have user
specified layout name and will be sent to the input method server. (Similar
effect will also take place if one customizes an existing keyboard layout.)
At that time, input method switcher [2] will be notified to update its
language list to include the newly created keyboard layout so that the user
can select it from the input method switcher menu afterward.
To make the customization persistent, it is one of the responsibilities of
the IIIM start-up client program (/usr/lib/iiim/iiimx-settings-init) which
will send the custom keyboard layout data, if any, to the input method server
at the start of a new GUI desktop session.
(3) Input mechanism on character(s):
The character(s) from a virtual keyboard is passed as input method committed
text to other applications. What that means is that the virtual keyboard
generated text data can be any Unicode character or text and will not be
restricted to XKeyEvent data of X11.
[Virtual keyboard]
|
|
character(s)
|
| +--character(s)--> [GUI client with last input focus]
V /
[Input Method Server]
(4) Pre-defined keyboard symbol and geometry data:
Virtual keyboard has pre-defined geometry and symbol data set that came from
XKB geometry and symbol data [1]. As of today, they are stored separately in
IIIM private directory and thus, during run-time, it does not need X11 XKB.
The symbol data are shared with IIIMF keyboard layout emulation functionality.
As previously discussed during the review of [1], once we have a set of
interfaces established between IM and Xlib/X servers (especially Xorg and
Xnewt) on the XKB operations, we will transparently replace this redundant
component by using the future interfaces between the IM and the X servers
so that there will be a single set of XKB data in the system for the above
mentioned system components.
(5) Candidates for future enhancement:
The first implementation from this project will not include the following.
They are possible candidates for future enhancement and we will come back
again with new PSARC case(s) as needed:
- System-wide availability of custom layout:
Currently, the customized layout data is only for and per user.
System-wide customization will be considered when there is a formal
requirement.
- Function keys and other features of physical keyboards:
The virtual keyboard input method at this point aims only at character
input. Emulating all functionalities of physical keyboard is not currently
considered. Thus function key support and such will be considered when
it is formally required.
INTERFACE STABILITY
There is no significantly notable interfaces imported. The project exports
the following interfaces.
Interfaces Stability Note
---------- --------- ----
/usr/bin/iiim-keyboard Committed Virtual keyboard itself [9]
/usr/lib/iiim/VKB.jar Project Private Virtual keyboard jar file
/usr/lib/iiim/libiiimvkb_jni.so Project Private Virtual keyboard lib file
/etc/iiim/geometry.vkz Project Private Virtual keyboard private
data file
/usr/share/gnome/help/iiim-keyboard/* GUI help file
Project Private
$HOME/.iiim/vkb/* Project Private Customized data
RELEASE BINDING
The project team asks for Micro/Patch release binding.
REFERENCES
[1] PSARC 2007/028 EMEA input methods with keyboard layout emulation
[2] PSARC 2005/525 IIIMF upgrade to revision 12
[3] Internet/Intranet Input Method Architecture:
http://www.openi18n.org/subgroups/im/iiimf/whitepaper/whitepaper.html
The same HTML file will be available at the case directory:
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/iiimf-whitepaper.html
[4] Internet-Intranet Input Method Protocol (IIIMP) specification:
http://www.openi18n.org/subgroups/im/iiimf/protocol/spec.html
The same HTML file will be available at the case directory:
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/iiimp-spec.html
[5] ASARC 1999/276 /usr/lib/im directory for platform independent Input Method
Systems
[6] ASARC 1998/515 Multi Script Input Support in UTF-8 locales
[7] ASARC 1998/422 IIIMP - Internet/Intranet Input Method Protocol
[8] ASARC 1997/359 Multilingual Input Method Server for X/Java/Corba
[9] iiim-keyboard(1) man page:
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/iiim-keyboard.1
[10] High-level Virtual Keyboard UI/Module design (v0.2):
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/VKB_design_02.pdf
[11] Draft keyboard window screen snapshots:
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/Keyboard.png
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/Input-mode-context-menu.png
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/Edit-mode-context-menu.png
[12] Draft control panel window screen dump:
http://www.opensolaris.org/os/community/arc/caselog/2009/041/materials/ControlPanel.png
6. Resources and Schedule
6.4. Steering Committee requested information
6.4.1. Consolidation C-team Name:
G11N
6.5. ARC review type: FastTrack
6.6. ARC Exposure: open
