Goal for improving the eBook reader user interface:
I've been doing some exploratory programming with GTK and Sugar,
trying to improve the user interface of the eBook reader, and make
it useable in book mode with the gamepad.
+ Support the game keypads in eBook mode.
+ Low level game keypad support
Need to remap low level keyboard scan codes to Linux keyboard codes.
The setolpckeys.c program remaps the keys and gamepad buttons.
Currently it maps both gamepads to the numeric keypad keys (KEY_KP8,
etc),
which the X server and GDK translates to directional keys (GDK_Up, etc).
I tried to map them to buttons (BTN_A, etc), but the X server seems
to ignore keycodes in that range.
The xorg.conf file has a keycode mask that looked like it might help,
but I couldn't get it to work.
Need to have unique keycodes reported for each of the two gamepads,
which are not the same as any keyboard keys, without any predefined
meanings
like arrow keys have.
Need to define special purpose keycodes just for the OLPC gamepad,
instead of trying to reuse existing but not appropriate keycodes.
What is the process for defining new keycodes in <linux/input.h>?
Here's my strawman proposal for some new keycodes.
Use keys ("KEY_*") instead of buttons ("BTN_*"), since they
seem to work better.
The 0x1b* range seems to be unused in <linux/input.h>,
and it's between other groups of keycodes, so I'll
propose using that range for the OLPC.
The UP/DOWN/LEFT/RIGHT keys correspond to the directional
keypad.
#define KEY_XO_GAMEPAD_UP 0x1b0
#define KEY_XO_GAMEPAD_DOWN 0x1b1
#define KEY_XO_GAMEPAD_LEFT 0x1b2
#define KEY_XO_GAMEPAD_RIGHT 0x1b3
The NORTH/SOUTH/EAST/WEST keys correspond to the other
buttons. Those names are agnostic to the button labels,
which may change from the current Playstation buttons
(X/O/Triangle/Square). Can anyone suggest better names for
the four buttons on the right?
#define KEY_XO_GAMEPAD_NORTH 0x1b4
#define KEY_XO_GAMEPAD_SOUTH 0x1b5
#define KEY_XO_GAMEPAD_EAST 0x1b6
#define KEY_XO_GAMEPAD_WEST 0x1b7
While we're at it, we could define keycodes for the other
OLPC buttons and switches on the screen. I think there are
some other sensor switches that could generate keycodes,
like opening the screen, rotating it around, and putting it
into book mode, so I will make some guesses at names for
them, just to get the discussion rolling.
#define KEY_XO_SCREEN_ROTATE 0x1b8
#define KEY_XO_SCREEN_POWER 0x1b9
#define KEY_XO_SCREEN_OPEN 0x1ba
#define KEY_XO_SCREEN_CLOSE 0x1bb
#define KEY_XO_SCREEN_IN 0x1bc
#define KEY_XO_SCREEN_OUT 0x1bd
Is there an exhaustive list of all buttons and switches and
events on the OLPC? Are any more planned? Which ones should
be assigned keycodes?
Rewrote setolpckeys.c code in Python (just uses ioctl, but needs to know
keycodes).
Writing utilities like that in Python instead of C makes it easier to
reconfigure the keys on the OLPC without a C compiler.
+ High level action support.
GTK uses "Actions" to define the actions available in an
application, independent of the user interface used to invoke
them. Actions can be bound to user interface widgets and
keyboard accelerators, and they can hide, show, enable and
disable the corresponding parts of the interface. You can
subclass Action to define custom toolbar buttons and menu items.
We need to define a generic way of navigating and executing the
application's actions from the gamepad.
We can make Sugar specific actions that create the appropriately
styled and customized Sugar user interface widgets.
Actions can be used to support navigation and operation of the
toolbar components from the gamepad:
Actions have a list of their "proxy" components (toolbar
buttons, menu items, etc).
Actions know how to execute a callback function, so the user
interface components tell the action to activate, instead of
calling the function themselves.
The actions also know their labels, icons and tooltips.
Actions can be shown and hidden, and all their proxies show and
hide.
Actions can be made sensitive or not (disabled), and all their
proxies enable or disable.
Actions have methods to construct toolbar buttons and menu
items, which subclasses can override to customize the user
interface. The higher level GTK UIManager calls these methods in
actions to create the user interface, although you can still use
Actions without the UIManager, by creating the components
yourself.
There are three standard kinds of actions: Action, which creates
a normal button or menu item, ToggleAction, which creates a
toggle button (checkbox), and RadioAction, which creates a radio
button (multiple choice).
The GTK toolbars and menus support keyboard navigation of
sensitive buttons, as well as showing tooltips on sensitive
buttons, but it won't show a tooltip on unsensitive (disabled)
buttons, tabbing skips over disabled buttons, and it kicks you
out of buttons that get disabled while you're using them, by
moving the focus into the next button (whatever that happens to
be).
All inactive items should show tooltips telling WHY they are
inactive, and WHAT you have to do before using them.
Unfortunately you can't get a tooltip on an inactive item.
This needs to be fixed, so we can display helpful tooltips and
documentation on any item whether active or not.
Unfortunlatey you can't navigate to an inactive item. This
needs to be fixed, so the user can use the gamepad to navigate
to an inactive item, to find out what it is and why it's
inactive. (Also, so you can navigate the interface predicably
with muscle memory, because the number of presses required to
navigate doesn't change depending on the state of the
application.)
A problem with the current GTK keyboard navigation behavior of
not allowing inactive items to have the input focus, is that it
violates the principle of least astonishment, makes the
interface less predictable, and interferes with type-ahead:
If you're focused on the "back page" button, and select it
until you get to the first page, then it will become inactive
(because you can't go back from the first page), which
results in the input focus being kicked out of the "back
page" button and throwing you into the "next page" button.
Imagine how hard this "astonishing" behavior would be to use
if you were visually impared. It would not be very obvious
that you had "bounced off" the first page and suddenly were
moving forward. Other toolbars might arbitrarily relocate the
input focus to an even more confusing button, when the one
you're using gets disabled.
It would be much less astonishing if the input focus simply
remained in the "back page" button when you hit the first
page, and a tooltop popped up saying "You can't go to the
previous page, because you are at the first page."
I've made a simple API for changing the sensitivity of a
component, that takes a "reason" string (translated text) to
show to the user in the tooltip of a disabled control (after
the normal text). I've changed the code in the eBook reader
that disables actions to figure out the most important reason
(there might be several reasons to disable a control, but
usually one is most important). It passes that reason to the
API that disables the action, which currently just stores it
away in a dictionary. Now it needs to be hooked up so it
actually shows the tooltip with the reason when disabled.
GTK has an "AcceleratorList" class that keeps a list of keyboard
accelerators which invoke actions. The UIManager helps manage
the accelerators for you automatically.
I think we should use accelerators for normal keyboard
acceleration of application actions, but implement the gamepad
navigation stuff as a higher level framework that uses a more
semantically meaningful model. (Not just low level tab/backtab
focus navigation, or simple global keyboard accelerators, but
an actual framework specifically designed to support browsing
and executing arbitrary Actions via the game controller, and
providing feedback about the reasons controls are disabled.)
The pygtk library has some methods on action_group to add
actions "action_group.add_actions(action_list)", toggle
actions "action_group.add_toggle_actions(action_list)", and
radio actions "action_group.add_radio_actions(action_list,
cur_value, callback, *args)". These all take an action_list
that's a list of action specification tuples. That interface
is more brittle and less extensible than it should be, because
it takes tuples instead of dictionaries, and it only makes
stock GTK actions.
We need a more flexible API than add_actions and its ilk, which
supports custom Sugar actions, and uses dictionaries instead
of tuples, so we can easily pass in additional optional
parameters without changing the API.
I have taken a first cut at rewriting pygtk's ActionGroup's
add_actions, add_toggle_actions and add_radio_actions
functions in Python, and changing them to use custom
SugarAction, SugarToggleAction and SugarRadioAction classes.
But I still don't think the add_actions_esque interface is
flexible enough. It needs an easy way to add other custom
widgets (like the search text input field), and it should make
it easy to customize the user interface classes and configure
the objects by passing in additional optional parameters and
configuration dictionaries. (For example, the action
specification should be a dictionary that has an optional key
to specify the toolbar button and menu item classes, and it
should be possible to plug in different kinds of user
interface controls than toolbars and menus, like pie menus and
gamepad specific interfaces).
GTK has an "atk" module that interfaces to the accessibility toolkit.
I'm not clear on just what this does and how it can help us,
and I would love to hear from somebody who's familiar with it.
I think we can do what we need to support the gamepad without
using the atk, but maybe it could be helpful. But my
impression is that it's more geared towards screen readers, is
tied closely to the user interface layout (declaring that this
label is associated with that control, etc), and it looks like
it takes a lot of lines of code to use (unless you use some
kind of framework that supports it automatically).
GTK has a "UIManager" that knows how to build menu bars, menus,
toolbars and buttons from XML files, and tie together actions,
accelerators, menus and toolbars. The XML describes the
structure and layout of the user interface, but refers to the
actions by name to configure the buttons and menu items with
labels, icons, tooltips, accelerators, callbacks, and custom
component classes. There is nothing in the XML about what label,
icon, tooltip or widget class to use -- just an Action
name. It's up to the Action to figure out the visual appearance
and behavior of the gui. That's why it should be easier to use
custom Action classes.
I think we need to write our own UIManager and accessibility
framework that addresses our specific needs (like the
focus/tooltip/disable reason issues discussed above, custom
sugar controls, gamepad support, etc), because I don't think
the UIManager (plus the pygtk utilities that go along with it)
are flexible enough to support our needs. I think it would be
easy to implement it in Python, in a way that would be a lot
more flexible and easier to extend that the current C
implementation of GTK's GuiManager.
+ Add support for more advanced evince features.
Added support for the various properties and functions that evince
supports.
Need access to the table of contents of the pdf file.
+ Integrate Poppler renderer for efficiently rendering PDF documents.
Make a higher level library independent interface to evince and poppler.
Factor out evince and poppler dependencies so you only need to import
the library that you need to render the document.
The enumerated types that evince uses should be passed as strings instead,
so the interface is independent of evince, and you don't have to define all
the GObject enumerated type classes just to use the interface.
_______________________________________________
Devel mailing list
Devel@laptop.org
http://mailman.laptop.org/mailman/listinfo/devel
