The following is a proposal about including LSR in GNOME desktop releases. It is a proposal about offering users and developers a different option in terms of an extensible assistive technology. It is in no way an attempt to supplant Orca as the default GNOME screen reader and magnifier. Nor is it intended to undermine the progress the community has made in terms of accessibility. We'd like to discuss our work openly and honestly with the GNOME community. I believe this is possible without inadvertently starting World War III. :)
In this post, I plan to tackle the boilerplate "new module requirements" up front, and then ease into the more critical questions near the end. In my comparison of Orca and LSR, I will attempt to stick to familiar ground: technical differences. I'd like to leave questions and comments about the user experience to end-users. I apologize up front for any inaccuracies, and invite corrections. == Purpose The Linux Screen Reader (http://live.gnome.org/LSR) project is an open source effort to develop an extensible assistive technology (AT) for the GNOME desktop environment. The goal of the project is to create a reusable development platform for building alternative and supplemental user interfaces in support of people with diverse disabilities. The primary use of the LSR platform is to give people with visual impairments access to the GNOME desktop and its business applications (e.g. Firefox, OpenOffice, Eclipse) using speech, Braille, and screen magnification. The extensions packaged with the LSR core are intended to meet this end. However, LSR's rich support for extensions can be used for a variety of other purposes, such as: * Supporting novel input and output methods, such as joysticks, web cams, game pads, audio icons, and environmental sounds. * Creating interfaces for other users, such as people with mobility impairments, people with cognitive impairments, children learning to use a screen reader, and so forth. * Supplementing the graphical GNOME desktop with useful audio feedback for sighted users, such as text-to-speech reporting of events in a monitored window. == Target LSR is being proposed for the desktop module set. Dependencies LSR relies on modules already included in GNOME releases. One possible new dependency is desktop-file-utils from freedesktop.org (http://www.freedesktop.org/wiki/Software/desktop-file-utils). We are uncertain if this package is already included with GNOME. If adding it is a problem, we can easily stop depending on it and use a manual method to install of our lsr.desktop file instead. == Resource usage LSR is already hosted in GNOME's subversion repository. More than five tarball releases have been posted on GNOME's FTP server over the past year. Future releases will follow suit. We are currently tracking LSR bugs in a GNOME bugzilla module. We make extensive use of irc.gimp.net for developer discussions in #lsr. Adoption Ubuntu has been packaging LSR in its universe repository since Edgy. The SpeakUp Modified accessibility repository (http://speakupmodified.org) has been packaging LSR for Fedora Core since version 5. OpenSuse has recently added LSR packages. GNOME-ness and community Internationalization All human-readable strings are marked for translation. Our autotools process is configured to build and install translations for strings in the code. Eight translations have been started by the GNOME i18n team to date. Documentation We have created extensive documentation about the LSR project on the Live! wiki and GNOME website including: * A getting started guide for users (http://live.gnome.org/LSR/GettingStarted) * The beginnings of a user guide to be turned into Docbook documentation viewable with Yelp (http://live.gnome.org/LSR/UserGuide) * A tutorial for creating LSR script extensions (http://live.gnome.org/LSR/ScriptDevelopers/GaimPerkTutorial) * API documentation generated using epydoc (http://www.gnome.org/~parente/lsr/epydoc/index.html) * Screen reader speech UI specification (http://www.gnome.org/~parente/lsr/ui/index.html) * LSR architecture specification (http://www.gnome.org/~parente/lsr/workbook/index.html) * Screencasts and presentations (http://live.gnome.org/LSR#pandp) == Look and feel The LSR settings dialog and other GUI dialogs use gtk to maintain consistent look and feel with the rest of the GNOME desktop. LSR interfaces with gnome-speech to support text-to-speech synthesis (in addition to other APIs). Preferences The front-end for configuring preferences in LSR is abstracted from the back-end used to serialize data. At present, the front-end allows configuration through the LSR settings dialog and the command line. The back-end persists the data in Python pickle files in the user's home directory. We would like to add a gconf back-end to LSR to better integrate with the GNOME desktop. Changing our working code simply hasn't been top priority. == Accessibility All dialogs in LSR are accessible. For instance, LSR can read its own interface to the user to a user with a visual impairment. Other assistive technologies such as on-screen keyboards should be able to work with LSR dialogs too. == Key Questions = How do Orca and LSR differ technically? The following are some major differences in the designs of LSR and Orca. = Extensions Orca has the concept of scripts which may be keyed to a particular application or toolkit. The mapping from script to application is one-to-one, though a script may inherit methods and attributes from other scripts. For instance, most Orca scripts for applications derive from a default script providing basic screen reader functionality with speech, Braille, and magnification. When a specific script isn't defined for an application, the default script is used. New scripts are placed in the site-packages/orca/scripts Python package, and the mapping from script to application name is defined in the settings.py file or inferred automatically from the name of the script. Orca also supports the addition of speech factories supporting speech APIs such as gnome-speech and Speech Dispatcher. Modules containing these factories are placed in the site-packages/orca directory, and should be configured for use by editing the settings.py file or chosen using the Orca preferences dialog. The Orca default script; the application scripts; the interfaces for speech, BrlTTY, and GNOME magnifier; and the Orca settings dialog constitute the Orca user interface. LSR has four extension types: * scripts: Respond to application and user events by producing output * devices: Enable input and output with available hardware * choosers: Support lengthy dialogue with the user * monitors: Display the flow of events to aid developers Extensions are packaged into profiles which define a user experience. Both the profiles and extensions are managed by a registrar which the user can access through the command line and through the LSR settings dialog (an extension itself). The registrar can also be controlled programmatically by extensions which may choose to load and unload other extensions at runtime. Extensions may live anywhere on disk, and may be registered for system wide use by a super user or local use by any user. A key feature of LSR is that there is no default user interface, only what a profile constructs from the available extensions. For example, the initial configuration of the "user" profile consists of scripts that respond to accessibility events using speech and Braille, devices supporting speech output and BrlTTY, and choosers for the settings dialog and the scripter loader/unloader dialog. Together, these extensions create a screen reader experience. Other profiles, however, can disregard some or all of our screen reader extensions, and reuse the LSR core for other purposes. For instance, we've constructed a "reminder" profile for people with aphasia containing a speech output device and a script that parses text in a user selected text area containing a user's daily todo list. Running LSR under this profile provides audible reminders of upcoming items in the list, which the user may continue to manage in their favorite editor as plain text. The script defines settings such as how far in advance reminder should be given, how reminders should be announced, what should happen if an item is overdue, and so on. (http://live.gnome.org/LSR/Extensions) For more on LSR's extensibility, see http://live.gnome.org/LSR/FrequentlyAskedQuestions#extensible = Customization Orca uses a settings.py file to hold its set of global settings. The available settings are pre-determined at design time, but their values can be changed by editing this file or using the Orca settings dialog. LSR supports settings per extension. An extension author may define a new setting object with a name, type, description, and other critical values (e.g. min and max). The settings dialog may then use this information to automatically generate an accessible interface for configuring the option. Any setting may be marked for persistence. For instance, our screen reader script for Gaim defines a boolean option named "Read background messages?" This option appears in the LSR settings dialog as an accessible checkbox. As another example, our IBM speech device extension has an option for baseline pitch. Since this setting is an integer numeric in a finite range, it appears as a slider in the settings dialog for the user to configure. The existence of the extension system provides customization in a different sense. Functionality may be added, removed, or changed by adding, removing, or exchanging extensions. If a particular user needs more descriptive labels on the toolbar buttons in Firefox, for instance, one might add a new script to his profile providing additional information when the buttons are read by the other scripts in LSR. This solution would require no changes to the existing Firefox script, and could be made available to just this one user. = Scripting Environment and Adapters Orca scripts consist of a mixture of calls directly to AT-SPI methods, helper functions to perform common tasks (e.g. Braille output), and calls to a base class script to perform default actions. LSR provides a managed scripting environment with its own API capable of, but not limited to, the following: * querying and manipulating accessible objects on the desktop * managing other extensions * registering global hotkeys * binding actions to arbitrary input device gestures * sending and styling output for a device * modifying settings with notifications for setting observers * modifying event flow * reloading scripts on the fly * executing code in other scripts without naming them explicitly (weak dependencies) * registering event handlers for signals from the focused control only, the foreground application, a background application, or any combination of these A key characteristic of LSR scripts is that they are devoid AT-SPI code. Scripts call methods in the LSR scripting API only. The scripting API, in turn, calls methods on adapter objects wrapping the platform accessibility API. Which adapter is used to fulfill a request depends on two factors: 1. If the adapter can handle the type of object being queried 2. If the adapter can handle the platform accessibility API in use The first requirement is checked by assertions defined in each adapter class. For instance, our TableInfoAdapter class indicates that it will only work with AT-SPI objects that provide a table interface. Likewise, our DefaultInfoAdapter class states that it will work with any AT-SPI object not handled by a more specialized adapter. The second requirement is determined at runtime by dynamically loading adapters targeted at the particular accessibility API on the system. At present, we only have adapters for AT-SPI. But additional adapters could be defined for accessibility APIs on other platforms (e.g. IAccessible2 on Windows) or future accessibility APIs not yet defined. The key benefit of this design is that the LSR scripts are isolated from changes to the accessibility API. If a new API appears or LSR is ported to another platform, all scripts will remain largely intact. = GNOME doesn't ship two of any kind of software. Why should LSR be a special case? GNOME packages both Dasher (http://svn.gnome.org/viewcvs/dasher/) and GOK (http://svn.gnome.org/viewcvs/gok/), two applications termed on-screen keyboards. These projects appear dissimilar even to non-users because of the radical differences in their visual UIs. The differences in interaction styles between Orca and LSR are not as apparent to the casual observer because most output is auditory or Braille, and input is done via a keyboard. Nevertheless, we are certain a screen reader user could report such differences. = So how do LSR and Orca differ in terms of the user experience? The technical differences mentioned above lead to differences in the screen reader user interfaces of Orca and LSR. For instance, the settings dialog in LSR is more dynamic because script writers can define options on a per script basis. The available settings will change based on what extensions the user has loaded. The ability to layer multiple scripts on an application can also lead to different experiences. For example, a LSR user might use the script chooser dialog to load a script that reads Python source code more intelligently in the active text editor program. Which text editor is in use does not matter, only that there is some text area in which the user is editing Python code. A more drastic difference will be seen if a user is operating under a profile that loads extensions other than the ones created for screen reading and magnification. Consider a user running under our example "reading" profile intended to aid people with reading disabilities. The scripts in this profile defines a hotkey which the user may press to see a picture related to the word selected in any application. The script pulls relevant pictures from Flickr and displays them in a chooser extension. The chooser has buttons for scrolling forward and backward through other similar pictures. (http://live.gnome.org/LSR/Extensions) Beyond mechanisms for logging, LSR also has a profile with additional tools suitable for developers looking to check the accessibility of their applications using a screen reader. Three monitor extensions show raw AT-SPI events, how the events are processed by scripts in LSR, and what output the scripts send to devices as a result. A developer script also defines additional hotkeys for determining what scripts are active, reloading scripts dynamically at runtime, muting output temporarily, and so on. Concerning the differences in the day-to-day experience of using LSR and Orca, users of both ATs are the most qualified to comment. I would invite them to do so, if their comments would have any impact on the acceptance or rejection of this proposal. Please speak up! In the meantime, we can state some of our LSR UI philosophies which have a noticeable effect on the screen reader interface: * Our default key bindings are based on kinesthetics rather than language-specific mnemonics. The key bindings are designed to remain in the same position and make sense in relation to one another no matter what keyboard or language is in use. * The default key bindings are not designed to mimic Windows screen readers. Again, we're all about position, not learned mnemonics. * The detail provided by the keyboard commands change with the context, but the underlying meaning stays the same. For example, our base "Where am I?" query is customized for reporting document context in Firefox (e.g. under the fifth heading, 50% through the page) versus standard GUI context (e.g. button in a panel in a frame) * The keyboard functions cycle through additional details when pressed repeatedly. For instance, our "read word" key binding says the word at the pointer on first press. If pressed again without an intervening command, it spell that word. If pressed a third time without intervention, it spells the word phonetically. A fourth press cycles back to the first function. This design allows us to group related functionality and avoid an explosion of keyboard bindings. * The capabilities of the available audio device are used to distinguish and aid selection of certain types of information. For instance, available voices are put to use distinguish content on the screen (e.g. menu item names), from context information (e.g. window title), from index information (e.g. item 1 of 10), from LSR error messages (e.g. failed to get information), from foreground/background information (e.g. voice to the left is from a background app, voice dead ahead is at the focus), etc.. Future development will enable users to define their own mappings from semantic meaning to output style, or even make everything sound the same if desired. = Why does LSR have to be part of GNOME? Why not a package in every distro? Like other applications, an assistive technology is not a one size fits all solution. Every user has slightly different needs and preferences, and might prefer one AT over another for certain tasks. But unlike installing a new text editor, getting a different AT up and running can be quite difficult without explicit desktop support: * Run your current AT * Fetch the new AT * Quit your current AT and run the new one to ensure it's working properly (problematic if new AT install went awry) * Set the new AT to run after login (difficult today in GNOME) * Log out * Log in (hopefully the session is configured properly so the new AT starts) Having the AT as part of GNOME would make this process easier. Switching among the available ATs would be tested and debugged before release, and the burden would not fall on each user or distro. = Isn't the name "Linux" Screen Reader a misnomer? You bet. LSR will run on any AT-SPI enabled platform. Moreover, some wily programmer may decide to port it to a platform in the future. Calling it Linux Screen Reader doesn't do anyone justice. If accepted as a GNOME module, we plan on remedying this problem by changing the project name. This should have minimal impact on the code base as the name is isolated in all but a few cases. A name along the lines of Open Screen Reader (open source, open extensibility, open community) is appealing, but still pigeonholes LSR as a screen reader, rather than a platform of sorts. We invite better suggestions. = Won't we need to do more work to make choosing a preferred AT easier or possible? We have already done most of the work by creating a new Accessibility panel for the GNOME Preferred Applications dialog. In this panel, a user may choose his or her preferred assistive technologies based on function. The necessary patches against GNOME 2.17 have already been reviewed and commented on by the proper module maintainers. Some members of the GNOME, Ubuntu, and Fedora communities have also contributed. Our patches need to be updated for GNOME 2.19 and committed. (http://live.gnome.org/GAP/ScratchPad/PreferredApplications) _______________________________________________ desktop-devel-list mailing list desktop-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/desktop-devel-list