Hi Bruce: After scraping you get into the domain of actually envisioning how a screen reader might work to try and emulate some features programmatically.
I was talking to a few folks over the weekend about this. Here is a good response from one of the fellows who worked on NVDA Project. It gives the 5k overview of what they work with, notice he mentions scraping and parsing what is what you are doing in your app. Many of the other technical items and structures are handled by WE for the most part and some exposed in the WE Object Model.My stumble in my last attempt at an "External Script" involved low-level handling of keyboard input and the fact the WE model didn't get at the dom code I needed to make the Forms Designer in vb.net read while native, managed uia actually, did expose the objects in the designer. Anyway, I thought you might get a kick out of seeing the 5k overview of how the NVDA Screen Reader is viewed by a Programmer and how he envisions how some of the technicals fit together from his viewpoint. Note I don't use NVDA, never worked on it and don't agree with everything he says - he is just another programmer but the keywords and technologies might be of interest as you develop your programming / scripting skillsets. Hi Rick, About the purpose of a screen reader versus self-voicing applications: in my opinion, it is better to create something that can provide access to majority of appications, with the rest coming in as external modules conforming to a set of screen reader's API. Just as human readers should be flexible in handling various document types and conventions, a screen reader should be quite flexible to handle most of the controls found in many applications and emerging standards. There's a saying in programming circles I like: do it right the first time, then worry about corner cases later. I believe we can apply this concept to screen reader technologies: first get it to read what can be read, then worry about corner cases (inaccessible controls, apps, standards, etc.) later as time goes by. Having worked on the NVDA project like some of us on this list, I can confidently say that developing a screen reader is a major undertaking. At a nutshell, a screen reader is a program which interprets elements of the user environment in ways which allows blind and visually impaired computer users to use the environment effectively. This process, involving various modules (see below) could take months to years to perfect, and the developer of a screen reader must be a visionary willing to learn today's standards and provide a way to interpret these standards (ARIA, User Interface Automation, threads, etc.) for the benefit of his or her screen reader users. Taking NVDA as an example, a screen reader is composed of various modules which must (and I emphasize the word "must") work together to gather, interpret and present user interface elements. These modules include, in order of importance, remote code communicator and injector (used to retrieve useful UI elements from programs), screen scraper and parser, accessibility API handlers and responders (MSAA, UIA, JAB, etc.), input processors (keyboard, braille display, touchscreen, speech recognition, etc.) and output presenters (speech, braille, motion, etc.). In addition, a screen reader, being a privileged software, must be aware of mechanisms provided by the operating system for security, memory access, network connectivity, input hooks, performance (UI presentation and IPC (inter-process communication)) and so on, as well as being sensitive to changes in UI environments across OS versions and among different accessibilityAPI versions, implementation and compliance by the OS and third-party applications (and in case of websites, compliance to web standards and guidelines on accessibility to ease the work of the screen reader in presenting a site's content). As for coding such a software, many would use C++ or another compiled language for speed, although some components might be coded in a scripting language like Python for ease of maintenance and extensions at the cost of performance. For a screen reader like NVDA which uses C++ for low-level communications subsystem (NVDA Helper) and Python for the high-level code, it is important to decide which component must be run at full speed and which components should easily was extendable by developers and users. Although many users would want a screen reader to be a performance champion in UI presentation, many would point out that due to technology in use (not just the screen reading algorithms, which most are parsing and communications, but also the framework in use and for compatibility) would prevent the screen reader from taking advantage of latest capabilities of operating systems, hardware features and programming models and interfaces. I mention NVDA a lot here because its source code has a lot of potential to be studied in academia in developing better screen reading algorithms and to provide a historical (and active) example of how a screen reader is developed. Hope this helps. Cheers, Joseph Now, Bruce, I hve delt with most of the above technical at one time or another so understand what he is trying to say but I don't agree with some of it for a "Targeted" project limited to asingle product like Internet Explorer or say Visual Studio or Sql Management Studio. I don't believe in ReInventing the wheel so if I ever do something along these lines I would either work in an existing scripting language or create a very, very limited project either self voicing or using one of the existing screen readers api capabilities. Anyway, thought youd get a kick out of seeing the stuff the screen reader guys deal with on a daily basis and might get some ideas for future research if interested. I don't use NVDA as I find WindowEyes usually adiquit and use narrator if needed to supplement it where narrator will help. Good Hunting: Rick USA
