Hi Ken, Just so you can think about this, almost any property or method of the accessible object can be spoken; you aren't restricted to just the ".text" or ".value" properties. Sometimes a given property of the accessible object is a string (such as .name) which can be spoken, or sometimes you have to test on the property (such as the .state.checked property, which returns true or false), and so you could then speak "checked" or "unchecked" inside of an IF statement. So you can turn anything which isn't a string or a number, into useful information which can be spoken, with only the smallest amount of programming. If you want to see what information you have to look at and work with, you can use the WE Event app, and set it to show you all of the MSAA info only, log it to a file, and then go execute some program. when you're done, go stop the logging, and you can look at all the MSAA info that program generated in the log, and this will give you some idea what you should look at in the app developer's manual, and what info you may have to work with in your efforts. Yes, I know having to learn to program for what you don't want to hear is counter-intuitive at first, but once you see the idea it goes faster. I suspect UIA won't be all that different for us; I don't know this, but I'd bet GW will find a way to give us some additional info, without having to completely throw out our programming efforts and experience so far. I think this is what Microsoft is doing, by mapping UIA info already into MSAA calls and structures, so it looks to us like we're still using MSAA. Thanks for the good wishes; I appreciate them. We're back to sleeping in the house normally, so that's a big step forward. Chip
_____ From: Scott, Ken [mailto:[email protected]] Sent: Monday, November 07, 2011 4:14 PM To: [email protected] Subject: RE: How to Limit Speaking of MSAA Information Chip, I am sorry to learn of the house flooding. I hope that the recovery goes well and quickly for you and your family. You are right to focus on the event handlers first. They are more important and can lead to ways of using MSAA controls if necessary. My problem as a non-developer with using MSAA was the whole concept of first blocking the Window-Eyes default speech for MSAA controls for certain events and then building from scratch a way of speaking a limited portion of the Window-Eyes MSAA speakable string for those events. My life, as a non-developer, would have been easier if there had been a way in Window-Eyes to identify and select the speakable parts of the MSAA control to use with a given application or there was an app like the GW Micro Window Manager app where MSAA controls' speakable components were listed and could be selected for speaking where they existed. I wonder if UI automation will present similar challenges. I know such an app is beyond my ability to create and there are probably factors that make such a generic app extremely difficult if not impossible to build. Again Thanks for your help and best of luck with your future classes. Ken Scott From: Chip Orange [mailto:[email protected]] Sent: Monday, November 07, 2011 11:25 AM To: [email protected] Subject: RE: How to Limit Speaking of MSAA Information Glad it helped Ken. I am going to put it on my list of future classes, when I'm done with going through writing event handlers for all the control types, to go back and think of more examples of things to do with MSAA. I've had a little hitch in my plans, as a pipe broke in my house Friday, and we came home to two inches of water in some rooms; I'll pick back up when we get this under control <blurp blurp bubbles>! Chip _____ From: Scott, Ken [mailto:[email protected]] Sent: Monday, November 07, 2011 12:22 PM To: '[email protected]' Subject: RE: How to Limit Speaking of MSAA Information Chip, I did review this sample earlier when you suggested it. I got so wrapped up in researching the parts that I did not understand that I completely forgot the speak value part which was one of the few that I understood right away. Thanks for sending it direct to me. It does cover a lot of core issues in using MSAA directly. Regards, Ken Scott From: Chip Orange [mailto:[email protected]] Sent: Saturday, November 05, 2011 6:09 PM To: [email protected] Subject: RE: How to Limit Speaking of MSAA Information Hi Ken, Sounds like you got your answer, but in case it's of any help to you, this was one of the examples for scripting class #18, and I'll just put the text of the example file in the message below. Chip ------ ' Scripting class 18 (7/3/2011) MSAA over-view from: http://en.wikipedia.org/wiki/Microsoft_Active_Accessibility MSAA communicates information by sending small chunks of information about elements of a program to the assistive technology object (AT). The four critical pieces of information on which the AT relies to help users interact with applications are an element's role, name, value, and state: Role: Conveys to users via AT what type of object a control is, such as a button or a table. Name: Provides a label for an element, such as Next on a button that moves users to the next page, or First Name for an edit box. Value: Provides the value of the specified object such as the value on a slider bar, or the information in an editable text box. Not all objects have a value. State: Identifies the current condition of the control, such as checked for a checkbox. State advises whether a control can be selected, focused, and/or other types of changeable functionality. ' example 1 ' shows how to block info from WE for a particular event, using MSAA ' (based on code from the MS Access app from GW) ' in this situation, WE is speaking incorrect information when some control(s) get focus in MS Access. ' this app blocks the focus event from WE, and then uses the focus event to cause it to speak the proper information instead. ' since the window onFocus event doesn't allow you to return some special value to prevent WE from handling the focus event, ' we have to prevent WE from seeing the MSAA focus event at all. ' get a COM automation link to the running copy of MS Access dim msAccessObj On Error Resume Next Set msAccessObj = GetObject(, "Access.Application") On Error Goto 0 Dim myMSAAEventSource Set myMSAAEventSource = MSAAEventSource ' make a copy of the object so we can modify it's properties myMSAAEventSource.Process = ClientInformation.ApplicationProcess ' Block focus so we can handle it ourselves Dim blockedFocusEvent Set blockedFocusEvent = myMSAAEventSource.BlockEvent(event_OBJECT_FOCUS, clientInformation.ApplicationProcess) ' now, none of the window onFocus or onChildFocus events for this process will be spoken by WE ' but the events for the window objects will still fire for this app. Dim msaaEventHandle ' now handle all focus events msaaEventHandle = ConnectEvent(myMSAAEventSource, "OnObjectFocus", OnObjectFocus") ' end of main body Sub MSAA_OnObjectFocus(accObj) ' event handler for MSAA focus events ' it tries to get the current control name and type info to be spoken from the MSAA event, ' and if not there, then from the COM link with MS Access. ' if it can't be found in either place, then it passes the focus event to WE, to allow WE to speak what it can. If Not accObj Is Nothing Then Dim handledEvent : handledEvent = False Dim accCopy : Set accCopy = accObj If Not accCopy Is Nothing Then Dim accWin : Set accWin = accCopy.EventWindow ' this is the window which had the focus event If Not accWin Is Nothing Then ' see if we have an active COM automation link to MS Access If Not msAccessObj Is Nothing Then ' have a COM automation link to MS Access, so it's possible to get info from it about the current control SpeakIt ActiveControlName(accCopy), accCopy.Value, ActiveControlType(accCopy) handledEvent = True End If End If End If If Not handledEvent Then ' have no COM link with MS Access, so let WE speak this focus event by passing it on to WE accObj.SimulateEvent event_OBJECT_FOCUS, apAll End If End If End Sub ' MSAA_OnObjectFocus(accObj) Function ActiveControlName(accObj) ' if MSAA didn't supply the control name, then get it from the COM automation link with MS Access ActiveControlName = "" If Not accObj Is Nothing Then ActiveControlName = accObj.Name Exit Function End If Dim screenObj, activeObj If Not msAccessObj Is Nothing Then On Error Resume Next Set screenObj = msAccessObj.Screen If Not screenObj.ActiveControl Is Nothing Then Set activeObj = screenObj.ActiveControl ActiveControlName = activeObj.Name Exit Function End If On Error Goto 0 End If End Function ' ActiveControlName(accObj) Function ActiveControlType(accObj) ' if MSAA didn't supply the control type, then get it from the COM automation link with MS Access ActiveControlType = "" If Not accObj Is Nothing Then Dim accRole : Set accRole = accObj.Role If Not accRole Is Nothing Then ActiveControlType = accRole.Text End If End If On Error Resume Next Dim screenObj, activeObj If Not msAccessObj Is Nothing Then Set screenObj = msAccessObj.Screen If Not screenObj Is Nothing Then Set activeObj = screenObj.ActiveControl If Not activeObj Is Nothing Then ActiveControlType = ControlTypeToText(activeObj.ControlType) End If End If End If On Error Goto 0 End Function ' ActiveControlType(accObj) ' end of example 1 ' example 2: ' mentioned in last week's examples, I used this in the MS Office app to help me read a listbox which never got focus, and so was never read. ' (it also doesn't supply all the standard properties or MSAA information, such as which item was just selected). ' here is shown some of the code which uses MSAA to tell me when an item in the listbox of interest has been selected, ' then, if it's the first time, I can go look at the listbox and get the selection background color. ' (this isn't normally anything you would have to do, it just shows some of the creative things you may have to do in order to get something to speak properly). '* in main body of app prepare for use of MSAA with: Set myMSAAEvents = msaaEventSource ' allows us to customize which MSAA events to watch ' hook the MSAA onObjectSelection events MSAAConnection3 = ConnectEvent(myMSAAEvents, "onObjectSelection", "MSAA_onObjectSelection") ' limit MSAA to the VBA process: myMSAAEvents.Process = VBAWindow.Process ' filters MSAA events for only this process Sub MSAA_OnObjectSelection(access) ' event handler ' called when MSAA signals some text is selected. ' note that almost all MSAA event handlers get just one parameter, an object whose type is "accessible". Dim accObj Dim oWindow If access Is Nothing Then Exit Sub Set accObj = access ' make a copy of the object, because it sometimes "disappears" on you ' don't do anything if this isn't for a list item On Error Resume Next If accObj.role Is Nothing Then Exit Sub If accObj.role.Value <> role_SYSTEM_LISTITEM Then Exit Sub If Err.Number <> 0 Then Exit Sub ' sometimes get an error referencing the value property On Error GoTo 0 ' this is a list item MSAA event On Error Resume Next If Not accObj.State.selected Then Exit Sub ' text was selected in this list item Set oWindow = accObj.EventWindow ' this is the window which generated the MSAA event If Err.Number <> 0 Then Exit Sub On Error GoTo 0 If oWindow Is Nothing Then exit sub ' don't do anything if this is not a listbox window If oWindow.Type <> wtListBox Then Exit Sub ' THIS IS a listbox control ' test to make sure it's the window we think it is If StrComp(UCase(oWindow.parent.className), "NAMELISTWNDCLASS", vbTextCompare) <> 0 Then Exit Sub ' and this is the correct listbox of the correct intellisense window of MS Office SpeakingIntelliSense = True If sHighlightColor = "" Then ' and we don't yet know the highlight color ' note: at this point the msaa_onobjectShow has already identified the listbox window, and has suppressed ' the speech resulting from the up/down arrows (because WE would otherwise just read the current line of code each time ' an arrow key is pressed when an intellisense list is open). sHighlightColor = highlightColorString(oWindow) queue "sayHighlightedItem" End If ' sHighlightColor = "" End Sub ' end of example 2 *** Warning *** There is a known bug with MSAA when running under XP. If events are being filtered by process, The MSAA events do fire as expected, but the parameter passed in is frequently empty (it will be empty if the author of the program did not explicitly handle MSAA information, but relied on Microsoft's "automatic" handling for standard controls). If you plan for a large app to run under xp, you may wish to arrange a small test under xp, so you can see if this effects your situation. Note: I did not receive this news in time to do my own research on the topic before class. _____ From: Scott, Ken [mailto:[email protected]] Sent: Friday, November 04, 2011 4:25 PM To: '[email protected]' Subject: How to Limit Speaking of MSAA Information Hi Window-Eyes scripters, I am trying to limit Window-Eyes to speak only part of MSAA information for a focused control that changes based on cursor movement. The MSAA elements that are currently spoken are: name role and value. I only need the value spoken. If I understand the developers reference and the MSAccessSupport app which was the one app that seem to be dealing with the same problem, I first have to block the default Window-Eyes action then script Window-Eyes to only speak the value portion of the MSAA string. I think I understand the block process well enough to script that part. I am not figuring out a way to speak just the value information. Can anyone suggest a way or direct me to an open app that does something similar so I can see how someone else solved the problem? Thanks in advance for all help. Regards, Ken Scott
