Hi Michael, On 2024-06-11 11:40, Michael Meeks wrote:
Some great questions and points here; forgive my intruding into the space again =)
No need to apologize - thanks a lot for your valuable input! :-)
* How do other interfaces (like AT-SPI Table, TableCell and Selection) expose information? Does e.g. the table report it only has 50 rows and 30 columns if that's what's visible on screen? Does cell Q227 report a row and column index of 0 if it's the first one in the visible area?I think exposing the whole thing through the Table interface may make some sense; it is clearly a crazy set of cells - and think it's reasonable to blame ATs if they use this interface for doing something silly.
In the meantime, Joanmarie has added some further very useful information in tdf#156657 comment [1], also giving a reference to a grid example from the ARIA WG [2].
There, the Table interface also only exposes the same amount of cells as are exposed via the a11y tree.
(An even more advanced example also managing selection that that example references [3] currently doesn't exist yet, unfortunately. ("This example has not yet been developed."))
Right; so - I mentioned "near to the screen" - by near; I mean we will probably want a number of things that are navigationally close: eg. "next heading" or somesuch - to lurk around as real & tracked peers. The content of the Navigator headings should prolly always be present in a writer document's object hierarcy IMHO. That should let ATs very quickly enumerate headings, jump focus to them with a simple API etc.
That sounds interesting, but in a way also like a rather strange tree to me if it contains elements of some type for the whole doc, but other parts of the document in between are missing.
From looking into the NVDA and Orca doc, other objects of interest are e.g. tables, lists, list items, or links. I suppose that including all of these (and more) in the tree could potentially again result in a large tree and the re-pagination update problems you've mentioned, for particular documents.
* How do screen readers implement features like "read the whole row"?This comes down to the navigation API I mentioned: having a good API to allow continuous screen-reading of large data-sets - with caching pre-loading & fetching along eg. a selection is really useful. Current writer behavior is far from optimal since you need to do something odd to get a simple navigation such as "next page" IIRC.Do they just read the part of the row that's currently visible on screen and leave out the rest? Or do they somehow implement some extra logicto retrieve the remaining content?Extra navigation / enumeration logic I think.
I see, the navigation API apparently is definitely an essential key aspect then.
If those need changes to the platform a11y APIs, we theoretically have more chances for these for some (the FLOSS ones: Linux/AT-SPI, Windows/IAccessible2, maybe Linux/Newton) than others (macOS/NSAccessibility, Windows/UIA - the latter not used by LO at this point in time).
though of course it is then ideal to have some nice navigation API support wrapped around thatWhat kind of API does that refer to? Existing or new API on the platform a11y level that LO (or the toolkits it uses) would then implement, or something else? Do you have anything particular in mind?I was actually somewhat optimistic about the UIA API Navigation API conceptually:https://learn.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-irawelementproviderfragment-navigate
AT-SPI's flows-from and flows-to relations (and ARIA's aria-flowto) seem somewhat similar to the UIA Navigation API you mention.
If they allow consistent access to off-screen content (related: tdf#96492), they could potentially be used to retrieve the previous/next heading,...
IIUC, that would require retrieving one object at a time via the AT-SPI bus, though, as compared to using e.g. AT-SPI Collection (which can only take into account objects that are already part of the a11y tree).
Although - I'd really suggest that a11y doesn't work against the application, and if navigating - it should allow the AT to scroll the actual visible/view-port to match what is being interrogated.
Interesting thought, and maybe that could be part of the solution, if it becomes clearer what that can look like in practice.
E.g. it would seem odd to me if an AT starts scrolling through the document if a "go to next heading/list item" navigation command is triggered, and then e.g. goes back if it doesn't find anything, because it can't otherwise access the previously off-screen content to search for the item.
to look into at some point. My idea so far is to also expose pages on the a11y level, which should avoid the problem of a single object (the document) having an enormous amount of children due to that.If there any general concerns about that, please raise them. :-)I guess this moves the problem to re-pagination; where we can get 300+ pages re-built for the sake of moving a single paragraph; then again - I guess if we are notifying changes in position on large sets of accessible peers we have a similar problem.
Good point! That could indeed be problematic performance-wise.
The feedback I've received from a11y experts so far is that off-screen doc content should *generally* be exposed on the a11y level, and limiting Calc to not do that with its huge amount of table cells is meant to be an exception to the rule in that regard (see e.g. the discussion in [2] and tdf#156657).I really think that's a mistake that will ultimately hurt ATs performance and that we should focus on the end-user use-cases we want to succeed with - rather than having an abstract absolutist pre-conception that we can expose everything in an efficient way =)
Sure - if there's a better way to properly make the AT use cases a reality, then let's go that route instead. :-)
Of course; I'm just one viewpoint. My strong feeling is that focusing on things that make it easier to code fast, simple ATs that meet the common use-cases people want is the vital thing; and I really think that trying to re-build arbitrary document models and synchronize them on the other end of a bus - particularly if we require lots of synchronous round-trips to interrogate the content - is not going to fly.On 11/06/2024 09:49, Michael Weghorn wrote:Otherwise, as long as the underlying platform a11y protocols are pull-based and given the input I've received up to this point, I tend to think that ATs actively querying the tree are primarily responsible for limiting that to a reasonable amount of information, but I'm thankful for any guidance here...Its a nice hope =) I'd want to create APIs that capture the common things that ATs want to do, make them easy, and really hard to screw up.But now I shut up ;-) we're working on the web side of this; caching bits in the browser and adding another protocol latency there - and I'm sure we want to be handling a reasonably bounded set of data there =)
Is there an easy way to test COOL a11y web and impacts of potential changes?(I just opened a sample Writer doc on nextcloud.documentfoundation.org and couldn't find the doc content via Accerciser in a quick test, but am also not very familiar with web content/browser a11y.)
As an additional note, one more potential source to get some interesting insights could be to check how NVDA's browse mode is currently implemented for MS Word, for example.
Thanks again for your thoughts! Regards, Michael [1] https://bugs.documentfoundation.org/show_bug.cgi?id=156657#c11 [2] https://www.w3.org/WAI/ARIA/apg/patterns/grid/examples/data-grids/[3] https://www.w3.org/WAI/ARIA/apg/patterns/grid/examples/advanced-data-grid/
OpenPGP_signature.asc
Description: OpenPGP digital signature
