[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-07-02 Thread Michael Weghorn

Hi Michael,

On 2024-07-02 16:46, Michael Meeks wrote:
As mentioned above: If you're aware of major issues regarding support 
for APIs use cases for the focused widget, please don't hesitate to 
mention them so they can be taken into account.


 I occasionally browse:

https://bugs.documentfoundation.org/showdependencytree.cgi?id=101912_resolved=1

 Which suggests there is no shortage of good things to do around the 
focused accessible peer. eg. "Formatted cell text is not exposed to ATs" =)


That one (tdf#115335 - "Formatted cell text is not exposed to assistive 
technologies") is indeed a good example and I'll try to get to that at 
some point before the LO 25.2 release.



 I expect there is a lot of lowish hanging fruit still; is there not ?


That depends on the definition of "lowish". :-)

Naturally, once the issues with the apparently most straightforward 
fixes are addressed, everything else seems a bit less low hanging.


But the truth is that with our codebase, this is often hard to tell 
without doing an actual initial analysis of the individual issues, and 
my plan is to keep doing that.


Regards,
Michael

--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy


[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-07-02 Thread Michael Meeks

Hi Michael,

On 14/06/2024 22:25, Michael Weghorn wrote:
I agree that proper handling of focus is essential and having that work 
properly is definitely a priority.

...
and vast-majority use-cases: of getting really good & complete API and 
feature coverage on the focused widget, before moving off into the 
more tricky stuff :-)


As mentioned above: If you're aware of major issues regarding support 
for APIs use cases for the focused widget, please don't hesitate to 
mention them so they can be taken into account.


I occasionally browse:

https://bugs.documentfoundation.org/showdependencytree.cgi?id=101912_resolved=1

	Which suggests there is no shortage of good things to do around the 
focused accessible peer. eg. "Formatted cell text is not exposed to ATs" =)


I expect there is a lot of lowish hanging fruit still; is there not ?

As a side note, my expectation is that that apart from identifying the 
right way to somehow make the whole document available to ATs, many 
aspects (or current LO shortcomings) that will show up in an effort to 
implement browse mode will be relevant independent of the exact API used 
to expose off-screen content, and addressing those issues will also be 
relevant for interacting with on-screen content (e.g. 
incomplete/incorrect support for specific a11y interfaces, as one 
example you mentioned earlier).


=)

Michael.

--
michael.me...@collabora.com <><, CEO Collabora Productivity
(M) +44 7795 666 147 - timezone usually UK / Europe

--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy


Re: [libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-19 Thread Michael Weghorn

On 2024-06-15 20:31, Michael Weghorn wrote:

On 2024-06-15 14:55, Jason J.G. White wrote:
My limited understanding of the new protocol proposed for Linux by the 
GNOME Foundation is that it is expected to use pipes for data 
transfer, giving better performance than DBus calls. So my naive 
question is: what would be the performance cost of transferring a 
large document over the proposed API? Could it be partly done in the 
background, so that the user can at least start to read/edit the 
document from the top while the data structures are built and sent to 
the screen reader?


To my knowledge, the new protocol is still in development stage, and 
handling of large documents was earlier explicitly mentioned as 
something that will still need further consideration.


I also haven't received any feedback to my ticket concerning that topic 
so far, see [1].



(...)


[1] https://gitlab.gnome.org/GNOME/at-spi2-core/-/issues/142


FYI, Matt Campbell (the developer working on Newton) published a blog 
post about the current status:

https://blogs.gnome.org/a11y/2024/06/18/update-on-newton-the-wayland-native-accessibility-project/

It has this about large documents:

"The biggest unresolved issue at this point is whether the push-based 
approach of Newton, the motivation for which I described in the previous 
post, will have unsolvable drawbacks, e.g. for large text documents. The 
current AccessKit implementation for GtkTextView pushes the full content 
of the document, with complete text layout information. On my brand new 
desktop computer, this has good performance even when reading an 800 KB 
ebook, but of course, there are bigger documents, and that’s a very fast 
computer. We will likely want to explore ways of incrementally pushing 
parts of the document based on what’s visible, adding and removing 
paragraphs as they go in and out of view. The challenge is to do this 
without breaking screen reader functionality that people have come to 
depend on, such as Orca’s Say All command. My best idea about how to 
handle this didn’t occur to me until after I had finished the current 
implementation. Anyway, we should start testing the current, naive 
implementation and see how far it takes us."




--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy


Re: [libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-15 Thread Michael Weghorn

On 2024-06-15 14:55, Jason J.G. White wrote:
My limited understanding of the new protocol proposed for Linux by the 
GNOME Foundation is that it is expected to use pipes for data transfer, 
giving better performance than DBus calls. So my naive question is: what 
would be the performance cost of transferring a large document over the 
proposed API? Could it be partly done in the background, so that the 
user can at least start to read/edit the document from the top while the 
data structures are built and sent to the screen reader?


To my knowledge, the new protocol is still in development stage, and 
handling of large documents was earlier explicitly mentioned as 
something that will still need further consideration.


I also haven't received any feedback to my ticket concerning that topic 
so far, see [1].


Other users may disagree, of course, but from my perspective, having the 
application hang while loading a large document would be unacceptable. 
However, having to wait a little if I first load a large document, then 
jump to the end of it (in the worst-case scenario) would be more 
acceptable. Obviously, loading a large document and then immediately 
retrieving a list of headings, links etc., is another scenario that 
would be subject to potential performance issues. It probably depends on 
what the over-all delays are.


I agree that any solution to expose the document content on the 
accessibility layer shouldn't cause LibreOffice to become unresponsive 
for a significant amount of time when opening a document.


[1] https://gitlab.gnome.org/GNOME/at-spi2-core/-/issues/142

--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy


Re: [libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-15 Thread Jason J.G. White


On 14/6/24 17:25, Michael Weghorn wrote:
Personal preferences aside, Windows/NVDA as the most widely used 
platform indeed generally has some priority for me, as does Writer 
over Calc over everything else.


There are other factors I also take into account, though, e.g. 
involvement/contributions from others like people working in certain 
areas, user requests/tickets, possibilities to cooperate (e.g. the 
Orca maintainer reworking Orca's LibreOffice support and providing a 
lot of helpful feedback and input) or productivity (my productivity on 
Linux is way higher than on Windows, so my take is that putting some 
extra initial effort in order to be able to do most of the analysis 
for issues *also* affecting Windows on Linux usually pays off, in 
particular since the platform APIs IAccessible/AT-SPI2 are fairly 
similar).


A further consideration is that, unlike Windows users, Linux users don't 
have the option of running Microsoft Office without setting up 
virtualization, rebooting the machine to a different environment, or 
using a different machine for the purpose. Of course, some Windows users 
might not be able to afford it, and they're also an important group to 
consider, as are those who prefer the LibreOffice interface.


I personally have access to Microsoft Office as well at the moment, but, 
obviously, not in my Linux environment.


I think the problem of disclosing large documents to accessibility APIs 
is real and important. I suspect this explains the extraordinary 
performance issue that occurs if you try to open a long document in 
Microsoft Word for Mac with the VoiceOver screen reader enabled - the 
application can be completely unresponsive for several minutes, even on 
a fast machine, while the document loads.


My limited understanding of the new protocol proposed for Linux by the 
GNOME Foundation is that it is expected to use pipes for data transfer, 
giving better performance than DBus calls. So my naive question is: what 
would be the performance cost of transferring a large document over the 
proposed API? Could it be partly done in the background, so that the 
user can at least start to read/edit the document from the top while the 
data structures are built and sent to the screen reader?


My assumption is that once the initial transfer is done, all the 
remaining updates are incremental, and relatively fast.


Other users may disagree, of course, but from my perspective, having the 
application hang while loading a large document would be unacceptable. 
However, having to wait a little if I first load a large document, then 
jump to the end of it (in the worst-case scenario) would be more 
acceptable. Obviously, loading a large document and then immediately 
retrieving a list of headings, links etc., is another scenario that 
would be subject to potential performance issues. It probably depends on 
what the over-all delays are.



--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy


[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-14 Thread Michael Weghorn

Hi Michael,

On 2024-06-13 14:49, Michael Meeks wrote:
AT-SPI's flows-from and flows-to relations (and ARIA's aria-flowto) 
seem somewhat similar to the UIA Navigation API you mention.


 =) Ultimately we dynamically create peers as these methods are 
called currently I imagine.


Yes, but that currently has issues, s. the previously mentioned tdf#96492.



If they allow consistent access to off-screen content (related: 
tdf#96492), they could potentially be used to retrieve the 
previous/next heading,...


 Sure; I guess the MS APIs have the problem that the interface 
implemented tends also to be the protocol for remote COM querying of 
peers whereas in Linux we can cut/de-couple that and can do better at 
least in theory.


IIUC, we'd still have a AT-SPI bus roundtrip for each 
flows-from/flows-to call, so using exclusively flows-from/flows-to to 
navigate through a larger document comes with a cost.
On the other hand, IIUC, when using AT-SPI Collection, the bus overhead 
would be lower as it's just one call to the interface method, then the 
filtering/... happens on the application side and the result is returned 
in a single D-Bus reply again. (I didn't have to do with it in practice 
yet, though.)


E.g. it would seem odd to me if an AT starts scrolling through the 
document if a "go to next heading/list item" navigation command is 
triggered, and then e.g. goes back if it doesn't find anything, 
because it can't otherwise access the previously off-screen content to 
search for the item.


 I guess; but I really expect that there are keybindings and/or well 
known actions for expert users that are used left and right, and that in 
reality tracking the focused peer and interrogating it is some 
overwhelming majority of the use cases to the point that first making 
that piece really, really good, fast & complete is far more important 
than anything else; but perhaps I'm mistaken.


I agree that proper handling of focus is essential and having that work 
properly is definitely a priority.


Of course there are issues and a plan to look into those, but my 
impression this far is that reporting focus in Writer works decently *in 
general*.
If you're aware of particularly pressing issues here, please don't 
hesitate to mention them (ideally directly in Bugzilla).


I don't think focus tracking by itself is sufficient for a really good 
a11y experience, though. When it comes to navigating through documents, 
some kind of "browse mode" that allows to navigate between objects in 
documents provides an experience that users are familiar with from e.g. 
web browsers or other office suites and is something that users have 
been asking for repeatedly. (LO's own navigator provides some similar 
functionality, but less well integrated into the screen reader.)


 From a prioritization perspective; I'd really suggest working on 
the majority platforms for the impaired: Windows/NVDA, 


Personal preferences aside, Windows/NVDA as the most widely used 
platform indeed generally has some priority for me, as does Writer over 
Calc over everything else.


There are other factors I also take into account, though, e.g. 
involvement/contributions from others like people working in certain 
areas, user requests/tickets, possibilities to cooperate (e.g. the Orca 
maintainer reworking Orca's LibreOffice support and providing a lot of 
helpful feedback and input) or productivity (my productivity on Linux is 
way higher than on Windows, so my take is that putting some extra 
initial effort in order to be able to do most of the analysis for issues 
*also* affecting Windows on Linux usually pays off, in particular since 
the platform APIs IAccessible/AT-SPI2 are fairly similar).


and vast-majority use-cases: of getting really good & complete API and feature coverage on 
the focused widget, before moving off into the more tricky stuff :-)


As mentioned above: If you're aware of major issues regarding support 
for APIs use cases for the focused widget, please don't hesitate to 
mention them so they can be taken into account.


 But now I shut up ;-) we're working on the web side of this; 
caching bits in the browser and adding another protocol latency there 
- and I'm sure we want to be handling a reasonably bounded set of 
data there =)


Is there an easy way to test COOL a11y web and impacts of potential 
changes?


 Ah - so; we tend to focus on the focused widget and things 'near' 
it - adjacent table cells etc. when populating our shadow DOM. But at 
some level the use-case we have for the a11y APIs is not really 
different than an AT would use I think.


(I just opened a sample Writer doc on nextcloud.documentfoundation.org 
and couldn't find the doc content via Accerciser in a quick test, but 
am also not very familiar with web content/browser a11y.)


 You will want:

     false


 Enabled in coolwsd.xml - and then to turn on screen-reading support.

 =)


Thanks, will try that!

    

[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-13 Thread Michael Weghorn

On 2024-06-12 16:55, Michael Weghorn wrote:
to look into at some point. My idea so far is to also expose pages on 
the a11y level, which should avoid the problem of a single object 
(the document) having an enormous amount of children due to that.

If there any general concerns about that, please raise them. :-)


 I guess this moves the problem to re-pagination; where we can get 
300+ pages re-built for the sake of moving a single paragraph; then 
again - I guess if we are notifying changes in position on large sets 
of accessible peers we have a similar problem.


Good point! That could indeed be problematic performance-wise.


The feedback I've received from a11y experts so far is that 
off-screen doc content should *generally* be exposed on the a11y 
level, and limiting Calc to not do that with its huge amount of table 
cells is meant to be an exception to the rule in that regard (see 
e.g. the discussion in [2] and tdf#156657).


 I really think that's a mistake that will ultimately hurt ATs 
performance and that we should focus on the end-user use-cases we want 
to succeed with - rather than having an abstract absolutist 
pre-conception that we can expose everything in an efficient way =)


Sure - if there's a better way to properly make the AT use cases a 
reality, then let's go that route instead. :-)


One thing that came to my mind is that there's an Action interface (in 
LO and platform a11y APIs) that can be used to provide a set of actions 
that can be triggered by ATs.
This might be usable to provide functionality without having to 
introduce new platform APIs (either permanently, or at least for initial 
prototyping).


I'm wondering whether one potential approach could e.g. be to provide 
different "modes" on how much Writer exposes in the a11y tree, and a way 
to switch between those.

From looking a bit further into NVDA and Orca doc and some experimenting
It seems to me that access to the whole document is needed in particular 
in (1) structural navigation/browse mode, but not (2) focus/editing mode.
If so, one idea might be to only expose the whole document in the a11y 
tree when in read-only "browse mode"/structural navigation mode, not 
when editing the doc.


Screen readers could e.g. explicitly request switching to "expose the 
whole doc" mode when switching to browse mode, and then back to "expose 
only on-screen objects" mode afterwards.
(Alternatively, other variants like explicitly requesting to include 
another page in the a11y tree,... so ATs can work with that could also 
be possible this way.)


Making exposing the whole tree opt-in and only used in read-only mode in 
practice might help address the major performance concerns raised wrt 
re-pagination for now. (The default remains unchanged; ATs can 
explicitly switch, but should then then know what they're doing.)


Or, depending on what exactly is needed for ATs, LO could at least in 
theory even provide navigation actions like "jump to the next 
header/list",... via the Action interface on the a11y layer right away, 
instead of ATs having to implement the logic themselves (and needing 
access to the whole document).
But then, I don't yet see how use cases like "Display a list of all 
headings in the doc" could easily be covered that way.


Just mentioning these thoughts here for now. What's the best way forward 
still needs further consideration/analysis and discussion.


--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy


[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-12 Thread Michael Weghorn

Hi Michael,

On 2024-06-11 11:40, Michael Meeks wrote:
 Some great questions and points here; forgive my intruding into the 
space again =)


No need to apologize - thanks a lot for your valuable input! :-)

* How do other interfaces (like AT-SPI Table, TableCell and Selection) 
expose information? Does e.g. the table report it only has 50 rows and 
30 columns if that's what's visible on screen? Does cell Q227 report a 
row and column index of 0 if it's the first one in the visible area?


 I think exposing the whole thing through the Table interface may 
make some sense; it is clearly a crazy set of cells - and think it's 
reasonable to blame ATs if they use this interface for doing something 
silly.


In the meantime, Joanmarie has added some further very useful 
information in tdf#156657 comment [1], also giving a reference to a grid 
example from the ARIA WG [2].


There, the Table interface also only exposes the same amount of cells as 
are exposed via the a11y tree.


(An even more advanced example also managing selection that that example 
references [3] currently doesn't exist yet, unfortunately. ("This 
example has not yet been developed."))


 Right; so - I mentioned "near to the screen" - by near; I mean we 
will probably want a number of things that are navigationally close: eg. 
"next heading" or somesuch - to lurk around as real & tracked peers. The 
content of the Navigator headings should prolly always be present in a 
writer document's object hierarcy IMHO. That should let ATs very quickly 
enumerate headings, jump focus to them with a simple API etc.


That sounds interesting, but in a way also like a rather strange tree to 
me if it contains elements of some type for the whole doc, but other 
parts of the document in between are missing.


From looking into the NVDA and Orca doc, other objects of interest are 
e.g. tables, lists, list items, or links. I suppose that including all 
of these (and more) in the tree could potentially again result in a 
large tree and the re-pagination update problems you've mentioned, for 
particular documents.


* How do screen readers implement features like "read the whole row"? 


 This comes down to the navigation API I mentioned: having a good 
API to allow continuous screen-reading of large data-sets - with caching 
pre-loading & fetching along eg. a selection is really useful. Current 
writer behavior is far from optimal since you need to do something odd 
to get a simple navigation such as "next page" IIRC.


Do they just read the part of the row that's currently visible on 
screen and leave out the rest? Or do they somehow implement some extra 
logic

to  retrieve the remaining content?


 Extra navigation / enumeration logic I think.


I see, the navigation API apparently is definitely an essential key 
aspect then.


If those need changes to the platform a11y APIs, we theoretically have 
more chances for these for some (the FLOSS ones: Linux/AT-SPI, 
Windows/IAccessible2, maybe Linux/Newton) than others 
(macOS/NSAccessibility, Windows/UIA - the latter not used by LO at this 
point in time).


though of course it is then ideal to have some nice navigation API 
support wrapped around that


What kind of API does that refer to? Existing or new API on the
platform a11y level that LO (or the toolkits it uses) would then
implement, or something else? Do you have anything particular in mind?


 I was actually somewhat optimistic about the UIA API Navigation API 
conceptually:


https://learn.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-irawelementproviderfragment-navigate


AT-SPI's flows-from and flows-to relations (and ARIA's aria-flowto) seem 
somewhat similar to the UIA Navigation API you mention.


If they allow consistent access to off-screen content (related: 
tdf#96492), they could potentially be used to retrieve the previous/next 
heading,...


IIUC, that would require retrieving one object at a time via the AT-SPI 
bus, though, as compared to using e.g. AT-SPI Collection (which can only 
take into account objects that are already part of the a11y tree).


 Although - I'd really suggest that a11y doesn't work against the 
application, and if navigating - it should allow the AT to scroll the 
actual visible/view-port to match what is being interrogated.


Interesting thought, and maybe that could be part of the solution, if it 
becomes clearer what that can look like in practice.


E.g. it would seem odd to me if an AT starts scrolling through the 
document if a "go to next heading/list item" navigation command is 
triggered, and then e.g. goes back if it doesn't find anything, because 
it can't otherwise access the previously off-screen content to search 
for the item.


to look into at some point. My idea so far is to also expose pages on 
the a11y level, which should avoid the problem of a single object (the 
document) having an enormous amount of children due to that.

If there any 

[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-11 Thread Michael Meeks

Hi Michael,

Some great questions and points here; forgive my intruding into the 
space again =)

On 11/06/2024 08:55, Michael Weghorn wrote:
Limiting children to the to "close to visible" cells sounds like a 
potential approach.


For a performant, caching AT I would argue this is arguably the only 
feasible option =) But ... the navigation API I mentioned is important.

However, that would IMHO still need a clear specification on how to 
implement it and how all relevant AT use cases are covered.


Agreed =)


Some aspects/questions that might need some further consideration:

* How do other interfaces (like AT-SPI Table, TableCell and Selection) 
expose information? Does e.g. the table report it only has 50 rows and 
30 columns if that's what's visible on screen? Does cell Q227 report a 
row and column index of 0 if it's the first one in the visible area?


I think exposing the whole thing through the Table interface may make 
some sense; it is clearly a crazy set of cells - and think it's reasonable to 
blame ATs if they use this interface for doing something silly.

* In some cases, off-screen children are of interest, e.g. if they are 
contained in the current selection. How should that be handled? (e.g. 
how does the screen reader announce something like "cell A1 to C100 
selected" if cell A1 "doesn't exist" because it's off-screen?


Right; so - I mentioned "near to the screen" - by near; I mean we will probably want 
a number of things that are navigationally close: eg. "next heading" or somesuch - to lurk 
around as real & tracked peers. The content of the Navigator headings should prolly always be 
present in a writer document's object hierarcy IMHO. That should let ATs very quickly enumerate 
headings, jump focus to them with a simple API etc.

* Exposing and caching all cells based on visibility means that whenever 
the view port changes, this needs to be actively updated (push 
approach), which comes with a cost (that I can't estimate right now).
(We currently have that for other modules, see e.g. comment [3] for 


That is the case; we do that for writer on page-down of course.

* How do screen readers implement features like "read the whole row"? 


This comes down to the navigation API I mentioned: having a good API to allow 
continuous screen-reading of large data-sets - with caching pre-loading & fetching along 
eg. a selection is really useful. Current writer behavior is far from optimal since you need 
to do something odd to get a simple navigation such as "next page" IIRC.

Do 
they just read the part of the row that's currently visible on screen 
and leave out the rest? Or do they somehow implement some extra logic

to  retrieve the remaining content?


Extra navigation / enumeration logic I think.

* Is navigating to an "arbitrary" cell still possible via a11y API, e.g. 
if some screen reader specific table navigation command implements "jump 
to the first/last cell in the table" or "select the current row")?


There should be a widget for this at the top of calc to jump to and 
select cells - if I were customizing an AT I'd use that.

though of course it is then ideal to have some nice navigation API 
support wrapped around that


What kind of API does that refer to? Existing or new API on the
platform a11y level that LO (or the toolkits it uses) would then
implement, or something else? Do you have anything particular in mind?


I was actually somewhat optimistic about the UIA API Navigation API 
conceptually:

https://learn.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-irawelementproviderfragment-navigate

Although - I'd really suggest that a11y doesn't work against the 
application, and if navigating - it should allow the AT to scroll the actual 
visible/view-port to match what is being interrogated.

I've been told repeatedly that the fact that Writer doesn't expose 
off-screen document content is indeed a problem as it breaks features 
like browse mode/document navigation in NVDA or Orca (see e.g. 
tdf#35652, tdf#137955, tdf#91739, tdf#96492).


I'm not convinced that "can't see it in Accerssisor" is a real problem; 
what we all want are fast, accurate and helpful ATs for the impaired. Perhaps I 
mis-understand, but browse-mode seems conceptually similar to navigating by scrolling the 
current window down a document - without changing the cursor/document focus.

to look into at some point. My idea so far is to also expose pages on 
the a11y level, which should avoid the problem of a single object (the 
document) having an enormous amount of children due to that.

If there any general concerns about that, please raise them. :-)


I guess this moves the problem to re-pagination; where we can get 300+ 
pages re-built for the sake of moving a single paragraph; then again - I guess 
if we are notifying changes in position on large sets of accessible 

[libreoffice-accessibility] Re: [a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

2024-06-11 Thread Michael Weghorn

On 2024-06-11 10:48, Michael Weghorn wrote:

On 2024-06-11 10:07, Samuel Thibault wrote:

Michael Weghorn, le mar. 11 juin 2024 07:55:47 +, a ecrit:
The feedback I've received from a11y experts so far is that 
off-screen doc
content should *generally* be exposed on the a11y level, and limiting 
Calc

to not do that with its huge amount of table cells is meant to be an
exception to the rule in that regard (see e.g. the discussion in [2] and
tdf#156657).


I guess we can have the same kind of issue with a >>100 pages writer
document?


I would expect that there should still be way less objects than for the 
Calc case and the child count of each individual object will be lower 
(when Writer pages are reflected in the a11y tree, which they currently 
are not).


In a first quick experiment with a local WIP branch, nothing was 
obviously broken right away when opening a ~800 pages Writer doc (but 
just containing simple text, nothing more complex) and moving around a 
bit while Orca was running or inspecting the tree using Accerciser.


But of course, if some AT (or some underlying library) recursively 
queries and caches the whole tree and very complex documents are 
involved, then similar performance issues seem possible. (macOS and Win 
also need explicit testing, of course.)


To me, this whole topic seems to be basically the same as the 
anticipated and still-to-be-solved performance aspect with the push 
approach that the alleged AT-SPI2 successor (codename "Newton") already 
mentions in its concept, see e.g.

https://gitlab.gnome.org/GNOME/at-spi2-core/-/issues/142

Hopefully, a clear concept will come out of the work on the new protocol.

Otherwise, as long as the underlying platform a11y protocols are 
pull-based and given the input I've received up to this point, I tend to 
think that ATs actively querying the tree are primarily responsible for 
limiting that to a reasonable amount of information, but I'm thankful 
for any guidance here...


Technically, platform-specific handling of how much information is 
exposed would be possible, obviously adding a certain amount of 
complexity and maintenance burden.




PS: CC'ing the accessibility mailing list, where people with more 
insights on the AT side are subscribed. Input welcome!

For reference: Thread starts here:
https://lists.freedesktop.org/archives/libreoffice/2024-June/092039.html


PPS: Now actually CC'ing the accessibility mailing list as mentioned in 
my previous email...


--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy