Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-09-13 23:56, Michael Weghorn wrote:
Or, if you're also attending LibreOffice 
conference next week in person, we could also take a look at it together 
then for example.


Colomban and I analyzed this a bit further yesterday, without coming up 
with an actual solution yet.


We noticed that at-spi2-core itself is using a similar approach to run 
its CI tests, which also fails when run in a Fedora 38 GNOME session.


Colomban found this existing issue about running tests using dbus-broker 
on which I've commented now, hoping to get input from upstream:

https://gitlab.gnome.org/GNOME/at-spi2-core/-/issues/49


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-09-14 11:05, Colomban Wendling wrote:
I'm on vacation this week, but could give this another try in my 
Fedora VM sometime afterwards. Or, if you're also attending 
LibreOffice conference next week in person, we could also take a look 
at it together then for example.


I'll be there, we can indeed try and pull some hairs together on this 
then :)


Great! :)


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Colomban Wendling]

Le 13/09/2023 à 22:56, Michael Weghorn a écrit :

On 2023-09-12 10:49, Colomban Wendling wrote:

All that is a bit confusing to me, but let me try ton understand:
- On a system using dbus-broker, when logged in under GNOME, test fails?
- On *the same system*, but logging-in through SSH works!??


Yes, that's what I've observed in a Fedora 38 VM.

That's worrying, as it suggests the running environment has too much 
influence on the test run, which is not good.


However, IIUC you suggest it might be a dbus/dbus-borken conflict 
somehow?


That's what it looked like to me during first analysis.

What might be happening is that at-spi-bus-launcher detects a systemd 
environment that makes it think running dbus-broker should work, but 
something about the systemd/D-Bus environment in the virtual X session 
provided by xfvb-run and dbus-launch doesn't match what's 
expected/required.


[…]


Ah, makes some sense indeed.  I guess if at-spi itself is not using the 
same DBus implementation as the app is it might not work indeed…  and 
IIUC here the ap will be using the dbus-launch one, which IIUC is gonna 
be dbus-daemon, and at-spi2 is using dbus-broker.


I am not knowledgeable about that, but maybe we could run in a separate 
cgroup to escape this?


Or maybe we could have a dbus-broker compatibility, either as you 
mention below (that I don't fully follow without diving deeper), or 
simply emulating the bits we need like --exit-with-session which should 
not be that hard to mimic (basically watch the Xvfb session lifetime, 
e.g. with `xprop -root -spy` and terminate our own dbus-broker when it 
exits).


I'm on vacation this week, but could give this another try in my Fedora 
VM sometime afterwards. Or, if you're also attending LibreOffice 
conference next week in person, we could also take a look at it together 
then for example.


I'll be there, we can indeed try and pull some hairs together on this 
then :)

Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-09-12 10:49, Colomban Wendling wrote:

All that is a bit confusing to me, but let me try ton understand:
- On a system using dbus-broker, when logged in under GNOME, test fails?
- On *the same system*, but logging-in through SSH works!??


Yes, that's what I've observed in a Fedora 38 VM.

That's worrying, as it suggests the running environment has too much 
influence on the test run, which is not good.


However, IIUC you suggest it might be a dbus/dbus-borken conflict 
somehow?


That's what it looked like to me during first analysis.

What might be happening is that at-spi-bus-launcher detects a systemd 
environment that makes it think running dbus-broker should work, but 
something about the systemd/D-Bus environment in the virtual X session 
provided by xfvb-run and dbus-launch doesn't match what's expected/required.


See also this comment in the at-spi-bus-launcher source code [1]:

  /* This detects whether we are running under systemd. We only try to
   * use dbus-broker if we are running under systemd because D-Bus
   * service activation won't work otherwise.
   */

The following sample program, based on the at-spi-bus-launcher-code, 
could explain why the SSH case works: The "systemd, use dbus-broker" 
case is triggered for me on Debian testing in a KDE Plasma X11 session 
(and probably the same way for a Fedora 38 GNOME session), but not in an 
SSH session ("no systemd, don't use dbus-broker").



---

// build: g++ -o test test.cxx -lsystemd

#include 
#include 
#include 

int main()
{

// s. 
https://gitlab.gnome.org/GNOME/at-spi2-core/-/blob/e7310f8a57cf9d9744f68059fad451cb3acd116a/bus/at-spi-bus-launcher.c#L453-460

char* unit;
if (sd_pid_get_user_unit (getpid (), ) >= 0)
std::cout << "systemd, use dbus-broker" << std::endl;
else
std::cout << "no systemd, don't use dbus-broker" << std::endl;
}

---

What if we try `dbus-broker-launch` (assuming it's compatible) 
instead of `dbus-launch` if the former is available? 
Would that help?



As Stephan mentioned already, it's not compatible as a drop-in-replacement.

Maybe this more complicated way mentioned in the Github discussion I 
mentioned earlier could work:


Question [2]:

"What is the recommended way to run a nested dbus session, if not by 
using dbus-run-session? I mostly use it to run debug instances of GNOME 
Shell, as well as the for the CI pipeline when running inside Docker."

Answer (next comment):

"We don't have a recommended way to do that. If that functionality is
needed, it should be straightforward to achieve by just invoking your
private instance of `dbus-broker-launch`. Socket-activation for the
listener-socket is mandatory, though. So I would imagine creating
`dbus-broker@.socket` and `dbus-broker@.service` as systemd template
units, which take a path as template-argument where to place the socket.
It should then be as simple as `systemctl start
dbus-broker@foobar.service` to get it up and running.

If, however, you intend to integrate it into a test-suite, I would
rather expect the test-suite to create the listener manually and pass it
through `execve(2)` to `dbus-broker-launch`."

I don't have any system to test this with yet, and it might take some 
time before I can have a F38 VM up and running with a LO build setup, 
but I'll try and see to this soon if it's a concern.


I'm on vacation this week, but could give this another try in my Fedora 
VM sometime afterwards. Or, if you're also attending LibreOffice 
conference next week in person, we could also take a look at it together 
then for example.



[1] 
https://gitlab.gnome.org/GNOME/at-spi2-core/-/blob/e7310f8a57cf9d9744f68059fad451cb3acd116a/bus/at-spi-bus-launcher.c#L453-460

[2] https://github.com/bus1/dbus-broker/issues/145#issuecomment-437874214


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Stephan Bergmann]

On 9/12/23 09:49, Colomban Wendling wrote:
However, IIUC you suggest it might be a dbus/dbus-borken conflict 
somehow?  What if we try `dbus-broker-launch` (assuming it's compatible) 
instead of `dbus-launch` if the former is available?  Would that help?


Doesn't appear to be compatible at all.  With


--- a/vcl/CppunitTest_vcl_gtk3_a11y.mk
+++ b/vcl/CppunitTest_vcl_gtk3_a11y.mk
@@ -13,7 +13,7 @@ $(call gb_CppunitTest_get_target,vcl_gtk3_a11y) : 
gb_TEST_ENV_VARS += SAL_USE_VC
 $(call gb_CppunitTest_get_target,vcl_gtk3_a11y) : gb_TEST_ENV_VARS += 
GDK_BACKEND=x11
 ifeq (,$(VCL_GTK3_TESTS_NO_XVFB))
 $(call gb_CppunitTest_get_target,vcl_gtk3_a11y) : \
-   ICECREAM_RUN += $(XVFB_RUN) $(DBUS_LAUNCH) --exit-with-session
+   ICECREAM_RUN += $(XVFB_RUN) /usr/bin/dbus-broker-launch 
--exit-with-session
 endif
 
 $(eval $(call gb_CppunitTest_CppunitTest,vcl_gtk3_a11y))


CppunitTest_vcl_gtk3_a11y trivially fails with


/usr/bin/dbus-broker-launch: unrecognized option '--exit-with-session'

Re: New tests: GTK3 platform accessibility

[Colomban Wendling]

Le 29/08/2023 à 14:36, Michael Weghorn a écrit :

[…]

This shows that there is an attempt to start a dbus session using 
dbus-broker-launch. This seems surprising at first, since 
dbus-run-session (and dbus-launch) use dbus-daemon in $PATH by default.


But then, there's an at-spi-bus-launcher involved for running a separate 
bus for AT-SPI/a11y, and the README [1] mentions that that is a separate 
instance of dbus-daemon or dbus-broker. From a first look into the 
source code [2], it looks like it prefers dbus-broker-launch when built 
with the corresponding option and running under systemd, but that 
apparently fails at that stage.


Sorry for the delay, I have been on vacation with minimal to no Internet.

All that is a bit confusing to me, but let me try ton understand:
- On a system using dbus-broker, when logged in under GNOME, test fails?
- On *the same system*, but logging-in through SSH works!??

That's worrying, as it suggests the running environment has too much 
influence on the test run, which is not good.


However, IIUC you suggest it might be a dbus/dbus-borken conflict 
somehow?  What if we try `dbus-broker-launch` (assuming it's compatible) 
instead of `dbus-launch` if the former is available?  Would that help?


I don't have any system to test this with yet, and it might take some 
time before I can have a F38 VM up and running with a LO build setup, 
but I'll try and see to this soon if it's a concern.


Regards,
Colomban

Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-08-28 19:54, Stephan Bergmann wrote:

Otherwise: Does the test work when run in an X11 session instead?


no; guess I need to install some piece of software that is missing from 
my machine


I can reproduce in a GNOME session in a Fedora 38 VM.

It seems to be related to GNOME and dbus-broker being used by default on 
Fedora, together with the gtk3 a11y test suite using xvfb and dbus-launch.


It works just fine for me with the same VM in an SSH or OpenBox session 
instead of the GNOME one.


The error can also be triggered with a simple sample program instead 
(based on what the gtk3 a11y test does):



// to build:
// g++ -g $(pkg-config --cflags atspi-2) -o atspi-sample-app 
atspi-sample-app.cxx $(pkg-config --libs atspi-2)


#include 
#include 

#include 

int main()
{
const int nDesktops = atspi_get_desktop_count();
for (int desktopId = 0; desktopId < nDesktops; desktopId++)
{
AtspiAccessible* pAcc = atspi_get_desktop(desktopId);
GError* pError = nullptr;
gint nChildCount = atspi_accessible_get_child_count(pAcc, );
std::cout << "pAcc: " << pAcc << ", childCount: " << 
nChildCount << ", error: " << pError

  << std::endl;
assert(nChildCount >= 0);
}
};


When running that app as follows, it fails the same way in the Fedora 38 
GNOME session:



$ xvfb-run dbus-launch --exit-with-session ./atspi-sample-app

(process:4584): dbind-WARNING **: 12:01:39.097: Couldn't get application list: 
Could not activate remote peer: unit failed.

(process:4584): GLib-GObject-CRITICAL **: 12:01:39.097: g_object_unref: 
assertion 'G_IS_OBJECT (object)' failed

(process:4584): dbind-WARNING **: 12:01:39.099: AT-SPI: Error in GetItems, 
sender=org.freedesktop.DBus, error=Could not activate remote peer: unit failed.
pAcc: 0xd0d500, childCount: -1, error: 0
atspi-sample-app: atspi-sample-app.cxx:19: int main(): Assertion `nChildCount 
>= 0' failed.
/usr/bin/xvfb-run: line 181:  4584 Aborted  



Running the same with dbus-run-session (hopefully doesn't affect the 
relevant behavior and) gives additional output:



$ xvfb-run dbus-run-session ./atspi-sample-app
dbus-daemon[4713]: [session uid=1000 pid=4713] Activating service name='org.a11y.Bus' requested by 
':1.0' (uid=1000 pid=4714 comm="./atspi-sample-app" label="kernel")
dbus-daemon[4713]: [session uid=1000 pid=4713] Successfully activated service 
'org.a11y.Bus'
Policy to allow eavesdropping in /usr/share/defaults/at-spi2/accessibility.conf 
+15: Eavesdropping is deprecated and ignored
Policy to allow eavesdropping in /usr/share/defaults/at-spi2/accessibility.conf 
+17: Eavesdropping is deprecated and ignored
dbus-daemon[4713]: [session uid=1000 pid=4713] Activating service name='org.freedesktop.systemd1' 
requested by ':1.2' (uid=1000 pid=4722 comm="/usr/bin/dbus-broker-launch 
--config-file=/usr/sha" label="kernel")
dbus-daemon[4713]: [session uid=1000 pid=4713] Activated service 
'org.freedesktop.systemd1' failed: Process org.freedesktop.systemd1 exited with 
status 1
dbus-daemon[4713]: [session uid=1000 pid=4713] Activating service name='org.freedesktop.systemd1' 
requested by ':1.2' (uid=1000 pid=4722 comm="/usr/bin/dbus-broker-launch 
--config-file=/usr/sha" label="kernel")
dbus-daemon[4713]: [session uid=1000 pid=4713] Activated service 
'org.freedesktop.systemd1' failed: Process org.freedesktop.systemd1 exited with 
status 1

(process:4714): dbind-WARNING **: 12:03:53.467: Couldn't get application list: 
Could not activate remote peer: unit failed.

(process:4714): GLib-GObject-CRITICAL **: 12:03:53.467: g_object_unref: 
assertion 'G_IS_OBJECT (object)' failed
dbus-daemon[4713]: [session uid=1000 pid=4713] Activating service name='org.freedesktop.systemd1' 
requested by ':1.2' (uid=1000 pid=4722 comm="/usr/bin/dbus-broker-launch 
--config-file=/usr/sha" label="kernel")
dbus-daemon[4713]: [session uid=1000 pid=4713] Activated service 
'org.freedesktop.systemd1' failed: Process org.freedesktop.systemd1 exited with 
status 1

(process:4714): dbind-WARNING **: 12:03:53.469: AT-SPI: Error in GetItems, 
sender=org.freedesktop.DBus, error=Could not activate remote peer: unit failed.
pAcc: 0x15fa500, childCount: -1, error: 0
atspi-sample-app: atspi-sample-app.cxx:19: int main(): Assertion `nChildCount 
>= 0' failed.


This shows that there is an attempt to start a dbus session using 
dbus-broker-launch. This seems surprising at first, since 
dbus-run-session (and dbus-launch) use dbus-daemon in $PATH by default.


But then, there's an at-spi-bus-launcher involved for running a separate 
bus for AT-SPI/a11y, and the README [1] mentions that that is a separate 
instance of dbus-daemon or dbus-broker. From a first look into the 
source code [2], it looks like it prefers dbus-broker-launch when built 
with the corresponding option and running under systemd, but that 
apparently fails at that stage.



I currently can't come with any idea how to fix/avoid that on our end, 
suggestions welcome.

Re: New tests: GTK3 platform accessibility

[Stephan Bergmann]

On 8/25/23 09:06, Michael Weghorn wrote:
In my test, it failed differently on Wayland, and forcing the X11 Gdk 
backend makes that work:

https://gerrit.libreoffice.org/c/core/+/156086

Does that fix your case as well?


no


Otherwise: Does the test work when run in an X11 session instead?


no; guess I need to install some piece of software that is missing from 
my machine

Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-08-24 13:38, Stephan Bergmann wrote:
When I --enable-atspi-tests (which passes configure) on my local Fedora 
38 Linux machine, CppunitTest_vcl_gtk3_a11y fails the



    assert(m_pAtspiApp);


in Atspi2TestBase::setUp 
(vcl/qa/cppunit/a11y/atspi2/atspi2testbase.hxx), apparently because



    Atspi::Accessible desktop(atspi_get_desktop(desktopId));


in Atspi2TestBase::getApp 
(vcl/qa/cppunit/a11y/atspi2/atspi2testbase.hxx) produces


** (cppunittester:1641471): WARNING **: 12:56:34.251: AT-SPI: Could 
not obtain desktop path or name



(cppunittester:1641471): dbind-WARNING **: 12:56:34.251: Couldn't get 
application list: Could not activate remote peer: unit failed.


(cppunittester:1641471): GLib-GObject-CRITICAL **: 12:56:34.251: 
g_object_unref: assertion 'G_IS_OBJECT (object)' failed


on stderr but returns an empty Atspi::Accessible with no children to 
iterate over.


Is there perhaps anything I would need to additionally install or to 
enable to make this work?  Or does it just not work on Wayland?


In my test, it failed differently on Wayland, and forcing the X11 Gdk 
backend makes that work:

https://gerrit.libreoffice.org/c/core/+/156086

Does that fix your case as well?
Otherwise: Does the test work when run in an X11 session instead?


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Stephan Bergmann]

On 8/1/23 19:30, Colomban Wendling wrote:
These tests are enabled by default if all dependencies are available. 
That is a working Xvfb [4], dbus-launch, and the AT-SPI2 library [5] 
which is used to communicate with the platform accessibility daemon.  It 
is also possible to either enable or disable them forcefully, using 
--enable-atspi-test and --disable-atspi-tests respectively.


When I --enable-atspi-tests (which passes configure) on my local Fedora 
38 Linux machine, CppunitTest_vcl_gtk3_a11y fails the



assert(m_pAtspiApp);


in Atspi2TestBase::setUp 
(vcl/qa/cppunit/a11y/atspi2/atspi2testbase.hxx), apparently because



Atspi::Accessible desktop(atspi_get_desktop(desktopId));


in Atspi2TestBase::getApp 
(vcl/qa/cppunit/a11y/atspi2/atspi2testbase.hxx) produces



** (cppunittester:1641471): WARNING **: 12:56:34.251: AT-SPI: Could not obtain 
desktop path or name


(cppunittester:1641471): dbind-WARNING **: 12:56:34.251: Couldn't get 
application list: Could not activate remote peer: unit failed.

(cppunittester:1641471): GLib-GObject-CRITICAL **: 12:56:34.251: 
g_object_unref: assertion 'G_IS_OBJECT (object)' failed


on stderr but returns an empty Atspi::Accessible with no children to 
iterate over.


Is there perhaps anything I would need to additionally install or to 
enable to make this work?  Or does it just not work on Wayland?

Re: [Libreoffice-qa] New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-08-01 19:30, Colomban Wendling wrote:
Platform accessibility (a11y) conformance tests for the GTK3 VCL [1] 
have been merged a couple days ago.  This is a set of tests validating 
that the a11y exposed to the platform on Linux using the GTK3 VCL is 
coherent with the internal representation in LO.  This can catch (and 
already had caught) issues in the VCL layer not correctly transmitting 
data over -- and even more [2].  You can read a bit more on the 
technical side of things on the wiki page [3].

[...]
[3] 
https://wiki.documentfoundation.org/Development/Accessibility_Unit_Tests#Platform_accessibility_tests


Thanks again, that's really great to have!

To mention it: Platform a11y tests are one of 3 ways that the a11y test 
framework written by Colomban provides to write Cppunit tests (or 
convert Java ones to C++). The other 2 are also described on the 
above-mentioned wiki page.


OpenPGP_signature
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-08-01 19:30, Colomban Wendling wrote:
Platform accessibility (a11y) conformance tests for the GTK3 VCL [1] 
have been merged a couple days ago.  This is a set of tests validating 
that the a11y exposed to the platform on Linux using the GTK3 VCL is 
coherent with the internal representation in LO.  This can catch (and 
already had caught) issues in the VCL layer not correctly transmitting 
data over -- and even more [2].  You can read a bit more on the 
technical side of things on the wiki page [3].

[...]
[3] 
https://wiki.documentfoundation.org/Development/Accessibility_Unit_Tests#Platform_accessibility_tests


Thanks again, that's really great to have!

To mention it: Platform a11y tests are one of 3 ways that the a11y test 
framework written by Colomban provides to write Cppunit tests (or 
convert Java ones to C++). The other 2 are also described on the 
above-mentioned wiki page.


OpenPGP_signature
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Michael Weghorn]

On 2023-08-03 16:58, Colomban Wendling wrote:
OK… that's already quite a lot.  As CI passes on that changes (well, the 
relevant ones passed, the pending ones are irrelevant for this change), 
what about merging it and seeing if that number goes down?


Sounds like a good way forward to me. I've merged
https://gerrit.libreoffice.org/c/core/+/155310 .


OpenPGP_signature
Description: OpenPGP digital signature

Re: New tests: GTK3 platform accessibility

[Colomban Wendling]

Le 03/08/2023 à 16:51, Stephan Bergmann a écrit :

On 8/3/23 16:37, Colomban Wendling wrote:
Do you have an error rate value?  So I know how many successful CI 
runs I need to suggest it did indeed improve something :)


Sorry, I at least cannot provide any useful numbers here.  I've noticed 
it both occasionally with the Jenkins builds of Gerrit patches of mine, 
and when browsing the  mailing 
list (see , 
where most of the "Tinderbox on Master for Linux-DbgUtil" reported in 
the last few days are due to this issue).


OK… that's already quite a lot.  As CI passes on that changes (well, the 
relevant ones passed, the pending ones are irrelevant for this change), 
what about merging it and seeing if that number goes down?

Re: New tests: GTK3 platform accessibility

[Stephan Bergmann]

On 8/3/23 16:37, Colomban Wendling wrote:
Do you have an error rate value?  So I know how many successful CI runs 
I need to suggest it did indeed improve something :)


Sorry, I at least cannot provide any useful numbers here.  I've noticed 
it both occasionally with the Jenkins builds of Gerrit patches of mine, 
and when browsing the  mailing 
list (see , 
where most of the "Tinderbox on Master for Linux-DbgUtil" reported in 
the last few days are due to this issue).

Re: New tests: GTK3 platform accessibility

[Colomban Wendling]

Le 03/08/2023 à 10:01, Stephan Bergmann a écrit :

On 8/1/23 19:30, Colomban Wendling wrote:
Linux CIs are running these tests, but anybody willing is encouraged 
to enable those locally and report any issue (don't hesitate to CC me).


That CppunitTest_vcl_gtk3_a11y appears to be a frequent cause of failure 
now on Jenkins, with the typically failure log containing some


[...]


Hum… I never hit that locally, but that looks like the CI machine could 
be overloaded somewhat, and just be very slow (normally the timeout 
should be 800+ms).  Anyway, I created 
https://gerrit.libreoffice.org/c/core/+/155310 which is a tentative 
workaround (increment the timeout), we can see if it still happen with this.


Do you have an error rate value?  So I know how many successful CI runs 
I need to suggest it did indeed improve something :)


Regards,
Colomban


found run attribute: MMToPixelRatio=
  matching atspi attribute is: mm-to-pixel-ratio=1.1
unknown:0:Test1::TestBody
An uncaught exception of type com.sun.star.uno.RuntimeException
- timeout from dbind at 
/home/tdf/lode/jenkins/workspace/lo_tb_master_linux_dbg/vcl/qa/cppunit/a11y/atspi2/atspiwrapper.hxx:302


warn:sw.core:19780:19780:sw/source/core/attr/calbck.cxx:196: a 
16SwTextFormatColl client added as listener to a 16SwTextFormatColl 
during client iteration.
warn:sw.core:19780:19780:sw/source/core/attr/calbck.cxx:196: a 
16SwLineNumberInfo client added as listener to a 12SwCharFormat during 
client iteration.

Test1::TestBody finished in: 153317ms
##Failure Location unknown## : Error
Test name: Test1::TestBody
An uncaught exception of type com.sun.star.uno.RuntimeException
- timeout from dbind at 
/home/tdf/lode/jenkins/workspace/lo_tb_master_linux_dbg/vcl/qa/cppunit/a11y/atspi2/atspiwrapper.hxx:302


Failures !!!
Run: 1   Failure total: 1   Failures: 0   Errors: 1

(cppunittester:19780): GLib-GObject-CRITICAL **: 20:29:18.296: 
g_object_unref: assertion 'G_IS_OBJECT (object)' failed


(cppunittester:19780): GLib-GObject-CRITICAL **: 20:29:18.296: 
g_object_unref: assertion 'G_IS_OBJECT (object)' failed


(cppunittester:19780): GLib-GObject-CRITICAL **: 20:29:18.296: 
g_object_unref: assertion 'G_IS_OBJECT (object)' failed

[...]

()

Re: New tests: GTK3 platform accessibility

[Stephan Bergmann]

On 8/1/23 19:30, Colomban Wendling wrote:
Linux CIs are running these tests, but anybody willing is encouraged to 
enable those locally and report any issue (don't hesitate to CC me).


That CppunitTest_vcl_gtk3_a11y appears to be a frequent cause of failure 
now on Jenkins, with the typically failure log containing some


[...]

found run attribute: MMToPixelRatio=
  matching atspi attribute is: mm-to-pixel-ratio=1.1
unknown:0:Test1::TestBody
An uncaught exception of type com.sun.star.uno.RuntimeException
- timeout from dbind at 
/home/tdf/lode/jenkins/workspace/lo_tb_master_linux_dbg/vcl/qa/cppunit/a11y/atspi2/atspiwrapper.hxx:302

warn:sw.core:19780:19780:sw/source/core/attr/calbck.cxx:196: a 
16SwTextFormatColl client added as listener to a 16SwTextFormatColl during 
client iteration.
warn:sw.core:19780:19780:sw/source/core/attr/calbck.cxx:196: a 
16SwLineNumberInfo client added as listener to a 12SwCharFormat during client 
iteration.
Test1::TestBody finished in: 153317ms
##Failure Location unknown## : Error
Test name: Test1::TestBody
An uncaught exception of type com.sun.star.uno.RuntimeException
- timeout from dbind at 
/home/tdf/lode/jenkins/workspace/lo_tb_master_linux_dbg/vcl/qa/cppunit/a11y/atspi2/atspiwrapper.hxx:302

Failures !!!
Run: 1   Failure total: 1   Failures: 0   Errors: 1

(cppunittester:19780): GLib-GObject-CRITICAL **: 20:29:18.296: g_object_unref: 
assertion 'G_IS_OBJECT (object)' failed

(cppunittester:19780): GLib-GObject-CRITICAL **: 20:29:18.296: g_object_unref: 
assertion 'G_IS_OBJECT (object)' failed

(cppunittester:19780): GLib-GObject-CRITICAL **: 20:29:18.296: g_object_unref: 
assertion 'G_IS_OBJECT (object)' failed

[...]

()

Re: New tests: GTK3 platform accessibility

[Samuel Thibault]

Colomban Wendling, le mar. 01 août 2023 19:30:05 +0200, a ecrit:
> Platform accessibility (a11y) conformance tests for the GTK3 VCL [1] have
> been merged a couple days ago.  This is a set of tests validating that the
> a11y exposed to the platform on Linux using the GTK3 VCL is coherent with
> the internal representation in LO.  This can catch (and already had caught)
> issues in the VCL layer not correctly transmitting data over -- and even
> more [2].

Yay, congrats!

Samuel

Re: New tests: GTK3 platform accessibility

[Caolán McNamara]

On Tue, 2023-08-01 at 19:30 +0200, Colomban Wendling wrote:
> Hello everyone,
> 
> Platform accessibility (a11y) conformance tests for the GTK3 VCL [1] 
> have been merged a couple days ago.  This is a set of tests
> validating that the a11y exposed to the platform on Linux using the
> GTK3 VCL is coherent with the internal representation in LO.

That's pretty impressive. Not at all easy to get that bootstrapped,
very nice.

[Libreoffice-qa] New tests: GTK3 platform accessibility

[Colomban Wendling]

Hello everyone,

Platform accessibility (a11y) conformance tests for the GTK3 VCL [1] 
have been merged a couple days ago.  This is a set of tests validating 
that the a11y exposed to the platform on Linux using the GTK3 VCL is 
coherent with the internal representation in LO.  This can catch (and 
already had caught) issues in the VCL layer not correctly transmitting 
data over -- and even more [2].  You can read a bit more on the 
technical side of things on the wiki page [3].


These tests are enabled by default if all dependencies are available. 
That is a working Xvfb [4], dbus-launch, and the AT-SPI2 library [5] 
which is used to communicate with the platform accessibility daemon.  It 
is also possible to either enable or disable them forcefully, using 
--enable-atspi-test and --disable-atspi-tests respectively.


Linux CIs are running these tests, but anybody willing is encouraged to 
enable those locally and report any issue (don't hesitate to CC me).


Anybody interested is also more than welcome to extend coverage of these 
tests!  At the moment only a subset of the a11y APIs are checked, yet it 
should not be very hard to add more.  If you're interested, give [3] a 
look and again, don't hesitate to get in touch with me.
If you're interested in making this work with another VCL (Qt6, GTK4, 
etc.) it would also be great, but probably require a tad more work.  It 
would however be very interesting, and would add coverage for another 
substantial module :)


You're also more than welcome to build on the concept for other 
platforms (e.g. Window or macos).  I expect those to look fairly 
similar, but require a wholly different platform wrapping layer 
(obviously they'd use something other than libatspi, and they might need 
to adapt to differences in the platform's expectations, although they 
are actually fairly similar).


Happy testing!
Regards,
Colomban

PS: huge thanks to everyone that helped me getting this finally done, 
including but not limited to testing, fixing blocker bugs, adjust CI 
environment, and everything else :)


[1] https://gerrit.libreoffice.org/c/core/+/153069
[2] e.g. https://bugs.documentfoundation.org/show_bug.cgi?id=155625
[3] 
https://wiki.documentfoundation.org/Development/Accessibility_Unit_Tests#Platform_accessibility_tests

[4] https://en.wikipedia.org/wiki/Xvfb
[5] https://docs.gtk.org/atspi2/index.html

New tests: GTK3 platform accessibility

[Colomban Wendling]

Hello everyone,

Platform accessibility (a11y) conformance tests for the GTK3 VCL [1] 
have been merged a couple days ago.  This is a set of tests validating 
that the a11y exposed to the platform on Linux using the GTK3 VCL is 
coherent with the internal representation in LO.  This can catch (and 
already had caught) issues in the VCL layer not correctly transmitting 
data over -- and even more [2].  You can read a bit more on the 
technical side of things on the wiki page [3].


These tests are enabled by default if all dependencies are available. 
That is a working Xvfb [4], dbus-launch, and the AT-SPI2 library [5] 
which is used to communicate with the platform accessibility daemon.  It 
is also possible to either enable or disable them forcefully, using 
--enable-atspi-test and --disable-atspi-tests respectively.


Linux CIs are running these tests, but anybody willing is encouraged to 
enable those locally and report any issue (don't hesitate to CC me).


Anybody interested is also more than welcome to extend coverage of these 
tests!  At the moment only a subset of the a11y APIs are checked, yet it 
should not be very hard to add more.  If you're interested, give [3] a 
look and again, don't hesitate to get in touch with me.
If you're interested in making this work with another VCL (Qt6, GTK4, 
etc.) it would also be great, but probably require a tad more work.  It 
would however be very interesting, and would add coverage for another 
substantial module :)


You're also more than welcome to build on the concept for other 
platforms (e.g. Window or macos).  I expect those to look fairly 
similar, but require a wholly different platform wrapping layer 
(obviously they'd use something other than libatspi, and they might need 
to adapt to differences in the platform's expectations, although they 
are actually fairly similar).


Happy testing!
Regards,
Colomban

PS: huge thanks to everyone that helped me getting this finally done, 
including but not limited to testing, fixing blocker bugs, adjust CI 
environment, and everything else :)


[1] https://gerrit.libreoffice.org/c/core/+/153069
[2] e.g. https://bugs.documentfoundation.org/show_bug.cgi?id=155625
[3] 
https://wiki.documentfoundation.org/Development/Accessibility_Unit_Tests#Platform_accessibility_tests

[4] https://en.wikipedia.org/wiki/Xvfb
[5] https://docs.gtk.org/atspi2/index.html