I spoke with Bill briefly about the dialer app failures, and what I
wrote on the terminal app bug applies here as well.

https://bugs.launchpad.net/ubuntu-terminal-app/+bug/1284594/comments/4

The TLDR, is tests need to be improved to fail well; that is it should
be obvious what failed when a failure occurs.

A statenotfound error from Autopilot simply means the object in question
didn't exist when you went to look for it. They often pop-up because of
timing issues. Likely the test failed before this point and you didn't
realize it. For instance a view didn't appear (app issue or platform
/qmlscene issue), a click didn't register (possible app bug or platform
bug, or potentially autopilot bug if click events aren't working), a UI
element wasn't found (app bug, test bug, or platform bug), or a UI
element wasn't able to be introspected (autopilot bug).

In the scenarios I mentioned above the issue is not not usually
autopilot. And indeed I would venture the bugs we see are app, platform,
or testcase bugs. Most certainly all of them have some sort of testcase
bug as the test is likely not failing well as mentioned above, leaving
this ambiguity on what went wrong and what the issue is.

So these issues are 1) Almost certainly not autopilot bugs, 2)
Definitely are testcase bugs to the extent the tests aren't failing
well, 3) Potentially the application, the test, or platform bugs.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1284316

Title:
  multiple StateNotFoundError failures in image 206

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/autopilot/+bug/1284316/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to