I am trying to improve the UI automation framework with the goal of
reducing disabled UI test from 42 to 10 and running time (xp release)
from 400s to 200s. Following is a list of issues I found so far (I am
still studying them):
1. Many UI commands are executed asynchronously, and test client
sleeps and polls (not always polls actually) the result. This process
is slow and error prone. And we need different timeout for release,
debug, and purify.
2. The mechanism to send command to certain window/tab is not
reliable. For example, I found GetLastActiveBrowserWindow and
ActivateTab don't work as intended.
3. UI_tests.exe, crash_service.exe, and chrome.exe share the same
mutex when writing log file. This makes things slow. In addition, when
a process crashes/shutdowns while writing log files, other process
will get WAIT_ABANDONED when waiting for mutex and this leads to an
infinite loop.
4. If Chrome crashes when running the test on buildbot, we can not
investigate the cause. This might be exact the situation we'd like the
test to catch but we have to disable the test most of the time.
The biggest pain I have is that when UI tests fail/crash on buildbot
or try server, we have little information to investigate. I would
image lots of people faced similar situation in the past and ended up
with disabling the test. I'd think we need to address this issue first
before making good improvement on UI tests. If we have good debugging
infrastructure in place, fixing UI flakiness is much easier. As a
minimal, we need to save crash dumps. Any thoughts on this?
Huan
--~--~---------~--~----~------------~-------~--~----~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~----------~----~----~----~------~----~------~--~---
