Hi Michael,
Agree. But taking into account having a way to run all the tests (including
ones w/ user interaction) is very useful for Windows Phone I propose the
following
1. No changes for non-WP platforms
2. For WP
a) Use the following condition for the tests which require user interaction
define(..., function(...) {
if (isWP8 && !runAll) return;
expect(...);
...
})
b) Current autotests will run w/o runAll option so won't require user
interaction
c) Add 'Run All Tests (Extended)' option specifically for WP8 where we will
have runAll == true
Motivation:
1. I don't think we should move such tests to manual tests for WP only to be
consistent with other platforms - we actually test api call and check result
2. By default all tests will run w/o any user interaction
3. We will have an option to quickly check all api before release via Run All
Tests (Extended). In other case we should have special information how to check
all the api and not to forget to run such special tests.
Thx!
Sergey
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Michal Mocny
Sent: Saturday, October 26, 2013 4:12 AM
To: dev
Subject: Re: Mobile spec tests and exclusion list
Auto tests should run automatically without intervention. If user actions is
needed for test to pass, we should call that something different (manual tests
have been used).
I think some varient of #3 is fine, this isn't a common problem. I wouldn't
even test for Medic specifically, since I want my auto tests to run
automatically even when testing by hand.
define(..., function(...) {
if (isWP8) return;
expect(...);
...
})
-Michal
On Fri, Oct 25, 2013 at 4:37 PM, Sergey Grebnov (Akvelon) <
[email protected]> wrote:
> Mobile spec autotests include tests which on some platforms require
> user interaction to complete. For example, contact save api on Windows
> Phone requires user to manually click on save button. This prevents
> the tests to be run as part of Medic test automation since test app
> just hangs on such api calls.
>
> Is Windows Phone special or there are similar problem on other platforms?
>
> I'm thinking about the following possible approaches:
> #1 Ad-hoc solution to Medic - replacing some test files as part of
> Medic functionality (some additional wp specific build step).
> #2 Extending mobile spec functionality- adding something like tests
> exclusion config where you can define test ids (or even the whole api)
> to be skipped. Such exclusion list could be generated on the fly and
> put to the app before starting tests.
> #3 If there are only few such tests we can probably add check for the
> current platform to determine whether to include the test. For example:
> if(!(window.MedicTestRunner && isWP8)) {testDefinition} Or the same
> way but inside the test to fail gracefully.
>
> Thoughts?
>
> Thx!
> Sergey
>
