Having a known failure in the tests on wp7 is no biggie, it has always been there. Just move on ...
Sent from my iPhone > On Oct 26, 2013, at 3:24 PM, Michal Mocny <[email protected]> wrote: > > We have a proposal and prototype on the table right now for re-working > tests to ship with plugins, defined according to auto and manual tests. > > To accomplish what you ask for would require a specialized testing app that > simply runs both at the same time. (this wouldn't be the default, but would > be easy to make). > > Thus, I think the tests shouldn't be modified (its hard to state at test > definition time in which fashion they should be used), the test runner > should. This wont solve the problem today, but perhaps in about a month it > could. > > -Micha > > > On Sat, Oct 26, 2013 at 6:41 AM, Sergey Grebnov (Akvelon) < > [email protected]> wrote: > >> Hi Michael, >> >> Agree. But taking into account having a way to run all the tests >> (including ones w/ user interaction) is very useful for Windows Phone I >> propose the following >> 1. No changes for non-WP platforms >> 2. For WP >> a) Use the following condition for the tests which require user >> interaction >> define(..., function(...) { >> if (isWP8 && !runAll) return; >> expect(...); >> ... >> }) >> b) Current autotests will run w/o runAll option so won't require user >> interaction >> c) Add 'Run All Tests (Extended)' option specifically for WP8 where we >> will have runAll == true >> >> Motivation: >> 1. I don't think we should move such tests to manual tests for WP only to >> be consistent with other platforms - we actually test api call and check >> result >> 2. By default all tests will run w/o any user interaction >> 3. We will have an option to quickly check all api before release via Run >> All Tests (Extended). In other case we should have special information how >> to check all the api and not to forget to run such special tests. >> >> Thx! >> Sergey >> -----Original Message----- >> From: [email protected] [mailto:[email protected]] On Behalf Of Michal >> Mocny >> Sent: Saturday, October 26, 2013 4:12 AM >> To: dev >> Subject: Re: Mobile spec tests and exclusion list >> >> Auto tests should run automatically without intervention. If user actions >> is needed for test to pass, we should call that something different (manual >> tests have been used). >> >> I think some varient of #3 is fine, this isn't a common problem. I >> wouldn't even test for Medic specifically, since I want my auto tests to >> run automatically even when testing by hand. >> >> define(..., function(...) { >> if (isWP8) return; >> expect(...); >> ... >> }) >> >> -Michal >> >> >> On Fri, Oct 25, 2013 at 4:37 PM, Sergey Grebnov (Akvelon) < >> [email protected]> wrote: >> >>> Mobile spec autotests include tests which on some platforms require >>> user interaction to complete. For example, contact save api on Windows >>> Phone requires user to manually click on save button. This prevents >>> the tests to be run as part of Medic test automation since test app >>> just hangs on such api calls. >>> >>> Is Windows Phone special or there are similar problem on other platforms? >>> >>> I'm thinking about the following possible approaches: >>> #1 Ad-hoc solution to Medic - replacing some test files as part of >>> Medic functionality (some additional wp specific build step). >>> #2 Extending mobile spec functionality- adding something like tests >>> exclusion config where you can define test ids (or even the whole api) >>> to be skipped. Such exclusion list could be generated on the fly and >>> put to the app before starting tests. >>> #3 If there are only few such tests we can probably add check for the >>> current platform to determine whether to include the test. For example: >>> if(!(window.MedicTestRunner && isWP8)) {testDefinition} Or the same >>> way but inside the test to fail gracefully. >>> >>> Thoughts? >>> >>> Thx! >>> Sergey >>
