On Fri, Feb 27, 2015 at 5:30 PM, Dmitry Blotsky <dblot...@microsoft.com> wrote:
> Thanks for the feedback, Michal! The issue we had with using the > checkboxes was that it was not easily automatable. A more simple solution > was to use a parameter to specify the plugins you want installed, instead > of installing all of them and then disabling them. Also the > dependencies-plugin in mobilespec is a hardcoded list, which requires more > manipulation to modify than specifying a list as a parameter. Using it as > the default is a great idea, but being able to override it during > invocation would be great too I think. What do you think? > I'm all for improving the tooling around this. Frankly that was the goal from the get go, and lots of teams have independently improved tests over time to fit their needs. Please feel free to experiment. Options: - At runtime, there is a window.tests object that holds metadata about all the plugins & tests, and it includes an "enabled" flag. This is what the checkboxes are modifying. Setting the initial "enabled" values on startup using some form of config (a file perhaps, or perhaps a <preference> that is checked by the tests plugin). - At compile time, the createmobilespec.js script already maintains a list of plugins (e.g. CORDOVA_REGISTRY_PLUGINS, TELERIK_VERIFIED_PLUGINS), and the installPlugins function installs all nested tests automatically. You could change the script to accept a list (command line, file, ...) or turn the script into a node module that accepts an arbitrary list of plugins and write different scripts in terms of it, etc. The dependency plugin can easily be replaced by moving the list directly into the createmobilespec.js script, making it easier to configure. Totally open to modifications here, please send PR if you are willing! > > Also: Jesse, and others, do you think it might be worthwhile to > collaborate and incorporate the features of paramedic (like specifying the > plugins to run) into mobilespec? That way we will have one tool for testing > throughout. I've definitely been sold on the idea of having mobilespec be > the main point for running tests. That way we can use it in all our testing > scenarios: local, Medic, Travis, etc. > > Kindly, > Dmitry > > -----Original Message----- > From: mmo...@google.com [mailto:mmo...@google.com] On Behalf Of Michal > Mocny > Sent: Wednesday, February 25, 2015 12:42 PM > To: dev > Subject: Re: Cordova Automated Testing > > I haven't followed this full thread, so sorry if this is out of context, > but wanted to point out: > > - Jason added support for testing a subset of plugins in mobile-spec by > way of checkboxes in the automated test runner. It defaults to all, but > you can change that. > - By design, the cordova-plugin-test-framework (and thus mobile-spec) will > only run tests for the plugins you have actually installed (actually, for > the nested plugin tests you have explicitly installed). The > createmobilespec.js script will by default install all core plugins, but > you can easily change that. > > -Michal > > On Wed, Feb 25, 2015 at 3:20 PM, Dmitry Blotsky <dblot...@microsoft.com> > wrote: > > > Thanks for the feedback, Jesse and Dmitriy! > > > > Judging from the responses, it definitely sounds like modifying > > mobilespec, or just plain using paramedic would be the way to go. I'm > > very much swinging in that direction. Medivac really only exists on > > its own because I wasn't comfortable enough with our git repos to make > > a branch off of mobilespec; because it really is just mobilespec with > > some things stripped away. > > > > Jesse: to me the primary value add was actually just simplicity. This > > guy doesn't depend on plugin-test-framework and has none of the > > heavier features like local storage and test segmentation. It's > > basically just a Cordova app with a single page to run Jasmine tests > > and an optional URI to post the results to, and the value is > > simplicity. It has very little code and has very few moving parts. > > > > Dmitriy Barkalov: I don't see what the use case is for adding > > plugin-test-framework to any Cordova app other than a blank one, and > > that is basically just doing manually what mobilespec does automatically. > > > > Kindly, > > Dmitry > > > > -----Original Message----- > > From: Jesse [mailto:purplecabb...@gmail.com] > > Sent: Wednesday, February 25, 2015 11:36 AM > > To: dev@cordova.apache.org > > Subject: Re: Cordova Automated Testing > > > > Thanks for contributing, however, I do not see where the value add is > > in this. > > mobile-spec does much of this already, as does > > cordova-plugin-test-framework. > > > > Adding an argument to test an individual plugin is a good idea, I > > would definitely support adding this to mobile-spec. > > > > My own cordova-paramedic is also designed for simplified plugin > > testing, so you can test an individual plugin, and see the results > > back on the command line. Primarily this was intended for CI with > > Travis, so we could see immediately if a plugin pull request breaks a > test on a platform. > > > > We can discuss further here, but I think medivac would be better > > served as a fork+pull request to cordova-medic > > > > gh:purplecabbage/cordova-paramedic > > > > > > @purplecabbage > > risingj.com > > > > On Wed, Feb 25, 2015 at 9:35 AM, Dmitriy Barkalov (Akvelon) < > > v-dmb...@microsoft.com> wrote: > > > > > Dmitry, > > > > > > I like your idea of adding tests from different formats. I.e. you > > > search for standard Cordova plugin tests like in core plugins and > > > for non-standard tests like plane test.js file. I think it is a good > > > point to provide a way for plugin authors to either reformat their > > > tests to existing format or provide new test format parser. > > > Unfortunately, we can't run away from requirement to use Jasmin 2.0 > > > but still it is more > > freedom. > > > > > > It also seems that in many ways your application is doing the same > > > things as plugin testing framework. It provides a page to run tests, > > > holds Jasmine and puts results into CouchDB. But it is less flexible > > > because cannot be included into other application, while any Cordova > > > application can install plugin testing framework and get a page to > > > run > > manual and automated tests. > > > > > > So I vote for porting some of the new features from this tool back > > > to plugin testing framework and letting developers reuse this > functionality. > > > > > > > > > Regards, Dmitriy > > > > > > -----Original Message----- > > > From: Dmitry Blotsky [mailto:dblot...@microsoft.com] > > > Sent: Wednesday, February 25, 2015 11:47 AM > > > To: dev@cordova.apache.org > > > Subject: Re: Cordova Automated Testing > > > > > > Hi Shazron, > > > > > > Thank you for your feedback! It actually doesn’t work on iOS because > > > it seems like the most recent whitelist changes broke things. I ran > > > "npm > > test" > > > for the code in master and some of them fail because of whitelisting. > > > That’s just my intuition though: I don’t know exactly why it’s not > > > working yet. > > > > > > Regarding reporting though, reporting to localhost is actually just > > > the default - you can pass a host and port on the CLI and it will > > > report to a CouchDB server like mobilespec did. There is an > > > immediate next step in my development efforts to make it not tied to > > > CouchDB, or to at least allow the user to specify the URI format for > reporting. > > > Also, thanks for pointing out the ‘npm install’ requirement. I added > > > it to the docs. The reporting to console also happens, but it’s > > currently just limited to console.log. > > > Medivac uses all the standard Jasmine reporters (HTML, console, and > > JSAPI). > > > > > > And indeed, I will definitely drop a line to Jesse. Is it > > > well-received in the Apache community to schedule meetings to > > > discuss details, or should detailed discussions also be happening on > > > the mailing > > list? > > > > > > Kindly, > > > Dmitry > > > > > > > On Feb 24, 2015, at 3:32 PM, Shazron <shaz...@gmail.com> wrote: > > > > > > > > Also, the tool does not work on the device (I tried iOS), unlike > > > > mobile-spec since it's trying to connect to localhost for reporting. > > > > > > > > On Mon, Feb 23, 2015 at 4:30 PM, Dmitry Blotsky > > > > <dblot...@microsoft.com> > > > wrote: > > > >> Hi list, > > > >> > > > >> Over the past few weeks, I've developed an automated > > > >> mobilespec-like > > > tool in the process of resuscitating Cordova's CI. It's tentatively > > > called cordova-medivac, and it can currently be found here: > > > http://github.com/dblotsky/cordova-medivac. I'd like to pitch it to > > > the community as a potential alternative or replacement for > > > mobilespec. There have been several tools in the past few weeks on > > > the list, and I've tried to incorporate the features they have into > > > this > > tool. > > > >> > > > >> It contains the following new features: > > > >> > > > >> - Testing any list of plugins at a time - not just the core > > > ones; they're passed as a CLI argument > > > >> > > > >> - Support for "new-style" plugin-test-framework-only tests, > > as > > > well just pure Jasmine spec files that don't use test-framework > > > >> > > > >> - No manual interaction required - it is very basic and > just > > > runs the Jasmine tests as soon as it starts > > > >> > > > >> - Very little code - it basically just creates a simple > dummy > > > app with plugins (like mobilespec) > > > >> > > > >> - Cleaner code - some refactoring was done to remove a few > > > code smells > > > >> > > > >> - No flaky UI > > > >> > > > >> - Optionally reporting test results to a URI (currently > > > assumed to be a CouchDB server) > > > >> > > > >> Current shortcomings: > > > >> > > > >> - The special plugins inside cordova-mobile-spec that > contain > > > tests have not yet been copied over > > > >> > > > >> - The code is similar, so some functionality is not exactly > > > the same, so it's not 100% compatible with mobilespec > > > >> > > > >> - Some error-checking is still missing (like verifying that > > > CLI/JS/lib are locally linked instead of global) > > > >> > > > >> - Cordova-JS Grunt build task is not being automatically > run > > > on creation > > > >> > > > >> - Barebones hard-coded list of "core" plugins; doesn't > > include > > > convenience lists like the new ones in mobilespec > > > >> > > > >> - Doesn't support current manual tests > > > >> > > > >> I'd like to ask you if you would support any of the following > > > >> proposed > > > ideas: > > > >> > > > >> - Reach 100% parity and replace cordova-mobile-spec with > this > > > tool or, > > > >> > > > >> - If not replace, then port some of the code cleanup and > new > > > features from this tool back to mobilespec or, > > > >> > > > >> - Use this tool for the CI (cordova-medic) only, and keep > > > using mobilespec for other tasks > > > >> > > > >> Kindly, > > > >> Dmitry > > > >> > > > > > > > > ------------------------------------------------------------------ > > > > -- > > > > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > > > > For additional commands, e-mail: dev-h...@cordova.apache.org > > > > > > > > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > > > For additional commands, e-mail: dev-h...@cordova.apache.org > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > > > For additional commands, e-mail: dev-h...@cordova.apache.org > > > > > >
