Andrew has raised that concern as well. My gut says that the bundling of a
few shorts scripts that get parsed but not run as long as they don't get
require() will not affect applications negatively (there are probably many
more significant overheads we live with today in cordova) -- but I
understand why that may not be useful default.
In that case, some ideas: (I recall these were proposed previously but not
sure by whom)
(a) Bundle tests as a plugin-within-the-plugin as such:
myplugin/
- plugin.xml
- src/...
- www/...
- tests/
- plugin.xml
- www/...
Which basically means the plugin tests live in the same repo/branch, and
are fetched as part of "plugin add", but are not moved into platforms/ on
cordova prepare by default, thus don't end up in your application (disk and
network are cheap, application startup and size costs are not, right?).
Then, to run tests, we basically need to iterate all plugins looking for a
nested tests/plugin.xml, and install those. This can be added to
CLI/Plugman, or just be a CLI hook even.
(b) add a <js-test-module> or <js-module type="test"> that is only used if
you run prepare with --test. Similar to the above, but I think requires
more CLI/config file changes, which I'm not a fan of.
(c) Just ship tests as a second plugin in a second repo, and document how
to install tests. Can then perhaps have a <dependency type=tests>. I
don't like this as much since its basically back to mobile-spec.
WDYT?
-Michal
On Thu, Jun 19, 2014 at 4:53 PM, Jesse <purplecabb...@gmail.com> wrote:
> re:
>
> Q: What do I do if my plugin tests must have very large assets?
> - A: Don't bundle those assets with your plugin. If you can, have your
> ...
>
>
> My concern is I do not want to see tests added to every project that uses a
> plugin, even if the assets are not large, there are implications to
> including the test framework + all the tests because they get loaded and
> processed with all of the plugins and will impact load time even if never
> run.
>
> 99.9% of the time the plugin tests will be used by us the plugin
> developers, and not the people who use the plugin in there apps.
>
> I agree, having the tester install the test harness plugin dependency
> themselves is probably a better option, as I see you have wrapped all tests
> inside a exports.defineAutoTests so we don't have to worry about
> describe/it/expects not being defined.
>
>
> @purplecabbage
> risingj.com
>
>
> On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmo...@chromium.org> wrote:
>
> > Jesee, the branch is NOT a requirement (I don't think I documented it as
> > such, except in the examples for installing plugins for initial look).
> > Actually, we should delete those stale branches now that we are moving
> > up-to-date tests into master. It was just for early experimentation on
> the
> > feature.
> >
> > Jesse, I'm not seeing the benefit of using plugins-tests.xml or the
> > dependency on the test plugin yet, may you elaborate your thoughts?
> >
> > My hope was that tests are just always installed alongside plugins. If
> > that is not a good idea for some particular plugin, say because it uses
> > huge assets, I elaborated my answer in the plugin FAQ (
> >
> >
> https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq
> > ):
> > FAQ
> >
> > -
> >
> > Q: Should I add org.apache.cordova.test-harness as a <dependancy> of
> my
> > plugin?
> > - A: No. The end-user should decide if they want to install the test
> > harness, not your plugin (most users won't).
> > -
> >
> > Q: What do I do if my plugin tests must have very large assets?
> > - A: Don't bundle those assets with your plugin. If you can, have your
> > tests fail gracefully if those assets don't don't exist (perhaps
> log
> > a
> > warning, perhaps fail a single asset-checking test, and skip the
> > rest).
> > Then, ideally download those assets automatically into local
> storage
> > the
> > first time tests run. Or create a manual test step to download
> > and install
> > assets. As a final alternative, split those test assets into a
> > separate
> > plugin, and instruct users to install that plugin to run your full
> > test
> > suite.
> > -
> >
> > Q: Should I ship my app with the test harness plugin installed?
> > - A: Not likely. If you want, you can. Then your app could even embed
> a
> > link to the test page (cdvtests/index.html) from a help section of
> > your app, to give end users a way to run your test suite out in
> > the feild.
> > That may help diagnose causes of issues within your app. Maybe.
> >
> >
> > =============
> >
> >
> > Feel free the debate those answers -- now is certainly the time -- but I
> > put a lot of effort to make it super flexible and to not require
> depending
> > on changes to CLI :P
> >
> > -Michal
> >
> >
> > On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabb...@gmail.com> wrote:
> >
> > > Sorry I missed providing feedback on this earlier ...
> > > Having a deeper look at this, I am not feeling great about the extra
> > > requirement that every plugin have an additional branch.
> > >
> > > Several concerns arise :
> > > - test branch can be out of sync with master
> > > - how do we test a specific version?
> > > - tests are not immediately visible when looking at master
> > > - differing versions of plugin.xml depending on the branch
> > >
> > > The majority of the work has been done (thanks Michal!), and mostly any
> > > suggestions I make will just require moving code and changing a few
> > > conventions.
> > >
> > > What if we? :
> > > 1. add a plugin-test.xml file which has the exact format of plugin.xml
> > > 2. keep tests/ and plugin-test.xml file in master branch
> > > 3. have plugman/cli support an additional flag --test so we can install
> > > like this:
> > > cordova plugin add
> > > http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git
> --test
> > > This would mean that in addition to processing the plugin.xml of the
> > > plugin, we would also process the plugin-test.xml file ( identical
> > > processing logic )
> > > 4. have all plugin-test.xml files declare a dependency on
> > > cordova-plugin-test-framework
> > >
> > > The above suggestions could also be used in conjuction with the cordova
> > run
> > > --tests platform mentioned by Michal, but without the need to manage
> the
> > > switching of branches.
> > >
> > >
> > > @purplecabbage
> > > risingj.com
> > >
> > >
> > > On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny <mmo...@chromium.org>
> > wrote:
> > >
> > > > Piotr: Actually I'm not sure how running tests in the harness would
> > work,
> > > > since the path to the resource may be different. However, in
> general,
> > > with
> > > > development using the harness you aren't making any changes to
> plugins.
> > > > The whole point is for app developers who want to modify only web
> > > > application bits and not deal with native compiles.
> > > >
> > > > In theory the app harness could support working on the js-modules of
> > > > plugins, but that sounds like a really niche idea. I'd not be
> opposed
> > to
> > > > someone working on it but I'm not sure you'll have luck finding
> > > volunteers.
> > > >
> > > > -Michal
> > > >
> > > >
> > > > On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny <mmo...@chromium.org>
> > > wrote:
> > > >
> > > > > At the time I went through my design iterations I just didn't want
> to
> > > > > necessarily depend on cordova tooling changes / documentation. In
> > > other
> > > > > words, someone else may have a different strategy for testing..
> > > > >
> > > > > My personal opinion would be have the test plugin ship with a
> plugin
> > > hook
> > > > > (those are in, right? or at least on their way), that will
> > > automatically
> > > > > update the start page if you pass a flag to run command. Ie, in an
> > > ideal
> > > > > world: `cordova run --tests` runs a plugin hook passing in --tests
> > > flag
> > > > > which changes the start page, in a way that isn't overwritten by
> the
> > > > > top-level config.xml.
> > > > >
> > > > > My 2 cents, since I don't want "our way" of testing mobile spec to
> be
> > > > "the
> > > > > only way" to test. Frameworks and opinions on testing change.
> > > > >
> > > > > -Michal
> > > > >
> > > > >
> > > > > On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa <pzal...@mozilla.com
> >
> > > > wrote:
> > > > >
> > > > >> One thing more - it would be great if user could create a test
> using
> > > > test
> > > > >> harness app as well. Is it also considered?
> > > > >>
> > > > >> Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez pisze:
> > > > >>
> > > > >> It would be a nice to have in the cli, aimed to just setup the
> > right
> > > > path
> > > > >>> in the config.xml, maybe along with an another argument to build,
> > > > >>> run/emulate as well.
> > > > >>> It sounds great.
> > > > >>>
> > > > >>>
> > > > >>> 2014-06-17 15:21 GMT-05:00 Piotr Zalewa <pzal...@mozilla.com>:
> > > > >>>
> > > > >>> Thanks Martin,
> > > > >>>>
> > > > >>>> Has it been considered to create a separate command "testrun" or
> > > > similar
> > > > >>>> which would remove the need to edit the config.xml?
> > > > >>>>
> > > > >>>> Dnia Tue Jun 17 11:58:33 2014 Michal Mocny pisze:
> > > > >>>>
> > > > >>>> Martin, thanks for posting those links.
> > > > >>>>
> > > > >>>>>
> > > > >>>>> And I'll look into the INFRA tickets I need to file to set up a
> > > repo
> > > > >>>>> for
> > > > >>>>> that plugin, since its ready to come out of labs.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Tue, Jun 17, 2014 at 2:06 PM, Martin Gonzalez <
> > > > >>>>> martin.c.glez.g...@gmail.com> wrote:
> > > > >>>>>
> > > > >>>>> This is the Cordova Plugin Test Framework readme.md, you can
> > > catch
> > > > >>>>> up
> > > > >>>>>
> > > > >>>>>> with
> > > > >>>>>> the functionality by reading some of the content:
> > > > >>>>>>
> > > > >>>>>> Repository:
> > > > >>>>>> https://github.com/apache/cordova-labs
> > > > >>>>>>
> > > > >>>>>> Docs:
> > > > >>>>>> https://github.com/apache/cordova-labs/blob/master/README.md
> > > > >>>>>>
> > > > >>>>>> https://github.com/apache/cordova-labs/blob/cdvtest/
> > > > >>>>>> cordova-plugin-test-framework/README.md
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> 2014-06-17 12:56 GMT-05:00 Piotr Zalewa <pzal...@mozilla.com
> >:
> > > > >>>>>>
> > > > >>>>>> a documentation explaining how it's gonna work
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Dnia Tue Jun 17 10:51:58 2014 Michal Mocny pisze:
> > > > >>>>>>>
> > > > >>>>>>> What do you mean?
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Tue, Jun 17, 2014 at 1:50 PM, Piotr Zalewa <
> > > > pzal...@mozilla.com>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> Is there any predev document?
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> Dnia Mon Jun 16 18:30:46 2014 Andrew Grieve pisze:
> > > > >>>>>>>>>
> > > > >>>>>>>>> Yeah, really exciting. Thanks for taking this on.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>> On Mon, Jun 16, 2014 at 3:42 PM, Michal Mocny <
> > > > >>>>>>>>>> mmo...@chromium.org>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Fantastic!
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I'll try to keep an eye out on the PR's, and please ping
> me
> > > if
> > > > >>>>>>>>>>> you
> > > > >>>>>>>>>>> would
> > > > >>>>>>>>>>> like any help.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> -Michal
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> On Mon, Jun 16, 2014 at 3:25 PM, Marcel Kinard <
> > > > >>>>>>>>>>> cmarc...@gmail.com>
> > > > >>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Hi, after some discussions here with IBM management,
> > > we’re
> > > > >>>>>>>>>>> going
> > > > >>>>>>>>>>> to
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> bring
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> in a couple extra interns for a week to jumpstart the
> > > > migration
> > > > >>>>>>>>>>>> of
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> the
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>> tests out of mobile-spec into the new plugin test framework.
> > > Staci
> > > > >>>>>>>
> > > > >>>>>>>> Cooper
> > > > >>>>>>>>>>>> will be leading this effort, and Martin Gonzalez will
> be a
> > > > part
> > > > >>>>>>>>>>>> of
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> it.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>
> > > > >>>>>>> So if you see a bunch of pull requests, this is what it is
> > for.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> We’ll
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>> get
> > > > >>>>>>>
> > > > >>>>>>>> the interns to submit an ICLA asap.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>> --
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Piotr Zalewa
> > > > >>>>>>>>> Mozilla
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> --
> > > > >>>>>>>>
> > > > >>>>>>> Piotr Zalewa
> > > > >>>>>>> Mozilla
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> Regards,
> > > > >>>>>> Martin Gonzalez
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>> --
> > > > >>>> Piotr Zalewa
> > > > >>>> Mozilla
> > > > >>>>
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >> --
> > > > >> Piotr Zalewa
> > > > >> Mozilla
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
