Option a) was what I suggested a ways back, and I still stand by it. I think it provides the greatest transparency, and simplicity, yet it is still very flexible. I don't think it would be hard to accomplish either. This is the small re-org I was hinting at, you've already done the hard part.
@purplecabbage risingj.com On Thu, Jun 19, 2014 at 3:45 PM, Michal Mocny <mmo...@chromium.org> wrote: > Andrew has raised that concern as well. My gut says that the bundling of a > few shorts scripts that get parsed but not run as long as they don't get > require() will not affect applications negatively (there are probably many > more significant overheads we live with today in cordova) -- but I > understand why that may not be useful default. > > In that case, some ideas: (I recall these were proposed previously but not > sure by whom) > (a) Bundle tests as a plugin-within-the-plugin as such: > myplugin/ > - plugin.xml > - src/... > - www/... > - tests/ > - plugin.xml > - www/... > > Which basically means the plugin tests live in the same repo/branch, and > are fetched as part of "plugin add", but are not moved into platforms/ on > cordova prepare by default, thus don't end up in your application (disk and > network are cheap, application startup and size costs are not, right?). > Then, to run tests, we basically need to iterate all plugins looking for a > nested tests/plugin.xml, and install those. This can be added to > CLI/Plugman, or just be a CLI hook even. > > (b) add a <js-test-module> or <js-module type="test"> that is only used if > you run prepare with --test. Similar to the above, but I think requires > more CLI/config file changes, which I'm not a fan of. > (c) Just ship tests as a second plugin in a second repo, and document how > to install tests. Can then perhaps have a <dependency type=tests>. I > don't like this as much since its basically back to mobile-spec. > > WDYT? > > -Michal > > > On Thu, Jun 19, 2014 at 4:53 PM, Jesse <purplecabb...@gmail.com> wrote: > > > re: > > > > Q: What do I do if my plugin tests must have very large assets? > > - A: Don't bundle those assets with your plugin. If you can, have your > > ... > > > > > > My concern is I do not want to see tests added to every project that > uses a > > plugin, even if the assets are not large, there are implications to > > including the test framework + all the tests because they get loaded and > > processed with all of the plugins and will impact load time even if never > > run. > > > > 99.9% of the time the plugin tests will be used by us the plugin > > developers, and not the people who use the plugin in there apps. > > > > I agree, having the tester install the test harness plugin dependency > > themselves is probably a better option, as I see you have wrapped all > tests > > inside a exports.defineAutoTests so we don't have to worry about > > describe/it/expects not being defined. > > > > > > @purplecabbage > > risingj.com > > > > > > On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmo...@chromium.org> > wrote: > > > > > Jesee, the branch is NOT a requirement (I don't think I documented it > as > > > such, except in the examples for installing plugins for initial look). > > > Actually, we should delete those stale branches now that we are moving > > > up-to-date tests into master. It was just for early experimentation on > > the > > > feature. > > > > > > Jesse, I'm not seeing the benefit of using plugins-tests.xml or the > > > dependency on the test plugin yet, may you elaborate your thoughts? > > > > > > My hope was that tests are just always installed alongside plugins. If > > > that is not a good idea for some particular plugin, say because it uses > > > huge assets, I elaborated my answer in the plugin FAQ ( > > > > > > > > > https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq > > > ): > > > FAQ > > > > > > - > > > > > > Q: Should I add org.apache.cordova.test-harness as a <dependancy> of > > my > > > plugin? > > > - A: No. The end-user should decide if they want to install the test > > > harness, not your plugin (most users won't). > > > - > > > > > > Q: What do I do if my plugin tests must have very large assets? > > > - A: Don't bundle those assets with your plugin. If you can, have > your > > > tests fail gracefully if those assets don't don't exist (perhaps > > log > > > a > > > warning, perhaps fail a single asset-checking test, and skip the > > > rest). > > > Then, ideally download those assets automatically into local > > storage > > > the > > > first time tests run. Or create a manual test step to download > > > and install > > > assets. As a final alternative, split those test assets into a > > > separate > > > plugin, and instruct users to install that plugin to run your > full > > > test > > > suite. > > > - > > > > > > Q: Should I ship my app with the test harness plugin installed? > > > - A: Not likely. If you want, you can. Then your app could even > embed > > a > > > link to the test page (cdvtests/index.html) from a help section > of > > > your app, to give end users a way to run your test suite out in > > > the feild. > > > That may help diagnose causes of issues within your app. Maybe. > > > > > > > > > ============= > > > > > > > > > Feel free the debate those answers -- now is certainly the time -- but > I > > > put a lot of effort to make it super flexible and to not require > > depending > > > on changes to CLI :P > > > > > > -Michal > > > > > > > > > On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabb...@gmail.com> > wrote: > > > > > > > Sorry I missed providing feedback on this earlier ... > > > > Having a deeper look at this, I am not feeling great about the extra > > > > requirement that every plugin have an additional branch. > > > > > > > > Several concerns arise : > > > > - test branch can be out of sync with master > > > > - how do we test a specific version? > > > > - tests are not immediately visible when looking at master > > > > - differing versions of plugin.xml depending on the branch > > > > > > > > The majority of the work has been done (thanks Michal!), and mostly > any > > > > suggestions I make will just require moving code and changing a few > > > > conventions. > > > > > > > > What if we? : > > > > 1. add a plugin-test.xml file which has the exact format of > plugin.xml > > > > 2. keep tests/ and plugin-test.xml file in master branch > > > > 3. have plugman/cli support an additional flag --test so we can > install > > > > like this: > > > > cordova plugin add > > > > http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git > > --test > > > > This would mean that in addition to processing the plugin.xml of the > > > > plugin, we would also process the plugin-test.xml file ( identical > > > > processing logic ) > > > > 4. have all plugin-test.xml files declare a dependency on > > > > cordova-plugin-test-framework > > > > > > > > The above suggestions could also be used in conjuction with the > cordova > > > run > > > > --tests platform mentioned by Michal, but without the need to manage > > the > > > > switching of branches. > > > > > > > > > > > > @purplecabbage > > > > risingj.com > > > > > > > > > > > > On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny <mmo...@chromium.org> > > > wrote: > > > > > > > > > Piotr: Actually I'm not sure how running tests in the harness would > > > work, > > > > > since the path to the resource may be different. However, in > > general, > > > > with > > > > > development using the harness you aren't making any changes to > > plugins. > > > > > The whole point is for app developers who want to modify only web > > > > > application bits and not deal with native compiles. > > > > > > > > > > In theory the app harness could support working on the js-modules > of > > > > > plugins, but that sounds like a really niche idea. I'd not be > > opposed > > > to > > > > > someone working on it but I'm not sure you'll have luck finding > > > > volunteers. > > > > > > > > > > -Michal > > > > > > > > > > > > > > > On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny <mmo...@chromium.org > > > > > > wrote: > > > > > > > > > > > At the time I went through my design iterations I just didn't > want > > to > > > > > > necessarily depend on cordova tooling changes / documentation. > In > > > > other > > > > > > words, someone else may have a different strategy for testing.. > > > > > > > > > > > > My personal opinion would be have the test plugin ship with a > > plugin > > > > hook > > > > > > (those are in, right? or at least on their way), that will > > > > automatically > > > > > > update the start page if you pass a flag to run command. Ie, in > an > > > > ideal > > > > > > world: `cordova run --tests` runs a plugin hook passing in > --tests > > > > flag > > > > > > which changes the start page, in a way that isn't overwritten by > > the > > > > > > top-level config.xml. > > > > > > > > > > > > My 2 cents, since I don't want "our way" of testing mobile spec > to > > be > > > > > "the > > > > > > only way" to test. Frameworks and opinions on testing change. > > > > > > > > > > > > -Michal > > > > > > > > > > > > > > > > > > On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa < > pzal...@mozilla.com > > > > > > > > wrote: > > > > > > > > > > > >> One thing more - it would be great if user could create a test > > using > > > > > test > > > > > >> harness app as well. Is it also considered? > > > > > >> > > > > > >> Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez pisze: > > > > > >> > > > > > >> It would be a nice to have in the cli, aimed to just setup the > > > right > > > > > path > > > > > >>> in the config.xml, maybe along with an another argument to > build, > > > > > >>> run/emulate as well. > > > > > >>> It sounds great. > > > > > >>> > > > > > >>> > > > > > >>> 2014-06-17 15:21 GMT-05:00 Piotr Zalewa <pzal...@mozilla.com>: > > > > > >>> > > > > > >>> Thanks Martin, > > > > > >>>> > > > > > >>>> Has it been considered to create a separate command "testrun" > or > > > > > similar > > > > > >>>> which would remove the need to edit the config.xml? > > > > > >>>> > > > > > >>>> Dnia Tue Jun 17 11:58:33 2014 Michal Mocny pisze: > > > > > >>>> > > > > > >>>> Martin, thanks for posting those links. > > > > > >>>> > > > > > >>>>> > > > > > >>>>> And I'll look into the INFRA tickets I need to file to set > up a > > > > repo > > > > > >>>>> for > > > > > >>>>> that plugin, since its ready to come out of labs. > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> On Tue, Jun 17, 2014 at 2:06 PM, Martin Gonzalez < > > > > > >>>>> martin.c.glez.g...@gmail.com> wrote: > > > > > >>>>> > > > > > >>>>> This is the Cordova Plugin Test Framework readme.md, you > can > > > > catch > > > > > >>>>> up > > > > > >>>>> > > > > > >>>>>> with > > > > > >>>>>> the functionality by reading some of the content: > > > > > >>>>>> > > > > > >>>>>> Repository: > > > > > >>>>>> https://github.com/apache/cordova-labs > > > > > >>>>>> > > > > > >>>>>> Docs: > > > > > >>>>>> > https://github.com/apache/cordova-labs/blob/master/README.md > > > > > >>>>>> > > > > > >>>>>> https://github.com/apache/cordova-labs/blob/cdvtest/ > > > > > >>>>>> cordova-plugin-test-framework/README.md > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>>> 2014-06-17 12:56 GMT-05:00 Piotr Zalewa < > pzal...@mozilla.com > > >: > > > > > >>>>>> > > > > > >>>>>> a documentation explaining how it's gonna work > > > > > >>>>>> > > > > > >>>>>>> > > > > > >>>>>>> Dnia Tue Jun 17 10:51:58 2014 Michal Mocny pisze: > > > > > >>>>>>> > > > > > >>>>>>> What do you mean? > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>> On Tue, Jun 17, 2014 at 1:50 PM, Piotr Zalewa < > > > > > pzal...@mozilla.com> > > > > > >>>>>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>> Is there any predev document? > > > > > >>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>>> Dnia Mon Jun 16 18:30:46 2014 Andrew Grieve pisze: > > > > > >>>>>>>>> > > > > > >>>>>>>>> Yeah, really exciting. Thanks for taking this on. > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>>> On Mon, Jun 16, 2014 at 3:42 PM, Michal Mocny < > > > > > >>>>>>>>>> mmo...@chromium.org> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Fantastic! > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> I'll try to keep an eye out on the PR's, and please > ping > > me > > > > if > > > > > >>>>>>>>>>> you > > > > > >>>>>>>>>>> would > > > > > >>>>>>>>>>> like any help. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> -Michal > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On Mon, Jun 16, 2014 at 3:25 PM, Marcel Kinard < > > > > > >>>>>>>>>>> cmarc...@gmail.com> > > > > > >>>>>>>>>>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Hi, after some discussions here with IBM > management, > > > > we’re > > > > > >>>>>>>>>>> going > > > > > >>>>>>>>>>> to > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> bring > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> in a couple extra interns for a week to jumpstart the > > > > > migration > > > > > >>>>>>>>>>>> of > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> the > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>> tests out of mobile-spec into the new plugin test > framework. > > > > Staci > > > > > >>>>>>> > > > > > >>>>>>>> Cooper > > > > > >>>>>>>>>>>> will be leading this effort, and Martin Gonzalez will > > be a > > > > > part > > > > > >>>>>>>>>>>> of > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> it. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>> > > > > > >>>>>>> So if you see a bunch of pull requests, this is what it is > > > for. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> We’ll > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>> get > > > > > >>>>>>> > > > > > >>>>>>>> the interns to submit an ICLA asap. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> -- > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Piotr Zalewa > > > > > >>>>>>>>> Mozilla > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> -- > > > > > >>>>>>>> > > > > > >>>>>>> Piotr Zalewa > > > > > >>>>>>> Mozilla > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>>> -- > > > > > >>>>>> Regards, > > > > > >>>>>> Martin Gonzalez > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>> -- > > > > > >>>> Piotr Zalewa > > > > > >>>> Mozilla > > > > > >>>> > > > > > >>>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >> -- > > > > > >> Piotr Zalewa > > > > > >> Mozilla > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > >
