re: Q: What do I do if my plugin tests must have very large assets? - A: Don't bundle those assets with your plugin. If you can, have your ...
My concern is I do not want to see tests added to every project that uses a plugin, even if the assets are not large, there are implications to including the test framework + all the tests because they get loaded and processed with all of the plugins and will impact load time even if never run. 99.9% of the time the plugin tests will be used by us the plugin developers, and not the people who use the plugin in there apps. I agree, having the tester install the test harness plugin dependency themselves is probably a better option, as I see you have wrapped all tests inside a exports.defineAutoTests so we don't have to worry about describe/it/expects not being defined. @purplecabbage risingj.com On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmo...@chromium.org> wrote: > Jesee, the branch is NOT a requirement (I don't think I documented it as > such, except in the examples for installing plugins for initial look). > Actually, we should delete those stale branches now that we are moving > up-to-date tests into master. It was just for early experimentation on the > feature. > > Jesse, I'm not seeing the benefit of using plugins-tests.xml or the > dependency on the test plugin yet, may you elaborate your thoughts? > > My hope was that tests are just always installed alongside plugins. If > that is not a good idea for some particular plugin, say because it uses > huge assets, I elaborated my answer in the plugin FAQ ( > > https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq > ): > FAQ > > - > > Q: Should I add org.apache.cordova.test-harness as a <dependancy> of my > plugin? > - A: No. The end-user should decide if they want to install the test > harness, not your plugin (most users won't). > - > > Q: What do I do if my plugin tests must have very large assets? > - A: Don't bundle those assets with your plugin. If you can, have your > tests fail gracefully if those assets don't don't exist (perhaps log > a > warning, perhaps fail a single asset-checking test, and skip the > rest). > Then, ideally download those assets automatically into local storage > the > first time tests run. Or create a manual test step to download > and install > assets. As a final alternative, split those test assets into a > separate > plugin, and instruct users to install that plugin to run your full > test > suite. > - > > Q: Should I ship my app with the test harness plugin installed? > - A: Not likely. If you want, you can. Then your app could even embed a > link to the test page (cdvtests/index.html) from a help section of > your app, to give end users a way to run your test suite out in > the feild. > That may help diagnose causes of issues within your app. Maybe. > > > ============= > > > Feel free the debate those answers -- now is certainly the time -- but I > put a lot of effort to make it super flexible and to not require depending > on changes to CLI :P > > -Michal > > > On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabb...@gmail.com> wrote: > > > Sorry I missed providing feedback on this earlier ... > > Having a deeper look at this, I am not feeling great about the extra > > requirement that every plugin have an additional branch. > > > > Several concerns arise : > > - test branch can be out of sync with master > > - how do we test a specific version? > > - tests are not immediately visible when looking at master > > - differing versions of plugin.xml depending on the branch > > > > The majority of the work has been done (thanks Michal!), and mostly any > > suggestions I make will just require moving code and changing a few > > conventions. > > > > What if we? : > > 1. add a plugin-test.xml file which has the exact format of plugin.xml > > 2. keep tests/ and plugin-test.xml file in master branch > > 3. have plugman/cli support an additional flag --test so we can install > > like this: > > cordova plugin add > > http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git --test > > This would mean that in addition to processing the plugin.xml of the > > plugin, we would also process the plugin-test.xml file ( identical > > processing logic ) > > 4. have all plugin-test.xml files declare a dependency on > > cordova-plugin-test-framework > > > > The above suggestions could also be used in conjuction with the cordova > run > > --tests platform mentioned by Michal, but without the need to manage the > > switching of branches. > > > > > > @purplecabbage > > risingj.com > > > > > > On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny <mmo...@chromium.org> > wrote: > > > > > Piotr: Actually I'm not sure how running tests in the harness would > work, > > > since the path to the resource may be different. However, in general, > > with > > > development using the harness you aren't making any changes to plugins. > > > The whole point is for app developers who want to modify only web > > > application bits and not deal with native compiles. > > > > > > In theory the app harness could support working on the js-modules of > > > plugins, but that sounds like a really niche idea. I'd not be opposed > to > > > someone working on it but I'm not sure you'll have luck finding > > volunteers. > > > > > > -Michal > > > > > > > > > On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny <mmo...@chromium.org> > > wrote: > > > > > > > At the time I went through my design iterations I just didn't want to > > > > necessarily depend on cordova tooling changes / documentation. In > > other > > > > words, someone else may have a different strategy for testing.. > > > > > > > > My personal opinion would be have the test plugin ship with a plugin > > hook > > > > (those are in, right? or at least on their way), that will > > automatically > > > > update the start page if you pass a flag to run command. Ie, in an > > ideal > > > > world: `cordova run --tests` runs a plugin hook passing in --tests > > flag > > > > which changes the start page, in a way that isn't overwritten by the > > > > top-level config.xml. > > > > > > > > My 2 cents, since I don't want "our way" of testing mobile spec to be > > > "the > > > > only way" to test. Frameworks and opinions on testing change. > > > > > > > > -Michal > > > > > > > > > > > > On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa <pzal...@mozilla.com> > > > wrote: > > > > > > > >> One thing more - it would be great if user could create a test using > > > test > > > >> harness app as well. Is it also considered? > > > >> > > > >> Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez pisze: > > > >> > > > >> It would be a nice to have in the cli, aimed to just setup the > right > > > path > > > >>> in the config.xml, maybe along with an another argument to build, > > > >>> run/emulate as well. > > > >>> It sounds great. > > > >>> > > > >>> > > > >>> 2014-06-17 15:21 GMT-05:00 Piotr Zalewa <pzal...@mozilla.com>: > > > >>> > > > >>> Thanks Martin, > > > >>>> > > > >>>> Has it been considered to create a separate command "testrun" or > > > similar > > > >>>> which would remove the need to edit the config.xml? > > > >>>> > > > >>>> Dnia Tue Jun 17 11:58:33 2014 Michal Mocny pisze: > > > >>>> > > > >>>> Martin, thanks for posting those links. > > > >>>> > > > >>>>> > > > >>>>> And I'll look into the INFRA tickets I need to file to set up a > > repo > > > >>>>> for > > > >>>>> that plugin, since its ready to come out of labs. > > > >>>>> > > > >>>>> > > > >>>>> On Tue, Jun 17, 2014 at 2:06 PM, Martin Gonzalez < > > > >>>>> martin.c.glez.g...@gmail.com> wrote: > > > >>>>> > > > >>>>> This is the Cordova Plugin Test Framework readme.md, you can > > catch > > > >>>>> up > > > >>>>> > > > >>>>>> with > > > >>>>>> the functionality by reading some of the content: > > > >>>>>> > > > >>>>>> Repository: > > > >>>>>> https://github.com/apache/cordova-labs > > > >>>>>> > > > >>>>>> Docs: > > > >>>>>> https://github.com/apache/cordova-labs/blob/master/README.md > > > >>>>>> > > > >>>>>> https://github.com/apache/cordova-labs/blob/cdvtest/ > > > >>>>>> cordova-plugin-test-framework/README.md > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> 2014-06-17 12:56 GMT-05:00 Piotr Zalewa <pzal...@mozilla.com>: > > > >>>>>> > > > >>>>>> a documentation explaining how it's gonna work > > > >>>>>> > > > >>>>>>> > > > >>>>>>> Dnia Tue Jun 17 10:51:58 2014 Michal Mocny pisze: > > > >>>>>>> > > > >>>>>>> What do you mean? > > > >>>>>>> > > > >>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Tue, Jun 17, 2014 at 1:50 PM, Piotr Zalewa < > > > pzal...@mozilla.com> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>> Is there any predev document? > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>>> Dnia Mon Jun 16 18:30:46 2014 Andrew Grieve pisze: > > > >>>>>>>>> > > > >>>>>>>>> Yeah, really exciting. Thanks for taking this on. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>>> On Mon, Jun 16, 2014 at 3:42 PM, Michal Mocny < > > > >>>>>>>>>> mmo...@chromium.org> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>> > > > >>>>>>>>>> Fantastic! > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> I'll try to keep an eye out on the PR's, and please ping me > > if > > > >>>>>>>>>>> you > > > >>>>>>>>>>> would > > > >>>>>>>>>>> like any help. > > > >>>>>>>>>>> > > > >>>>>>>>>>> -Michal > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> On Mon, Jun 16, 2014 at 3:25 PM, Marcel Kinard < > > > >>>>>>>>>>> cmarc...@gmail.com> > > > >>>>>>>>>>> wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>> Hi, after some discussions here with IBM management, > > we’re > > > >>>>>>>>>>> going > > > >>>>>>>>>>> to > > > >>>>>>>>>>> > > > >>>>>>>>>>> bring > > > >>>>>>>>>>> > > > >>>>>>>>>>>> in a couple extra interns for a week to jumpstart the > > > migration > > > >>>>>>>>>>>> of > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> the > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>> tests out of mobile-spec into the new plugin test framework. > > Staci > > > >>>>>>> > > > >>>>>>>> Cooper > > > >>>>>>>>>>>> will be leading this effort, and Martin Gonzalez will be a > > > part > > > >>>>>>>>>>>> of > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> it. > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>> > > > >>>>>>> So if you see a bunch of pull requests, this is what it is > for. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> We’ll > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>> get > > > >>>>>>> > > > >>>>>>>> the interns to submit an ICLA asap. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>> -- > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> Piotr Zalewa > > > >>>>>>>>> Mozilla > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>> > > > >>>>>>> Piotr Zalewa > > > >>>>>>> Mozilla > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>>> -- > > > >>>>>> Regards, > > > >>>>>> Martin Gonzalez > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> -- > > > >>>> Piotr Zalewa > > > >>>> Mozilla > > > >>>> > > > >>>> > > > >>> > > > >>> > > > >>> > > > >> -- > > > >> Piotr Zalewa > > > >> Mozilla > > > >> > > > > > > > > > > > > > >
