I think we just lead by example. Sent from my iPhone
> On Jun 19, 2014, at 6:18 PM, Michal Mocny <mmo...@chromium.org> wrote: > > +1 I agree, this would be awesome. > > New question, should this merely be the "standard" we adhere to for core > plugins, or should we actively make it difficult for plugin devs to not > ship tests directly with plugins? (Not sure how we could accomplish that, > so I hope its just a convention that applies to our work). > > -Michal > > >> On Thu, Jun 19, 2014 at 7:48 PM, Jesse <purplecabb...@gmail.com> wrote: >> >> My ultimate would be this: >> >> cordova create TestFilePlugin >> cd TestFilePlugin >> cordova platform add android >> cordova plugin add >> >> http://git-wip-us.apache.org/repos/asf/cordova-labs.git#cdvtest:cordova-plugin-test-framework >> cordova plugin add ../cordova-plugin-file/ >> cordova plugin add ../cordova-plugin-file/test/ >> cordova run android --start cdvtests/index.html >> >> Then do this for each plugin, and for each platform >> Then do this for all combinations of plugins >> ... >> >> Note the run --start does not yet exist, but this would be awesome! >> >> >> @purplecabbage >> risingj.com >> >> >>> On Thu, Jun 19, 2014 at 4:15 PM, Jesse <purplecabb...@gmail.com> wrote: >>> >>> Option a) was what I suggested a ways back, and I still stand by it. >>> I think it provides the greatest transparency, and simplicity, yet it is >>> still very flexible. >>> I don't think it would be hard to accomplish either. This is the small >>> re-org I was hinting at, you've already done the hard part. >>> >>> >>> @purplecabbage >>> risingj.com >>> >>> >>>> On Thu, Jun 19, 2014 at 3:45 PM, Michal Mocny <mmo...@chromium.org> >>> wrote: >>> >>>> Andrew has raised that concern as well. My gut says that the bundling >> of >>>> a >>>> few shorts scripts that get parsed but not run as long as they don't get >>>> require() will not affect applications negatively (there are probably >> many >>>> more significant overheads we live with today in cordova) -- but I >>>> understand why that may not be useful default. >>>> >>>> In that case, some ideas: (I recall these were proposed previously but >> not >>>> sure by whom) >>>> (a) Bundle tests as a plugin-within-the-plugin as such: >>>> myplugin/ >>>> - plugin.xml >>>> - src/... >>>> - www/... >>>> - tests/ >>>> - plugin.xml >>>> - www/... >>>> >>>> Which basically means the plugin tests live in the same repo/branch, and >>>> are fetched as part of "plugin add", but are not moved into platforms/ >> on >>>> cordova prepare by default, thus don't end up in your application (disk >>>> and >>>> network are cheap, application startup and size costs are not, right?). >>>> Then, to run tests, we basically need to iterate all plugins looking >> for a >>>> nested tests/plugin.xml, and install those. This can be added to >>>> CLI/Plugman, or just be a CLI hook even. >>>> >>>> (b) add a <js-test-module> or <js-module type="test"> that is only used >> if >>>> you run prepare with --test. Similar to the above, but I think requires >>>> more CLI/config file changes, which I'm not a fan of. >>>> (c) Just ship tests as a second plugin in a second repo, and document >> how >>>> to install tests. Can then perhaps have a <dependency type=tests>. I >>>> don't like this as much since its basically back to mobile-spec. >>>> >>>> WDYT? >>>> >>>> -Michal >>>> >>>> >>>>> On Thu, Jun 19, 2014 at 4:53 PM, Jesse <purplecabb...@gmail.com> wrote: >>>>> >>>>> re: >>>>> >>>>> Q: What do I do if my plugin tests must have very large assets? >>>>> - A: Don't bundle those assets with your plugin. If you can, have >>>> your >>>>> ... >>>>> >>>>> >>>>> My concern is I do not want to see tests added to every project that >>>> uses a >>>>> plugin, even if the assets are not large, there are implications to >>>>> including the test framework + all the tests because they get loaded >> and >>>>> processed with all of the plugins and will impact load time even if >>>> never >>>>> run. >>>>> >>>>> 99.9% of the time the plugin tests will be used by us the plugin >>>>> developers, and not the people who use the plugin in there apps. >>>>> >>>>> I agree, having the tester install the test harness plugin dependency >>>>> themselves is probably a better option, as I see you have wrapped all >>>> tests >>>>> inside a exports.defineAutoTests so we don't have to worry about >>>>> describe/it/expects not being defined. >>>>> >>>>> >>>>> @purplecabbage >>>>> risingj.com >>>>> >>>>> >>>>>> On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmo...@chromium.org> >>>>> wrote: >>>>> >>>>>> Jesee, the branch is NOT a requirement (I don't think I documented >> it >>>> as >>>>>> such, except in the examples for installing plugins for initial >> look). >>>>>> Actually, we should delete those stale branches now that we are >>>> moving >>>>>> up-to-date tests into master. It was just for early experimentation >>>> on >>>>> the >>>>>> feature. >>>>>> >>>>>> Jesse, I'm not seeing the benefit of using plugins-tests.xml or the >>>>>> dependency on the test plugin yet, may you elaborate your thoughts? >>>>>> >>>>>> My hope was that tests are just always installed alongside plugins. >>>> If >>>>>> that is not a good idea for some particular plugin, say because it >>>> uses >>>>>> huge assets, I elaborated my answer in the plugin FAQ ( >> https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq >>>>>> ): >>>>>> FAQ >>>>>> >>>>>> - >>>>>> >>>>>> Q: Should I add org.apache.cordova.test-harness as a <dependancy> >>>> of >>>>> my >>>>>> plugin? >>>>>> - A: No. The end-user should decide if they want to install the >>>> test >>>>>> harness, not your plugin (most users won't). >>>>>> - >>>>>> >>>>>> Q: What do I do if my plugin tests must have very large assets? >>>>>> - A: Don't bundle those assets with your plugin. If you can, have >>>> your >>>>>> tests fail gracefully if those assets don't don't exist >> (perhaps >>>>> log >>>>>> a >>>>>> warning, perhaps fail a single asset-checking test, and skip >> the >>>>>> rest). >>>>>> Then, ideally download those assets automatically into local >>>>> storage >>>>>> the >>>>>> first time tests run. Or create a manual test step to download >>>>>> and install >>>>>> assets. As a final alternative, split those test assets into a >>>>>> separate >>>>>> plugin, and instruct users to install that plugin to run your >>>> full >>>>>> test >>>>>> suite. >>>>>> - >>>>>> >>>>>> Q: Should I ship my app with the test harness plugin installed? >>>>>> - A: Not likely. If you want, you can. Then your app could even >>>> embed >>>>> a >>>>>> link to the test page (cdvtests/index.html) from a help >> section >>>> of >>>>>> your app, to give end users a way to run your test suite out >> in >>>>>> the feild. >>>>>> That may help diagnose causes of issues within your app. >> Maybe. >>>>>> >>>>>> >>>>>> ============= >>>>>> >>>>>> >>>>>> Feel free the debate those answers -- now is certainly the time -- >>>> but I >>>>>> put a lot of effort to make it super flexible and to not require >>>>> depending >>>>>> on changes to CLI :P >>>>>> >>>>>> -Michal >>>>>> >>>>>> >>>>>> On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabb...@gmail.com> >>>> wrote: >>>>>> >>>>>>> Sorry I missed providing feedback on this earlier ... >>>>>>> Having a deeper look at this, I am not feeling great about the >> extra >>>>>>> requirement that every plugin have an additional branch. >>>>>>> >>>>>>> Several concerns arise : >>>>>>> - test branch can be out of sync with master >>>>>>> - how do we test a specific version? >>>>>>> - tests are not immediately visible when looking at master >>>>>>> - differing versions of plugin.xml depending on the branch >>>>>>> >>>>>>> The majority of the work has been done (thanks Michal!), and >> mostly >>>> any >>>>>>> suggestions I make will just require moving code and changing a >> few >>>>>>> conventions. >>>>>>> >>>>>>> What if we? : >>>>>>> 1. add a plugin-test.xml file which has the exact format of >>>> plugin.xml >>>>>>> 2. keep tests/ and plugin-test.xml file in master branch >>>>>>> 3. have plugman/cli support an additional flag --test so we can >>>> install >>>>>>> like this: >>>>>>> cordova plugin add >>>>>>> http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git >>>>> --test >>>>>>> This would mean that in addition to processing the plugin.xml of >> the >>>>>>> plugin, we would also process the plugin-test.xml file ( identical >>>>>>> processing logic ) >>>>>>> 4. have all plugin-test.xml files declare a dependency on >>>>>>> cordova-plugin-test-framework >>>>>>> >>>>>>> The above suggestions could also be used in conjuction with the >>>> cordova >>>>>> run >>>>>>> --tests platform mentioned by Michal, but without the need to >> manage >>>>> the >>>>>>> switching of branches. >>>>>>> >>>>>>> >>>>>>> @purplecabbage >>>>>>> risingj.com >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny < >> mmo...@chromium.org> >>>>>> wrote: >>>>>>> >>>>>>>> Piotr: Actually I'm not sure how running tests in the harness >>>> would >>>>>> work, >>>>>>>> since the path to the resource may be different. However, in >>>>> general, >>>>>>> with >>>>>>>> development using the harness you aren't making any changes to >>>>> plugins. >>>>>>>> The whole point is for app developers who want to modify only >> web >>>>>>>> application bits and not deal with native compiles. >>>>>>>> >>>>>>>> In theory the app harness could support working on the >> js-modules >>>> of >>>>>>>> plugins, but that sounds like a really niche idea. I'd not be >>>>> opposed >>>>>> to >>>>>>>> someone working on it but I'm not sure you'll have luck finding >>>>>>> volunteers. >>>>>>>> >>>>>>>> -Michal >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny < >>>> mmo...@chromium.org> >>>>>>> wrote: >>>>>>>> >>>>>>>>> At the time I went through my design iterations I just didn't >>>> want >>>>> to >>>>>>>>> necessarily depend on cordova tooling changes / documentation. >>>> In >>>>>>> other >>>>>>>>> words, someone else may have a different strategy for >> testing.. >>>>>>>>> >>>>>>>>> My personal opinion would be have the test plugin ship with a >>>>> plugin >>>>>>> hook >>>>>>>>> (those are in, right? or at least on their way), that will >>>>>>> automatically >>>>>>>>> update the start page if you pass a flag to run command. Ie, >>>> in an >>>>>>> ideal >>>>>>>>> world: `cordova run --tests` runs a plugin hook passing in >>>> --tests >>>>>>> flag >>>>>>>>> which changes the start page, in a way that isn't overwritten >> by >>>>> the >>>>>>>>> top-level config.xml. >>>>>>>>> >>>>>>>>> My 2 cents, since I don't want "our way" of testing mobile >> spec >>>> to >>>>> be >>>>>>>> "the >>>>>>>>> only way" to test. Frameworks and opinions on testing >> change. >>>>>>>>> >>>>>>>>> -Michal >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa < >>>> pzal...@mozilla.com >>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> One thing more - it would be great if user could create a >> test >>>>> using >>>>>>>> test >>>>>>>>>> harness app as well. Is it also considered? >>>>>>>>>> >>>>>>>>>> Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez pisze: >>>>>>>>>> >>>>>>>>>> It would be a nice to have in the cli, aimed to just setup >> the >>>>>> right >>>>>>>> path >>>>>>>>>>> in the config.xml, maybe along with an another argument to >>>> build, >>>>>>>>>>> run/emulate as well. >>>>>>>>>>> It sounds great. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2014-06-17 15:21 GMT-05:00 Piotr Zalewa < >> pzal...@mozilla.com >>>>> : >>>>>>>>>>> >>>>>>>>>>> Thanks Martin, >>>>>>>>>>>> >>>>>>>>>>>> Has it been considered to create a separate command >>>> "testrun" or >>>>>>>> similar >>>>>>>>>>>> which would remove the need to edit the config.xml? >>>>>>>>>>>> >>>>>>>>>>>> Dnia Tue Jun 17 11:58:33 2014 Michal Mocny pisze: >>>>>>>>>>>> >>>>>>>>>>>> Martin, thanks for posting those links. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> And I'll look into the INFRA tickets I need to file to set >>>> up a >>>>>>> repo >>>>>>>>>>>>> for >>>>>>>>>>>>> that plugin, since its ready to come out of labs. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Jun 17, 2014 at 2:06 PM, Martin Gonzalez < >>>>>>>>>>>>> martin.c.glez.g...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> This is the Cordova Plugin Test Framework readme.md, >> you >>>> can >>>>>>> catch >>>>>>>>>>>>> up >>>>>>>>>>>>> >>>>>>>>>>>>>> with >>>>>>>>>>>>>> the functionality by reading some of the content: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Repository: >>>>>>>>>>>>>> https://github.com/apache/cordova-labs >>>>>>>>>>>>>> >>>>>>>>>>>>>> Docs: >>>> https://github.com/apache/cordova-labs/blob/master/README.md >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://github.com/apache/cordova-labs/blob/cdvtest/ >>>>>>>>>>>>>> cordova-plugin-test-framework/README.md >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2014-06-17 12:56 GMT-05:00 Piotr Zalewa < >>>> pzal...@mozilla.com >>>>>> : >>>>>>>>>>>>>> >>>>>>>>>>>>>> a documentation explaining how it's gonna work >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Dnia Tue Jun 17 10:51:58 2014 Michal Mocny pisze: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What do you mean? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Jun 17, 2014 at 1:50 PM, Piotr Zalewa < >>>>>>>> pzal...@mozilla.com> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is there any predev document? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Dnia Mon Jun 16 18:30:46 2014 Andrew Grieve pisze: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yeah, really exciting. Thanks for taking this on. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Mon, Jun 16, 2014 at 3:42 PM, Michal Mocny < >>>>>>>>>>>>>>>>>> mmo...@chromium.org> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Fantastic! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'll try to keep an eye out on the PR's, and please >>>> ping >>>>> me >>>>>>> if >>>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>> like any help. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -Michal >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 16, 2014 at 3:25 PM, Marcel Kinard < >>>>>>>>>>>>>>>>>>> cmarc...@gmail.com> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi, after some discussions here with IBM >>>> management, >>>>>>> we’re >>>>>>>>>>>>>>>>>>> going >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> bring >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> in a couple extra interns for a week to jumpstart >> the >>>>>>>> migration >>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>> tests out of mobile-spec into the new plugin test >>>> framework. >>>>>>> Staci >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cooper >>>>>>>>>>>>>>>>>>>> will be leading this effort, and Martin Gonzalez >> will >>>>> be a >>>>>>>> part >>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> it. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> So if you see a bunch of pull requests, this is what it >>>> is >>>>>> for. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> We’ll >>>>>>>>>>>>>> get >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> the interns to submit an ICLA asap. >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Piotr Zalewa >>>>>>>>>>>>>>>>> Mozilla >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Piotr Zalewa >>>>>>>>>>>>>>> Mozilla >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> Martin Gonzalez >>>>>>>>>>>>> -- >>>>>>>>>>>> Piotr Zalewa >>>>>>>>>>>> Mozilla >>>>>>>>>> -- >>>>>>>>>> Piotr Zalewa >>>>>>>>>> Mozilla >>
