Re: mobile-spec and releases: How do we test?
Hi, Right now, the work live in the cordova-labs repo under the CDVTest branch ( https://github.com/apache/cordova-labs/tree/cdvtest). Specifically, there is a test plugin and test harness app. Its still a work in progress, and while several of the API tests from mobile-spec were moved to CDVTest branches of the core plugin, the mobile-spec app is currently still the canonical source for tests. Sorry I dropped the ball on filing JIRA issues, but David and I have been working on the test harness for our downstream cordova distribution (for cca). I'll be pushing all the applicable bits back up within the coming weeks and then will document clearly what needs to happen to migrate mobile-spec to see if we all agree on direction. I guess: stay tuned. -Michal On Sun, Feb 23, 2014 at 11:05 PM, 罗琦 l...@polyvi.com wrote: Hi all, How's the progress? Where's JIRA I could follow? We're building a Cordova-based framework and working closely to Cordova daily bits. We're still keeping tests in each plugin repo, and manually sync with Cordova-Mobile-Spec, that's even more painful after most of tests got removed from plugin repos. We added a little script to cli (a new command actually) to integrate tests of all currently installed plugins into to target app, that's way how we perform testing for now. Need some advice, thanks. -- Original -- From: Michal Mocnymmo...@chromium.org; Date: Thu, Oct 31, 2013 11:35 PM To: devdev@cordova.apache.org; Subject: Re: mobile-spec and releases: How do we test? This is awesome progress, guys, thanks for the help. I'm going to put all the bits together and compile a list of tasks left and write-up instructions for those who have yet to take a look. If everyone on the lists is still happy with the direction, I'll move those to JIRA. On Thu, Oct 31, 2013 at 11:08 AM, David Kemp drk...@chromium.org wrote: I converted the couchdb reporter to the 2.0 style and added it to the repo. It works(hard coded config), but still needs the configuration components completed and some final adjustments to the data. On Thu, Oct 31, 2013 at 11:05 AM, Anis KADRI anis.ka...@gmail.com wrote: I ported the contacts plugin [1] to the new style and I found the process to be more or less straightforward. I also kept the eval in there but there might be a better way ? [1] http://goo.gl/uhnwtz On Wed, Oct 30, 2013 at 3:42 PM, Michal Mocny mmo...@chromium.org wrote: Sadly, we are approaching an in-between time of moving the mobile-spec tests out of the app and into plugins. We are still investigating the best way to do this without disruption. For what its worth: several plugins now have a 'cdvtest' branch which supplies new-style tests ripped out of mobile-spec. If you are having issues cleaning up the old style tests, take a look at the new ones (or try porting yourself). I'm going to write up a doc with the summary of the state of testing within a day or so given the results of this week to make it easier for you (and others) to pick up. -Michal On Wed, Oct 30, 2013 at 1:54 PM, Naik, Archana na...@lab126.com wrote: Thanks Michal. You answered my questions. More to elaborate on my question: I am testing amazon-fireos port(platform) with all plug-ins using mobile-spec. I am seeing some failures in 3.1.0 version because of test cases timing out. I am pretty new to cordova and still in learning phase. :) I am trying to understand these failures. Interestingly they pass on 3.0.x version. Archana From: Michal Mocny mmo...@chromium.org Date: Wednesday, October 30, 2013 10:27 AM To: Naik, Archana na...@lab126.com Cc: dev@cordova.apache.org dev@cordova.apache.org, Michal Mocny mmo...@chromium.org Subject: Re: mobile-spec and releases: How do we test? May you clarify? Right now, there is no formal way to test plugins, we are trying to invent that way now. Check out cordova-labs repo's cdvtest branch for a sample app plugin to track progress. Jasmine is hosted in that sample app, but plugins will not directly know/care. Any testing framework which is api-compatible should work. In practice, I'm not sure how compatible they all are, so it may very well be limited to jasmine -- but it does mean you can make local modifications such as our CI is doing to create a custom test reporter. -Michal On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote: Hi, Guys While on this topic, I have a question: how do I test individual plug-in? Where is the this jasmine version specified? Thanks Archana On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote: Here are some
Re: mobile-spec and releases: How do we test?
Hi all, How's the progress? Where's JIRA I could follow? We're building a Cordova-based framework and working closely to Cordova daily bits. We're still keeping tests in each plugin repo, and manually sync with Cordova-Mobile-Spec, that's even more painful after most of tests got removed from plugin repos. We added a little script to cli (a new command actually) to integrate tests of all currently installed plugins into to target app, that's way how we perform testing for now. Need some advice, thanks. -- Original -- From: Michal Mocnymmo...@chromium.org; Date: Thu, Oct 31, 2013 11:35 PM To: devdev@cordova.apache.org; Subject: Re: mobile-spec and releases: How do we test? This is awesome progress, guys, thanks for the help. I'm going to put all the bits together and compile a list of tasks left and write-up instructions for those who have yet to take a look. If everyone on the lists is still happy with the direction, I'll move those to JIRA. On Thu, Oct 31, 2013 at 11:08 AM, David Kemp drk...@chromium.org wrote: I converted the couchdb reporter to the 2.0 style and added it to the repo. It works(hard coded config), but still needs the configuration components completed and some final adjustments to the data. On Thu, Oct 31, 2013 at 11:05 AM, Anis KADRI anis.ka...@gmail.com wrote: I ported the contacts plugin [1] to the new style and I found the process to be more or less straightforward. I also kept the eval in there but there might be a better way ? [1] http://goo.gl/uhnwtz On Wed, Oct 30, 2013 at 3:42 PM, Michal Mocny mmo...@chromium.org wrote: Sadly, we are approaching an in-between time of moving the mobile-spec tests out of the app and into plugins. We are still investigating the best way to do this without disruption. For what its worth: several plugins now have a 'cdvtest' branch which supplies new-style tests ripped out of mobile-spec. If you are having issues cleaning up the old style tests, take a look at the new ones (or try porting yourself). I'm going to write up a doc with the summary of the state of testing within a day or so given the results of this week to make it easier for you (and others) to pick up. -Michal On Wed, Oct 30, 2013 at 1:54 PM, Naik, Archana na...@lab126.com wrote: Thanks Michal. You answered my questions. More to elaborate on my question: I am testing amazon-fireos port(platform) with all plug-ins using mobile-spec. I am seeing some failures in 3.1.0 version because of test cases timing out. I am pretty new to cordova and still in learning phase. :) I am trying to understand these failures. Interestingly they pass on 3.0.x version. Archana From: Michal Mocny mmo...@chromium.org Date: Wednesday, October 30, 2013 10:27 AM To: Naik, Archana na...@lab126.com Cc: dev@cordova.apache.org dev@cordova.apache.org, Michal Mocny mmo...@chromium.org Subject: Re: mobile-spec and releases: How do we test? May you clarify? Right now, there is no formal way to test plugins, we are trying to invent that way now. Check out cordova-labs repo's cdvtest branch for a sample app plugin to track progress. Jasmine is hosted in that sample app, but plugins will not directly know/care. Any testing framework which is api-compatible should work. In practice, I'm not sure how compatible they all are, so it may very well be limited to jasmine -- but it does mean you can make local modifications such as our CI is doing to create a custom test reporter. -Michal On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote: Hi, Guys While on this topic, I have a question: how do I test individual plug-in? Where is the this jasmine version specified? Thanks Archana On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote: Here are some links to jasmine-2 docs since its a hard time finding them: http://jasmine.github.io/2.0/introduction.html https://github.com/pivotal/jasmine/blob/master/release_notes/20rc5.md On Wed, Oct 30, 2013 at 10:16 AM, Michal Mocny mmo...@chromium.org wrote: On Tue, Oct 29, 2013 at 5:29 PM, Bryan Higgins br...@bryanhiggins.netwrote: I just converted geolocation to the new test style [1] I'm happy with the process overall, and I find the jasmine 2 tests are more succinct. There are a few things worth noting: - I kept the eval code in. At google today, it was discussed that this may not be the best approach. - Jasmine 2: You must hit at least one expect statement or the test will timeout even though done was called. We could file a bug (I ran into it during setup once too), but really, what is the worth
Re: mobile-spec and releases: How do we test?
I ported the contacts plugin [1] to the new style and I found the process to be more or less straightforward. I also kept the eval in there but there might be a better way ? [1] http://goo.gl/uhnwtz On Wed, Oct 30, 2013 at 3:42 PM, Michal Mocny mmo...@chromium.org wrote: Sadly, we are approaching an in-between time of moving the mobile-spec tests out of the app and into plugins. We are still investigating the best way to do this without disruption. For what its worth: several plugins now have a 'cdvtest' branch which supplies new-style tests ripped out of mobile-spec. If you are having issues cleaning up the old style tests, take a look at the new ones (or try porting yourself). I'm going to write up a doc with the summary of the state of testing within a day or so given the results of this week to make it easier for you (and others) to pick up. -Michal On Wed, Oct 30, 2013 at 1:54 PM, Naik, Archana na...@lab126.com wrote: Thanks Michal. You answered my questions. More to elaborate on my question: I am testing amazon-fireos port(platform) with all plug-ins using mobile-spec. I am seeing some failures in 3.1.0 version because of test cases timing out. I am pretty new to cordova and still in learning phase. :) I am trying to understand these failures. Interestingly they pass on 3.0.x version. Archana From: Michal Mocny mmo...@chromium.org Date: Wednesday, October 30, 2013 10:27 AM To: Naik, Archana na...@lab126.com Cc: dev@cordova.apache.org dev@cordova.apache.org, Michal Mocny mmo...@chromium.org Subject: Re: mobile-spec and releases: How do we test? May you clarify? Right now, there is no formal way to test plugins, we are trying to invent that way now. Check out cordova-labs repo's cdvtest branch for a sample app plugin to track progress. Jasmine is hosted in that sample app, but plugins will not directly know/care. Any testing framework which is api-compatible should work. In practice, I'm not sure how compatible they all are, so it may very well be limited to jasmine -- but it does mean you can make local modifications such as our CI is doing to create a custom test reporter. -Michal On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote: Hi, Guys While on this topic, I have a question: how do I test individual plug-in? Where is the this jasmine version specified? Thanks Archana On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote: Here are some links to jasmine-2 docs since its a hard time finding them: http://jasmine.github.io/2.0/introduction.html https://github.com/pivotal/jasmine/blob/master/release_notes/20rc5.md On Wed, Oct 30, 2013 at 10:16 AM, Michal Mocny mmo...@chromium.org wrote: On Tue, Oct 29, 2013 at 5:29 PM, Bryan Higgins br...@bryanhiggins.netwrote: I just converted geolocation to the new test style [1] I'm happy with the process overall, and I find the jasmine 2 tests are more succinct. There are a few things worth noting: - I kept the eval code in. At google today, it was discussed that this may not be the best approach. - Jasmine 2: You must hit at least one expect statement or the test will timeout even though done was called. We could file a bug (I ran into it during setup once too), but really, what is the worth of a test if there are no expect clauses? - I did not remove the manual test page [2]. It would be great if we had a convention for importing these. (ie test harness looks for a page at www/test/plugin-id.html and adds a link to it if it exists) I'm going to work on a solution for this today. The thought is that the plugin has a single module that can define all auto/manual tests, and the test-harness choses which to display where. [1] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603 [2] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=blob;f=test/manual.html;h=6ec2eed03e18c2efaa8710094d60930bb32227ba;hb =459a01c01e8dfa2a688d25483bb48c46d8e2 On Tue, Oct 15, 2013 at 12:50 PM, David Kemp drk...@chromium.org wrote: In spite of that fact that it needs a tooling change, I like the added mode tag / prepare steps. The tooling change should be small and it means no runtime impact on apps. I love the approach - a very positive step to cleaning up testing. David Kemp On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny mmo...@chromium.org wrote: Braden, you're right, good catch. Discussing locally how we could support prepare --mode=... in the most generalized form, we remembered an old suggestion to just support mode tags. The benefits seem to be: - No need to add custom tag-prefix/attributes for the combinations of js-module mode=, asset mode=, etc etc - More visually
Re: mobile-spec and releases: How do we test?
I converted the couchdb reporter to the 2.0 style and added it to the repo. It works(hard coded config), but still needs the configuration components completed and some final adjustments to the data. On Thu, Oct 31, 2013 at 11:05 AM, Anis KADRI anis.ka...@gmail.com wrote: I ported the contacts plugin [1] to the new style and I found the process to be more or less straightforward. I also kept the eval in there but there might be a better way ? [1] http://goo.gl/uhnwtz On Wed, Oct 30, 2013 at 3:42 PM, Michal Mocny mmo...@chromium.org wrote: Sadly, we are approaching an in-between time of moving the mobile-spec tests out of the app and into plugins. We are still investigating the best way to do this without disruption. For what its worth: several plugins now have a 'cdvtest' branch which supplies new-style tests ripped out of mobile-spec. If you are having issues cleaning up the old style tests, take a look at the new ones (or try porting yourself). I'm going to write up a doc with the summary of the state of testing within a day or so given the results of this week to make it easier for you (and others) to pick up. -Michal On Wed, Oct 30, 2013 at 1:54 PM, Naik, Archana na...@lab126.com wrote: Thanks Michal. You answered my questions. More to elaborate on my question: I am testing amazon-fireos port(platform) with all plug-ins using mobile-spec. I am seeing some failures in 3.1.0 version because of test cases timing out. I am pretty new to cordova and still in learning phase. :) I am trying to understand these failures. Interestingly they pass on 3.0.x version. Archana From: Michal Mocny mmo...@chromium.org Date: Wednesday, October 30, 2013 10:27 AM To: Naik, Archana na...@lab126.com Cc: dev@cordova.apache.org dev@cordova.apache.org, Michal Mocny mmo...@chromium.org Subject: Re: mobile-spec and releases: How do we test? May you clarify? Right now, there is no formal way to test plugins, we are trying to invent that way now. Check out cordova-labs repo's cdvtest branch for a sample app plugin to track progress. Jasmine is hosted in that sample app, but plugins will not directly know/care. Any testing framework which is api-compatible should work. In practice, I'm not sure how compatible they all are, so it may very well be limited to jasmine -- but it does mean you can make local modifications such as our CI is doing to create a custom test reporter. -Michal On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote: Hi, Guys While on this topic, I have a question: how do I test individual plug-in? Where is the this jasmine version specified? Thanks Archana On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote: Here are some links to jasmine-2 docs since its a hard time finding them: http://jasmine.github.io/2.0/introduction.html https://github.com/pivotal/jasmine/blob/master/release_notes/20rc5.md On Wed, Oct 30, 2013 at 10:16 AM, Michal Mocny mmo...@chromium.org wrote: On Tue, Oct 29, 2013 at 5:29 PM, Bryan Higgins br...@bryanhiggins.netwrote: I just converted geolocation to the new test style [1] I'm happy with the process overall, and I find the jasmine 2 tests are more succinct. There are a few things worth noting: - I kept the eval code in. At google today, it was discussed that this may not be the best approach. - Jasmine 2: You must hit at least one expect statement or the test will timeout even though done was called. We could file a bug (I ran into it during setup once too), but really, what is the worth of a test if there are no expect clauses? - I did not remove the manual test page [2]. It would be great if we had a convention for importing these. (ie test harness looks for a page at www/test/plugin-id.html and adds a link to it if it exists) I'm going to work on a solution for this today. The thought is that the plugin has a single module that can define all auto/manual tests, and the test-harness choses which to display where. [1] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603 [2] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=blob;f=test/manual.html;h=6ec2eed03e18c2efaa8710094d60930bb32227ba;hb =459a01c01e8dfa2a688d25483bb48c46d8e2 On Tue, Oct 15, 2013 at 12:50 PM, David Kemp drk...@chromium.org wrote: In spite of that fact that it needs a tooling change, I like the added mode tag / prepare steps. The tooling change should be small and it means no runtime impact on apps. I love the approach - a very positive step to cleaning up testing. David Kemp On Tue, Oct
Re: mobile-spec and releases: How do we test?
This is awesome progress, guys, thanks for the help. I'm going to put all the bits together and compile a list of tasks left and write-up instructions for those who have yet to take a look. If everyone on the lists is still happy with the direction, I'll move those to JIRA. On Thu, Oct 31, 2013 at 11:08 AM, David Kemp drk...@chromium.org wrote: I converted the couchdb reporter to the 2.0 style and added it to the repo. It works(hard coded config), but still needs the configuration components completed and some final adjustments to the data. On Thu, Oct 31, 2013 at 11:05 AM, Anis KADRI anis.ka...@gmail.com wrote: I ported the contacts plugin [1] to the new style and I found the process to be more or less straightforward. I also kept the eval in there but there might be a better way ? [1] http://goo.gl/uhnwtz On Wed, Oct 30, 2013 at 3:42 PM, Michal Mocny mmo...@chromium.org wrote: Sadly, we are approaching an in-between time of moving the mobile-spec tests out of the app and into plugins. We are still investigating the best way to do this without disruption. For what its worth: several plugins now have a 'cdvtest' branch which supplies new-style tests ripped out of mobile-spec. If you are having issues cleaning up the old style tests, take a look at the new ones (or try porting yourself). I'm going to write up a doc with the summary of the state of testing within a day or so given the results of this week to make it easier for you (and others) to pick up. -Michal On Wed, Oct 30, 2013 at 1:54 PM, Naik, Archana na...@lab126.com wrote: Thanks Michal. You answered my questions. More to elaborate on my question: I am testing amazon-fireos port(platform) with all plug-ins using mobile-spec. I am seeing some failures in 3.1.0 version because of test cases timing out. I am pretty new to cordova and still in learning phase. :) I am trying to understand these failures. Interestingly they pass on 3.0.x version. Archana From: Michal Mocny mmo...@chromium.org Date: Wednesday, October 30, 2013 10:27 AM To: Naik, Archana na...@lab126.com Cc: dev@cordova.apache.org dev@cordova.apache.org, Michal Mocny mmo...@chromium.org Subject: Re: mobile-spec and releases: How do we test? May you clarify? Right now, there is no formal way to test plugins, we are trying to invent that way now. Check out cordova-labs repo's cdvtest branch for a sample app plugin to track progress. Jasmine is hosted in that sample app, but plugins will not directly know/care. Any testing framework which is api-compatible should work. In practice, I'm not sure how compatible they all are, so it may very well be limited to jasmine -- but it does mean you can make local modifications such as our CI is doing to create a custom test reporter. -Michal On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote: Hi, Guys While on this topic, I have a question: how do I test individual plug-in? Where is the this jasmine version specified? Thanks Archana On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote: Here are some links to jasmine-2 docs since its a hard time finding them: http://jasmine.github.io/2.0/introduction.html https://github.com/pivotal/jasmine/blob/master/release_notes/20rc5.md On Wed, Oct 30, 2013 at 10:16 AM, Michal Mocny mmo...@chromium.org wrote: On Tue, Oct 29, 2013 at 5:29 PM, Bryan Higgins br...@bryanhiggins.netwrote: I just converted geolocation to the new test style [1] I'm happy with the process overall, and I find the jasmine 2 tests are more succinct. There are a few things worth noting: - I kept the eval code in. At google today, it was discussed that this may not be the best approach. - Jasmine 2: You must hit at least one expect statement or the test will timeout even though done was called. We could file a bug (I ran into it during setup once too), but really, what is the worth of a test if there are no expect clauses? - I did not remove the manual test page [2]. It would be great if we had a convention for importing these. (ie test harness looks for a page at www/test/plugin-id.html and adds a link to it if it exists) I'm going to work on a solution for this today. The thought is that the plugin has a single module that can define all auto/manual tests, and the test-harness choses which to display where. [1] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603 [2] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git
Re: mobile-spec and releases: How do we test?
May you clarify?
Right now, there is no formal way to test plugins, we are trying to invent
that way now. Check out cordova-labs repo's cdvtest branch for a sample
app plugin to track progress.
Jasmine is hosted in that sample app, but plugins will not directly
know/care. Any testing framework which is api-compatible should work. In
practice, I'm not sure how compatible they all are, so it may very well be
limited to jasmine -- but it does mean you can make local modifications
such as our CI is doing to create a custom test reporter.
-Michal
On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote:
Hi, Guys
While on this topic, I have a question: how do I test individual plug-in?
Where is the this jasmine version specified?
Thanks
Archana
On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote:
Here are some links to jasmine-2 docs since its a hard time finding them:
http://jasmine.github.io/2.0/introduction.html
https://github.com/pivotal/jasmine/blob/master/release_notes/20rc5.md
On Wed, Oct 30, 2013 at 10:16 AM, Michal Mocny mmo...@chromium.org
wrote:
On Tue, Oct 29, 2013 at 5:29 PM, Bryan Higgins
br...@bryanhiggins.netwrote:
I just converted geolocation to the new test style [1]
I'm happy with the process overall, and I find the jasmine 2 tests are
more
succinct.
There are a few things worth noting:
- I kept the eval code in. At google today, it was discussed that this
may
not be the best approach.
- Jasmine 2: You must hit at least one expect statement or the test
will
timeout even though done was called.
We could file a bug (I ran into it during setup once too), but really,
what is the worth of a test if there are no expect clauses?
- I did not remove the manual test page [2]. It would be great if we
had a
convention for importing these. (ie test harness looks for a page at
www/test/plugin-id.html and adds a link to it if it exists)
I'm going to work on a solution for this today. The thought is that the
plugin has a single module that can define all auto/manual tests, and
the
test-harness choses which to display where.
[1]
https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git
;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603
[2]
https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git
;a=blob;f=test/manual.html;h=6ec2eed03e18c2efaa8710094d60930bb32227ba;hb
=459a01c01e8dfa2a688d25483bb48c46d8e2
On Tue, Oct 15, 2013 at 12:50 PM, David Kemp drk...@chromium.org
wrote:
In spite of that fact that it needs a tooling change, I like the
added
mode tag / prepare steps.
The tooling change should be small and it means no runtime impact on
apps.
I love the approach - a very positive step to cleaning up testing.
David Kemp
On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny mmo...@chromium.org
wrote:
Braden, you're right, good catch.
Discussing locally how we could support prepare --mode=... in the
most
generalized form, we remembered an old suggestion to just support
mode
tags.
The benefits seem to be:
- No need to add custom tag-prefix/attributes for the combinations
of
js-module mode=, asset mode=, etc etc
- More visually apparent when reading a plugin.xml file, akin to
platforms tag
The drawbacks seem to be:
- Not all descendant tags are easy to support for a given mode (ie,
dependency)
Summarizing the options currently discussed in this thread:
- new js-test-module tag. Not general enough solution to support
tests
bundling assets, so -1 from me for this reason.
- mode=... attribute for a set of whitelisted tags (js-module,
asset,
...)
- mode name=... tag for a set of whitelisted descendant
tags (js-module, asset, ...)
The last two options are really functionally equivalent, but given
that
we
opted for platform tag instead of attribute, I'm also favoring a
mode
tag.
-Michal
On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson
bra...@chromium.org
wrote:
It's not true that adding these tests only creates larger
binaries.
They
will be fetched and parsed by the plugin loader code at app
startup
time.
It goes through the list of all plugins in cordova_plugins.js and
loads
them all. That parses them, and runs the outermost layer, which
is
the
wrapping function function(require, module, exports) { ... }. So
there
is
still runtime memory and CPU impact.
I support js-test-module tags or js-module ... test or
whatever.
Similarly, prepare for tests. I realize we want to avoid tooling
support,
but I think tooling support is a lesser evil than production
performance
impact.
Overall approach makes me happy.
Braden
On
Re: mobile-spec and releases: How do we test?
Sadly, we are approaching an in-between time of moving the mobile-spec tests out of the app and into plugins. We are still investigating the best way to do this without disruption. For what its worth: several plugins now have a 'cdvtest' branch which supplies new-style tests ripped out of mobile-spec. If you are having issues cleaning up the old style tests, take a look at the new ones (or try porting yourself). I'm going to write up a doc with the summary of the state of testing within a day or so given the results of this week to make it easier for you (and others) to pick up. -Michal On Wed, Oct 30, 2013 at 1:54 PM, Naik, Archana na...@lab126.com wrote: Thanks Michal. You answered my questions. More to elaborate on my question: I am testing amazon-fireos port(platform) with all plug-ins using mobile-spec. I am seeing some failures in 3.1.0 version because of test cases timing out. I am pretty new to cordova and still in learning phase. :) I am trying to understand these failures. Interestingly they pass on 3.0.x version. Archana From: Michal Mocny mmo...@chromium.org Date: Wednesday, October 30, 2013 10:27 AM To: Naik, Archana na...@lab126.com Cc: dev@cordova.apache.org dev@cordova.apache.org, Michal Mocny mmo...@chromium.org Subject: Re: mobile-spec and releases: How do we test? May you clarify? Right now, there is no formal way to test plugins, we are trying to invent that way now. Check out cordova-labs repo's cdvtest branch for a sample app plugin to track progress. Jasmine is hosted in that sample app, but plugins will not directly know/care. Any testing framework which is api-compatible should work. In practice, I'm not sure how compatible they all are, so it may very well be limited to jasmine -- but it does mean you can make local modifications such as our CI is doing to create a custom test reporter. -Michal On Wed, Oct 30, 2013 at 12:57 PM, Naik, Archana na...@lab126.com wrote: Hi, Guys While on this topic, I have a question: how do I test individual plug-in? Where is the this jasmine version specified? Thanks Archana On 10/30/13 7:26 AM, Michal Mocny mmo...@chromium.org wrote: Here are some links to jasmine-2 docs since its a hard time finding them: http://jasmine.github.io/2.0/introduction.html https://github.com/pivotal/jasmine/blob/master/release_notes/20rc5.md On Wed, Oct 30, 2013 at 10:16 AM, Michal Mocny mmo...@chromium.org wrote: On Tue, Oct 29, 2013 at 5:29 PM, Bryan Higgins br...@bryanhiggins.netwrote: I just converted geolocation to the new test style [1] I'm happy with the process overall, and I find the jasmine 2 tests are more succinct. There are a few things worth noting: - I kept the eval code in. At google today, it was discussed that this may not be the best approach. - Jasmine 2: You must hit at least one expect statement or the test will timeout even though done was called. We could file a bug (I ran into it during setup once too), but really, what is the worth of a test if there are no expect clauses? - I did not remove the manual test page [2]. It would be great if we had a convention for importing these. (ie test harness looks for a page at www/test/plugin-id.html and adds a link to it if it exists) I'm going to work on a solution for this today. The thought is that the plugin has a single module that can define all auto/manual tests, and the test-harness choses which to display where. [1] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603 [2] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git ;a=blob;f=test/manual.html;h=6ec2eed03e18c2efaa8710094d60930bb32227ba;hb =459a01c01e8dfa2a688d25483bb48c46d8e2 On Tue, Oct 15, 2013 at 12:50 PM, David Kemp drk...@chromium.org wrote: In spite of that fact that it needs a tooling change, I like the added mode tag / prepare steps. The tooling change should be small and it means no runtime impact on apps. I love the approach - a very positive step to cleaning up testing. David Kemp On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny mmo...@chromium.org wrote: Braden, you're right, good catch. Discussing locally how we could support prepare --mode=... in the most generalized form, we remembered an old suggestion to just support mode tags. The benefits seem to be: - No need to add custom tag-prefix/attributes for the combinations of js-module mode=, asset mode=, etc etc - More visually apparent when reading a plugin.xml file, akin to platforms tag The drawbacks seem to be: - Not all descendant tags are easy to support for a given mode (ie, dependency) Summarizing the options currently discussed in this thread: - new js-test
Re: mobile-spec and releases: How do we test?
I just converted geolocation to the new test style [1]
I'm happy with the process overall, and I find the jasmine 2 tests are more
succinct.
There are a few things worth noting:
- I kept the eval code in. At google today, it was discussed that this may
not be the best approach.
- Jasmine 2: You must hit at least one expect statement or the test will
timeout even though done was called.
- I did not remove the manual test page [2]. It would be great if we had a
convention for importing these. (ie test harness looks for a page at
www/test/plugin-id.html and adds a link to it if it exists)
[1]
https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603
[2]
https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git;a=blob;f=test/manual.html;h=6ec2eed03e18c2efaa8710094d60930bb32227ba;hb=459a01c01e8dfa2a688d25483bb48c46d8e2
On Tue, Oct 15, 2013 at 12:50 PM, David Kemp drk...@chromium.org wrote:
In spite of that fact that it needs a tooling change, I like the added
mode tag / prepare steps.
The tooling change should be small and it means no runtime impact on apps.
I love the approach - a very positive step to cleaning up testing.
David Kemp
On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny mmo...@chromium.org
wrote:
Braden, you're right, good catch.
Discussing locally how we could support prepare --mode=... in the most
generalized form, we remembered an old suggestion to just support mode
tags.
The benefits seem to be:
- No need to add custom tag-prefix/attributes for the combinations of
js-module mode=, asset mode=, etc etc
- More visually apparent when reading a plugin.xml file, akin to
platforms tag
The drawbacks seem to be:
- Not all descendant tags are easy to support for a given mode (ie,
dependency)
Summarizing the options currently discussed in this thread:
- new js-test-module tag. Not general enough solution to support tests
bundling assets, so -1 from me for this reason.
- mode=... attribute for a set of whitelisted tags (js-module,
asset,
...)
- mode name=... tag for a set of whitelisted descendant
tags (js-module, asset, ...)
The last two options are really functionally equivalent, but given that
we
opted for platform tag instead of attribute, I'm also favoring a mode
tag.
-Michal
On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson
bra...@chromium.org
wrote:
It's not true that adding these tests only creates larger binaries.
They
will be fetched and parsed by the plugin loader code at app startup
time.
It goes through the list of all plugins in cordova_plugins.js and loads
them all. That parses them, and runs the outermost layer, which is the
wrapping function function(require, module, exports) { ... }. So there
is
still runtime memory and CPU impact.
I support js-test-module tags or js-module ... test or whatever.
Similarly, prepare for tests. I realize we want to avoid tooling
support,
but I think tooling support is a lesser evil than production
performance
impact.
Overall approach makes me happy.
Braden
On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny mmo...@chromium.org
wrote:
On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve agri...@chromium.org
wrote:
The eval of the jasmine interface deserves mention. Is the
motivation
there that tests can choose to use another testing framework? That's
why
you don't just make jasmine functions globals?
I was hoping the plugins would need to depend on anything but CDVTest,
and
not expect any globals. I guess, though, that CDVTest still expects
the
app to provide to a test framework and some other stuff, so in the end
its
no different. I was hedging on being able to update CDVTest in the
future
for whatever we need, and all 3rdparty plugins would not need
updating.
eval() could be used to do all sorts of clever things if we needed..
One nit pick just from reading your email is that this will cause
the
test
js-modules to be injected into apps that use the plugins. I think
probably
we will want to update the tools to recognize a js-test-module. We
*could* work around it by adding the tests to new plugins that
depend
on
the thing they are testing, but I think changing the tools would be
nicer.
I also mentioned splitting tests into second plugin but thats overkill
except in extreme circumstances. Note that the tests aren't actually
loaded unless you require them, so its just a matter of larger
binaries
which could be filtered out manually.
My person preference would be to support conditional build types,
which
have come up before. ie, cordova prepare debug, cordova prepare
release,
cordova prepare test -- and plugin.xml could specify a different set
of
js-module for either.
A specific
Re: mobile-spec and releases: How do we test?
It's not true that adding these tests only creates larger binaries. They
will be fetched and parsed by the plugin loader code at app startup time.
It goes through the list of all plugins in cordova_plugins.js and loads
them all. That parses them, and runs the outermost layer, which is the
wrapping function function(require, module, exports) { ... }. So there is
still runtime memory and CPU impact.
I support js-test-module tags or js-module ... test or whatever.
Similarly, prepare for tests. I realize we want to avoid tooling support,
but I think tooling support is a lesser evil than production performance
impact.
Overall approach makes me happy.
Braden
On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny mmo...@chromium.org wrote:
On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve agri...@chromium.org
wrote:
The eval of the jasmine interface deserves mention. Is the motivation
there that tests can choose to use another testing framework? That's why
you don't just make jasmine functions globals?
I was hoping the plugins would need to depend on anything but CDVTest, and
not expect any globals. I guess, though, that CDVTest still expects the
app to provide to a test framework and some other stuff, so in the end its
no different. I was hedging on being able to update CDVTest in the future
for whatever we need, and all 3rdparty plugins would not need updating.
eval() could be used to do all sorts of clever things if we needed..
One nit pick just from reading your email is that this will cause the
test
js-modules to be injected into apps that use the plugins. I think
probably
we will want to update the tools to recognize a js-test-module. We
*could* work around it by adding the tests to new plugins that depend on
the thing they are testing, but I think changing the tools would be
nicer.
I also mentioned splitting tests into second plugin but thats overkill
except in extreme circumstances. Note that the tests aren't actually
loaded unless you require them, so its just a matter of larger binaries
which could be filtered out manually.
My person preference would be to support conditional build types, which
have come up before. ie, cordova prepare debug, cordova prepare release,
cordova prepare test -- and plugin.xml could specify a different set of
js-module for either.
A specific js-test-module would be fine, but isnt 0 new tooling.
Also, I forgot to mention, but we do need to add support for getting the
full list of plugins installed, which should be trivial to add to
modulemapper (maybe there is already a way to reach in, but I don't think
we have a documented api for it).
Another nit is that it would be nice if the CordovaTests app didn't
depend
on any plugins. e.g., don't have it depend on device plugin.
CordovaTests doesn't explicitly depend on device plugin, except that at the
moment I have it printing the same info that MobileSpec does at startup.
This could be wrapped in a try{}catch in case the require fails, but its
good info to have in the common case.
Overall, really like the approach!
Thanks!
On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny mmo...@chromium.org
wrote:
TLDR; I've implemented a plugin testing strategy that requires 0 new
tooling features, can support non-core plugins, and hopefully even
support
a variety of methods for running/reporting the test results. Super
alpha
preview, but take a look if you like the direction!
NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
UPDATED: Converted three existing plugins to this new style. Code is
on
feature branches of respective repos (cdvtest branch):
org.apache.cordova.device
org.apache.cordova.device-motion
org.chromium.storage
(must clone locally, switch to branch, and plugin add from local path)
Now, any plugin that wants to join in on the fun needs to provide a
js-module
src=... name=test / and use this template:
exports.init = function() {
eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
'this'));
// TESTS GO HERE
describe(..., function() {
it(...);
});
};
(The eval is optional but super useful; it will inject the jasmine
interface into your scope, so you don't have to type jasmine.describe,
jasmine.it, etc. Not sure of any cleaner way to do this.)
STEPS:
1. create a new cordova project
2. import the CordovaTest app into your www
3. add any or all of the above plugins
4. give it a run, and try out the auto tests (all pass on ipad, some
still
fail on android)
Lots of work left to do, but hopefully good enough to whet your
appetite.
One thing to note, I've attempted to make CDVTest and plugin tests
unaware
of the specific jasmine version the app is hosting (so that it can be
swapped without changing all plugins), but it must support the new style
Re: mobile-spec and releases: How do we test?
Braden, you're right, good catch.
Discussing locally how we could support prepare --mode=... in the most
generalized form, we remembered an old suggestion to just support mode
tags.
The benefits seem to be:
- No need to add custom tag-prefix/attributes for the combinations of
js-module mode=, asset mode=, etc etc
- More visually apparent when reading a plugin.xml file, akin to
platforms tag
The drawbacks seem to be:
- Not all descendant tags are easy to support for a given mode (ie,
dependency)
Summarizing the options currently discussed in this thread:
- new js-test-module tag. Not general enough solution to support tests
bundling assets, so -1 from me for this reason.
- mode=... attribute for a set of whitelisted tags (js-module, asset,
...)
- mode name=... tag for a set of whitelisted descendant
tags (js-module, asset, ...)
The last two options are really functionally equivalent, but given that we
opted for platform tag instead of attribute, I'm also favoring a mode
tag.
-Michal
On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson bra...@chromium.orgwrote:
It's not true that adding these tests only creates larger binaries. They
will be fetched and parsed by the plugin loader code at app startup time.
It goes through the list of all plugins in cordova_plugins.js and loads
them all. That parses them, and runs the outermost layer, which is the
wrapping function function(require, module, exports) { ... }. So there is
still runtime memory and CPU impact.
I support js-test-module tags or js-module ... test or whatever.
Similarly, prepare for tests. I realize we want to avoid tooling support,
but I think tooling support is a lesser evil than production performance
impact.
Overall approach makes me happy.
Braden
On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny mmo...@chromium.org wrote:
On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve agri...@chromium.org
wrote:
The eval of the jasmine interface deserves mention. Is the motivation
there that tests can choose to use another testing framework? That's why
you don't just make jasmine functions globals?
I was hoping the plugins would need to depend on anything but CDVTest, and
not expect any globals. I guess, though, that CDVTest still expects the
app to provide to a test framework and some other stuff, so in the end its
no different. I was hedging on being able to update CDVTest in the future
for whatever we need, and all 3rdparty plugins would not need updating.
eval() could be used to do all sorts of clever things if we needed..
One nit pick just from reading your email is that this will cause the
test
js-modules to be injected into apps that use the plugins. I think
probably
we will want to update the tools to recognize a js-test-module. We
*could* work around it by adding the tests to new plugins that depend on
the thing they are testing, but I think changing the tools would be
nicer.
I also mentioned splitting tests into second plugin but thats overkill
except in extreme circumstances. Note that the tests aren't actually
loaded unless you require them, so its just a matter of larger binaries
which could be filtered out manually.
My person preference would be to support conditional build types, which
have come up before. ie, cordova prepare debug, cordova prepare release,
cordova prepare test -- and plugin.xml could specify a different set of
js-module for either.
A specific js-test-module would be fine, but isnt 0 new tooling.
Also, I forgot to mention, but we do need to add support for getting the
full list of plugins installed, which should be trivial to add to
modulemapper (maybe there is already a way to reach in, but I don't think
we have a documented api for it).
Another nit is that it would be nice if the CordovaTests app didn't
depend
on any plugins. e.g., don't have it depend on device plugin.
CordovaTests doesn't explicitly depend on device plugin, except that at
the
moment I have it printing the same info that MobileSpec does at startup.
This could be wrapped in a try{}catch in case the require fails, but its
good info to have in the common case.
Overall, really like the approach!
Thanks!
On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny mmo...@chromium.org
wrote:
TLDR; I've implemented a plugin testing strategy that requires 0 new
tooling features, can support non-core plugins, and hopefully even
support
a variety of methods for running/reporting the test results. Super
alpha
preview, but take a look if you like the direction!
NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
UPDATED: Converted three existing plugins to this new style. Code
is on
feature branches of respective repos (cdvtest branch):
org.apache.cordova.device
org.apache.cordova.device-motion
org.chromium.storage
(must clone locally, switch to branch, and plugin
Re: mobile-spec and releases: How do we test?
In spite of that fact that it needs a tooling change, I like the added
mode tag / prepare steps.
The tooling change should be small and it means no runtime impact on apps.
I love the approach - a very positive step to cleaning up testing.
David Kemp
On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny mmo...@chromium.org wrote:
Braden, you're right, good catch.
Discussing locally how we could support prepare --mode=... in the most
generalized form, we remembered an old suggestion to just support mode
tags.
The benefits seem to be:
- No need to add custom tag-prefix/attributes for the combinations of
js-module mode=, asset mode=, etc etc
- More visually apparent when reading a plugin.xml file, akin to
platforms tag
The drawbacks seem to be:
- Not all descendant tags are easy to support for a given mode (ie,
dependency)
Summarizing the options currently discussed in this thread:
- new js-test-module tag. Not general enough solution to support tests
bundling assets, so -1 from me for this reason.
- mode=... attribute for a set of whitelisted tags (js-module, asset,
...)
- mode name=... tag for a set of whitelisted descendant
tags (js-module, asset, ...)
The last two options are really functionally equivalent, but given that we
opted for platform tag instead of attribute, I'm also favoring a mode
tag.
-Michal
On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson bra...@chromium.org
wrote:
It's not true that adding these tests only creates larger binaries. They
will be fetched and parsed by the plugin loader code at app startup time.
It goes through the list of all plugins in cordova_plugins.js and loads
them all. That parses them, and runs the outermost layer, which is the
wrapping function function(require, module, exports) { ... }. So there is
still runtime memory and CPU impact.
I support js-test-module tags or js-module ... test or whatever.
Similarly, prepare for tests. I realize we want to avoid tooling support,
but I think tooling support is a lesser evil than production performance
impact.
Overall approach makes me happy.
Braden
On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny mmo...@chromium.org
wrote:
On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve agri...@chromium.org
wrote:
The eval of the jasmine interface deserves mention. Is the motivation
there that tests can choose to use another testing framework? That's
why
you don't just make jasmine functions globals?
I was hoping the plugins would need to depend on anything but CDVTest,
and
not expect any globals. I guess, though, that CDVTest still expects the
app to provide to a test framework and some other stuff, so in the end
its
no different. I was hedging on being able to update CDVTest in the
future
for whatever we need, and all 3rdparty plugins would not need updating.
eval() could be used to do all sorts of clever things if we needed..
One nit pick just from reading your email is that this will cause the
test
js-modules to be injected into apps that use the plugins. I think
probably
we will want to update the tools to recognize a js-test-module. We
*could* work around it by adding the tests to new plugins that depend
on
the thing they are testing, but I think changing the tools would be
nicer.
I also mentioned splitting tests into second plugin but thats overkill
except in extreme circumstances. Note that the tests aren't actually
loaded unless you require them, so its just a matter of larger binaries
which could be filtered out manually.
My person preference would be to support conditional build types, which
have come up before. ie, cordova prepare debug, cordova prepare
release,
cordova prepare test -- and plugin.xml could specify a different set of
js-module for either.
A specific js-test-module would be fine, but isnt 0 new tooling.
Also, I forgot to mention, but we do need to add support for getting the
full list of plugins installed, which should be trivial to add to
modulemapper (maybe there is already a way to reach in, but I don't
think
we have a documented api for it).
Another nit is that it would be nice if the CordovaTests app didn't
depend
on any plugins. e.g., don't have it depend on device plugin.
CordovaTests doesn't explicitly depend on device plugin, except that at
the
moment I have it printing the same info that MobileSpec does at startup.
This could be wrapped in a try{}catch in case the require fails, but
its
good info to have in the common case.
Overall, really like the approach!
Thanks!
On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny mmo...@chromium.org
wrote:
TLDR; I've implemented a plugin testing strategy that requires 0 new
tooling features, can support non-core plugins, and hopefully even
support
a variety of methods for running/reporting the test results. Super
Re: mobile-spec and releases: How do we test?
Sorry keep meaning to respond. I like Michal's first step but growing to a
full suite of tools. Are you currently tackling this Braden? I feel like it
is related to the Medic stuff and maybe we should throw one of our guys on
the problem fully.
On Sep 27, 2013 5:10 PM, Braden Shepherdson bra...@chromium.org wrote:
Which one?
On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux b...@brian.io wrote:
I really like your proposal as a starting point. Very simple but would
allow for in-app testing as well as on the cmd line if we so wish.
On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny mmo...@chromium.org
wrote:
I was looking over some old emails from this list on plugin testing,
and
an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la release/debug
and
have target-specific plugin.xml tags (assets, js-modules,
source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add
anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have (i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but
we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed
plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow toggling
of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of
library
code for creating tests, and plugins can use it to register their
tests.
- This makes it easier to register manual tests in a common format
for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
bra...@chromium.org
wrote:
Here's an off-the-top-of-my-head sketch of how we might do
Voltron
tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo Automated
/
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to the
current
mobile-spec's
- That index reads a file akin to cordova_plugins.js
(cordova_tests.js,
maybe?) generated by the CLI, containing the info from the test
tags.
- It has navigation similar to the current mobile-spec, with
buttons
for the automatic and manual sections. Auto has All and then
each
module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana
csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron and
be
DRY
and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as
you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the future
to
include
information about testing (i.e. Directory containing tests
files,
test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come
with
the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp
drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running
(derived
from
Medic) uses only the mobilespec tests. It does not yet use
tests
collected
from the plugins. Its been talked about, but not gone
anywhere.
David Kemp
On Wed, Sep 25, 2013 at 7:58 PM, Jesse
purplecabb...@gmail.com
javascript:;
wrote:
Yeah, I have
Re: mobile-spec and releases: How do we test?
I'm throwing something together right now, actually. I'll post my current
progress today so you can take a look.
On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux b...@brian.io wrote:
Sorry keep meaning to respond. I like Michal's first step but growing to a
full suite of tools. Are you currently tackling this Braden? I feel like it
is related to the Medic stuff and maybe we should throw one of our guys on
the problem fully.
On Sep 27, 2013 5:10 PM, Braden Shepherdson bra...@chromium.org wrote:
Which one?
On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux b...@brian.io wrote:
I really like your proposal as a starting point. Very simple but would
allow for in-app testing as well as on the cmd line if we so wish.
On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny mmo...@chromium.org
wrote:
I was looking over some old emails from this list on plugin testing,
and
an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la release/debug
and
have target-specific plugin.xml tags (assets, js-modules,
source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while
now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add
anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have
(i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others
have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but
we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed
plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow toggling
of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of
library
code for creating tests, and plugins can use it to register their
tests.
- This makes it easier to register manual tests in a common
format
for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
bra...@chromium.org
wrote:
Here's an off-the-top-of-my-head sketch of how we might do
Voltron
tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo
Automated
/
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to
the
current
mobile-spec's
- That index reads a file akin to cordova_plugins.js
(cordova_tests.js,
maybe?) generated by the CLI, containing the info from the
test
tags.
- It has navigation similar to the current mobile-spec,
with
buttons
for the automatic and manual sections. Auto has All and then
each
module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana
csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron
and
be
DRY
and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as
you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the
future
to
include
information about testing (i.e. Directory containing tests
files,
test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come
with
the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp
drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running
(derived
from
Re: mobile-spec and releases: How do we test?
TLDR; I've implemented a plugin testing strategy that requires 0 new
tooling features, can support non-core plugins, and hopefully even support
a variety of methods for running/reporting the test results. Super alpha
preview, but take a look if you like the direction!
NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
UPDATED: Converted three existing plugins to this new style. Code is on
feature branches of respective repos (cdvtest branch):
org.apache.cordova.device
org.apache.cordova.device-motion
org.chromium.storage
(must clone locally, switch to branch, and plugin add from local path)
Now, any plugin that wants to join in on the fun needs to provide a js-module
src=... name=test / and use this template:
exports.init = function() {
eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
'this'));
// TESTS GO HERE
describe(..., function() {
it(...);
});
};
(The eval is optional but super useful; it will inject the jasmine
interface into your scope, so you don't have to type jasmine.describe,
jasmine.it, etc. Not sure of any cleaner way to do this.)
STEPS:
1. create a new cordova project
2. import the CordovaTest app into your www
3. add any or all of the above plugins
4. give it a run, and try out the auto tests (all pass on ipad, some still
fail on android)
Lots of work left to do, but hopefully good enough to whet your appetite.
One thing to note, I've attempted to make CDVTest and plugin tests unaware
of the specific jasmine version the app is hosting (so that it can be
swapped without changing all plugins), but it must support the new style
interface for async tests (ie, accept a done callback). This is the style
that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use (I'm
using jasmine 2.0 rc3). This means that core plugin tests require some
code transformations, but the net effect is cleaner tests and more common
style with our node tools' tests.
Manual tests are not implemented yet.
-Michal
On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny mmo...@chromium.org wrote:
I'm throwing something together right now, actually. I'll post my current
progress today so you can take a look.
On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux b...@brian.io wrote:
Sorry keep meaning to respond. I like Michal's first step but growing to a
full suite of tools. Are you currently tackling this Braden? I feel like
it
is related to the Medic stuff and maybe we should throw one of our guys on
the problem fully.
On Sep 27, 2013 5:10 PM, Braden Shepherdson bra...@chromium.org
wrote:
Which one?
On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux b...@brian.io wrote:
I really like your proposal as a starting point. Very simple but would
allow for in-app testing as well as on the cmd line if we so wish.
On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny mmo...@chromium.org
wrote:
I was looking over some old emails from this list on plugin testing,
and
an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la
release/debug
and
have target-specific plugin.xml tags (assets, js-modules,
source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while
now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny
mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add
anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have
(i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others
have
mentioned TAP).
- Any app is free to run those plugin tests in any which way,
but
we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed
plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow
toggling
of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of
library
code for creating tests, and plugins can use it to register
their
tests.
- This makes it easier to register manual tests in a common
format
for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing
Re: mobile-spec and releases: How do we test?
The eval of the jasmine interface deserves mention. Is the motivation there
that tests can choose to use another testing framework? That's why you
don't just make jasmine functions globals?
One nit pick just from reading your email is that this will cause the test
js-modules to be injected into apps that use the plugins. I think probably
we will want to update the tools to recognize a js-test-module. We
*could* work around it by adding the tests to new plugins that depend on
the thing they are testing, but I think changing the tools would be nicer.
Another nit is that it would be nice if the CordovaTests app didn't depend
on any plugins. e.g., don't have it depend on device plugin.
Overall, really like the approach!
On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny mmo...@chromium.org wrote:
TLDR; I've implemented a plugin testing strategy that requires 0 new
tooling features, can support non-core plugins, and hopefully even support
a variety of methods for running/reporting the test results. Super alpha
preview, but take a look if you like the direction!
NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
UPDATED: Converted three existing plugins to this new style. Code is on
feature branches of respective repos (cdvtest branch):
org.apache.cordova.device
org.apache.cordova.device-motion
org.chromium.storage
(must clone locally, switch to branch, and plugin add from local path)
Now, any plugin that wants to join in on the fun needs to provide a
js-module
src=... name=test / and use this template:
exports.init = function() {
eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
'this'));
// TESTS GO HERE
describe(..., function() {
it(...);
});
};
(The eval is optional but super useful; it will inject the jasmine
interface into your scope, so you don't have to type jasmine.describe,
jasmine.it, etc. Not sure of any cleaner way to do this.)
STEPS:
1. create a new cordova project
2. import the CordovaTest app into your www
3. add any or all of the above plugins
4. give it a run, and try out the auto tests (all pass on ipad, some still
fail on android)
Lots of work left to do, but hopefully good enough to whet your appetite.
One thing to note, I've attempted to make CDVTest and plugin tests unaware
of the specific jasmine version the app is hosting (so that it can be
swapped without changing all plugins), but it must support the new style
interface for async tests (ie, accept a done callback). This is the style
that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use (I'm
using jasmine 2.0 rc3). This means that core plugin tests require some
code transformations, but the net effect is cleaner tests and more common
style with our node tools' tests.
Manual tests are not implemented yet.
-Michal
On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny mmo...@chromium.org
wrote:
I'm throwing something together right now, actually. I'll post my
current
progress today so you can take a look.
On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux b...@brian.io wrote:
Sorry keep meaning to respond. I like Michal's first step but growing
to a
full suite of tools. Are you currently tackling this Braden? I feel like
it
is related to the Medic stuff and maybe we should throw one of our guys
on
the problem fully.
On Sep 27, 2013 5:10 PM, Braden Shepherdson bra...@chromium.org
wrote:
Which one?
On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux b...@brian.io wrote:
I really like your proposal as a starting point. Very simple but
would
allow for in-app testing as well as on the cmd line if we so wish.
On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny mmo...@chromium.org
wrote:
I was looking over some old emails from this list on plugin
testing,
and
an
idea that was proposed way back was to ship plugin tests as a
second
plugin. That way, you can chose to install tests, or not, and
know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la
release/debug
and
have target-specific plugin.xml tags (assets, js-modules,
source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while
now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny
mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add
anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document
a
convention
of where they should live, and what signature it should have
(i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common
Re: mobile-spec and releases: How do we test?
On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve agri...@chromium.org wrote:
The eval of the jasmine interface deserves mention. Is the motivation
there that tests can choose to use another testing framework? That's why
you don't just make jasmine functions globals?
I was hoping the plugins would need to depend on anything but CDVTest, and
not expect any globals. I guess, though, that CDVTest still expects the
app to provide to a test framework and some other stuff, so in the end its
no different. I was hedging on being able to update CDVTest in the future
for whatever we need, and all 3rdparty plugins would not need updating.
eval() could be used to do all sorts of clever things if we needed..
One nit pick just from reading your email is that this will cause the test
js-modules to be injected into apps that use the plugins. I think probably
we will want to update the tools to recognize a js-test-module. We
*could* work around it by adding the tests to new plugins that depend on
the thing they are testing, but I think changing the tools would be nicer.
I also mentioned splitting tests into second plugin but thats overkill
except in extreme circumstances. Note that the tests aren't actually
loaded unless you require them, so its just a matter of larger binaries
which could be filtered out manually.
My person preference would be to support conditional build types, which
have come up before. ie, cordova prepare debug, cordova prepare release,
cordova prepare test -- and plugin.xml could specify a different set of
js-module for either.
A specific js-test-module would be fine, but isnt 0 new tooling.
Also, I forgot to mention, but we do need to add support for getting the
full list of plugins installed, which should be trivial to add to
modulemapper (maybe there is already a way to reach in, but I don't think
we have a documented api for it).
Another nit is that it would be nice if the CordovaTests app didn't depend
on any plugins. e.g., don't have it depend on device plugin.
CordovaTests doesn't explicitly depend on device plugin, except that at the
moment I have it printing the same info that MobileSpec does at startup.
This could be wrapped in a try{}catch in case the require fails, but its
good info to have in the common case.
Overall, really like the approach!
Thanks!
On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny mmo...@chromium.org wrote:
TLDR; I've implemented a plugin testing strategy that requires 0 new
tooling features, can support non-core plugins, and hopefully even support
a variety of methods for running/reporting the test results. Super alpha
preview, but take a look if you like the direction!
NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
UPDATED: Converted three existing plugins to this new style. Code is on
feature branches of respective repos (cdvtest branch):
org.apache.cordova.device
org.apache.cordova.device-motion
org.chromium.storage
(must clone locally, switch to branch, and plugin add from local path)
Now, any plugin that wants to join in on the fun needs to provide a
js-module
src=... name=test / and use this template:
exports.init = function() {
eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
'this'));
// TESTS GO HERE
describe(..., function() {
it(...);
});
};
(The eval is optional but super useful; it will inject the jasmine
interface into your scope, so you don't have to type jasmine.describe,
jasmine.it, etc. Not sure of any cleaner way to do this.)
STEPS:
1. create a new cordova project
2. import the CordovaTest app into your www
3. add any or all of the above plugins
4. give it a run, and try out the auto tests (all pass on ipad, some still
fail on android)
Lots of work left to do, but hopefully good enough to whet your appetite.
One thing to note, I've attempted to make CDVTest and plugin tests unaware
of the specific jasmine version the app is hosting (so that it can be
swapped without changing all plugins), but it must support the new style
interface for async tests (ie, accept a done callback). This is the style
that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use (I'm
using jasmine 2.0 rc3). This means that core plugin tests require some
code transformations, but the net effect is cleaner tests and more common
style with our node tools' tests.
Manual tests are not implemented yet.
-Michal
On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny mmo...@chromium.org
wrote:
I'm throwing something together right now, actually. I'll post my
current
progress today so you can take a look.
On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux b...@brian.io wrote:
Sorry keep meaning to respond. I like Michal's first step but growing
to a
full suite of tools. Are you currently tackling this Braden? I feel
like
it
is related to the Medic stuff and
Re: mobile-spec and releases: How do we test?
I was looking over some old emails from this list on plugin testing, and an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la release/debug and
have target-specific plugin.xml tags (assets, js-modules, source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny mmo...@chromium.org wrote:
I would suggest perhaps a simpler approach, which doesn't add anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have (i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow toggling of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of library
code for creating tests, and plugins can use it to register their tests.
- This makes it easier to register manual tests in a common format for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
bra...@chromium.org
wrote:
Here's an off-the-top-of-my-head sketch of how we might do Voltron
tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo Automated /
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to the
current
mobile-spec's
- That index reads a file akin to cordova_plugins.js
(cordova_tests.js,
maybe?) generated by the CLI, containing the info from the test tags.
- It has navigation similar to the current mobile-spec, with
buttons
for the automatic and manual sections. Auto has All and then each
module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron and be DRY
and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the future to
include
information about testing (i.e. Directory containing tests files,
test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come with the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running
(derived
from
Medic) uses only the mobilespec tests. It does not yet use tests
collected
from the plugins. Its been talked about, but not gone anywhere.
David Kemp
On Wed, Sep 25, 2013 at 7:58 PM, Jesse purplecabb...@gmail.com
javascript:;
wrote:
Yeah, I have pushed some changes to mobile-spec, and when I did
I
also
copied the tests into the plugin involved.
Until we get the magic test runner happening, I think we just
keep
duplicating.
@purplecabbage
risingj.com
On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill
stevengil...@gmail.com
javascript:;
wrote:
We copied over tests into plugins when we first broke them
out,
but
I
don't
believe they have been updated.
I would say for now to just add the tests to mobile spec, and
possibly in
the future we go all voltron to build mobile spec and keep
tests
with
their
corresponding plugins.
On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser
bows...@gmail.com
javascript:;
wrote:
Hey
Right now, I'm working on a weird file issue that requires
me
to
update mobile-spec, but I'm wondering where the tests should
live.
Re: mobile-spec and releases: How do we test?
Hum I think keeping tests with the plugin is a better approach, keeps code
and test together on a single repo for a plugin.
Maybe plugman should not install the test folder located on the root of the
plugin by default unless an optional flag --test is pass
plugman install --test ...
cordova plugin add com.plugin --test
--Carlos
On Fri, Sep 27, 2013 at 9:28 AM, Michal Mocny mmo...@chromium.org wrote:
I was looking over some old emails from this list on plugin testing, and an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la release/debug and
have target-specific plugin.xml tags (assets, js-modules, source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have (i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow toggling of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of
library
code for creating tests, and plugins can use it to register their
tests.
- This makes it easier to register manual tests in a common format for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
bra...@chromium.org
wrote:
Here's an off-the-top-of-my-head sketch of how we might do Voltron
tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo Automated /
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to the
current
mobile-spec's
- That index reads a file akin to cordova_plugins.js
(cordova_tests.js,
maybe?) generated by the CLI, containing the info from the test
tags.
- It has navigation similar to the current mobile-spec, with
buttons
for the automatic and manual sections. Auto has All and then each
module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana
csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron and be
DRY
and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the future to
include
information about testing (i.e. Directory containing tests files,
test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come with
the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running
(derived
from
Medic) uses only the mobilespec tests. It does not yet use
tests
collected
from the plugins. Its been talked about, but not gone anywhere.
David Kemp
On Wed, Sep 25, 2013 at 7:58 PM, Jesse
purplecabb...@gmail.com
javascript:;
wrote:
Yeah, I have pushed some changes to mobile-spec, and when I
did
I
also
copied the tests into the plugin involved.
Until we get the magic test runner happening, I think we just
keep
duplicating.
@purplecabbage
risingj.com
On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill
stevengil...@gmail.com
javascript:;
wrote:
We copied over tests into plugins when we first broke them
out,
but
I
Re: mobile-spec and releases: How do we test?
I really like your proposal as a starting point. Very simple but would
allow for in-app testing as well as on the cmd line if we so wish.
On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny mmo...@chromium.org wrote:
I was looking over some old emails from this list on plugin testing, and an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la release/debug and
have target-specific plugin.xml tags (assets, js-modules, source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have (i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow toggling of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of
library
code for creating tests, and plugins can use it to register their
tests.
- This makes it easier to register manual tests in a common format for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
bra...@chromium.org
wrote:
Here's an off-the-top-of-my-head sketch of how we might do Voltron
tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo Automated /
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to the
current
mobile-spec's
- That index reads a file akin to cordova_plugins.js
(cordova_tests.js,
maybe?) generated by the CLI, containing the info from the test
tags.
- It has navigation similar to the current mobile-spec, with
buttons
for the automatic and manual sections. Auto has All and then each
module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana
csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron and be
DRY
and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the future to
include
information about testing (i.e. Directory containing tests files,
test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come with
the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running
(derived
from
Medic) uses only the mobilespec tests. It does not yet use
tests
collected
from the plugins. Its been talked about, but not gone anywhere.
David Kemp
On Wed, Sep 25, 2013 at 7:58 PM, Jesse
purplecabb...@gmail.com
javascript:;
wrote:
Yeah, I have pushed some changes to mobile-spec, and when I
did
I
also
copied the tests into the plugin involved.
Until we get the magic test runner happening, I think we just
keep
duplicating.
@purplecabbage
risingj.com
On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill
stevengil...@gmail.com
javascript:;
wrote:
We copied over tests into plugins when we first broke them
out,
but
I
don't
believe they have been updated.
I would say for now to just add the tests to mobile spec,
and
possibly in
the future we go all voltron to build mobile spec
Re: mobile-spec and releases: How do we test?
Which one?
On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux b...@brian.io wrote:
I really like your proposal as a starting point. Very simple but would
allow for in-app testing as well as on the cmd line if we so wish.
On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny mmo...@chromium.org wrote:
I was looking over some old emails from this list on plugin testing, and
an
idea that was proposed way back was to ship plugin tests as a second
plugin. That way, you can chose to install tests, or not, and know
explicitly if they are being copied into your final project.
An alternative would be to support build targets a la release/debug and
have target-specific plugin.xml tags (assets, js-modules, source-file..).
-Michal
On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux b...@brian.io wrote:
I think this is basically what we've been proposing for a while now.
On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny mmo...@chromium.org
wrote:
I would suggest perhaps a simpler approach, which doesn't add
anything
new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a
convention
of where they should live, and what signature it should have (i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but we
ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed plugin,
runs
them, and aggregates results.
- It could report results to some shared server, allow toggling of
tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of
library
code for creating tests, and plugins can use it to register their
tests.
- This makes it easier to register manual tests in a common format
for
core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
bra...@chromium.org
wrote:
Here's an off-the-top-of-my-head sketch of how we might do Voltron
tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo Automated
/
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to the
current
mobile-spec's
- That index reads a file akin to cordova_plugins.js
(cordova_tests.js,
maybe?) generated by the CLI, containing the info from the test
tags.
- It has navigation similar to the current mobile-spec, with
buttons
for the automatic and manual sections. Auto has All and then each
module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana
csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron and be
DRY
and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the future to
include
information about testing (i.e. Directory containing tests files,
test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come with
the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running
(derived
from
Medic) uses only the mobilespec tests. It does not yet use
tests
collected
from the plugins. Its been talked about, but not gone
anywhere.
David Kemp
On Wed, Sep 25, 2013 at 7:58 PM, Jesse
purplecabb...@gmail.com
javascript:;
wrote:
Yeah, I have pushed some changes to mobile-spec, and when I
did
I
also
copied the tests into the plugin involved.
Until we get the magic test runner happening, I think we
just
keep
duplicating.
@purplecabbage
risingj.com
On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill
stevengil...@gmail.com
javascript:;
wrote:
We copied over tests into plugins when we first
Re: mobile-spec and releases: How do we test?
Currently, the automated test system that we have running (derived from Medic) uses only the mobilespec tests. It does not yet use tests collected from the plugins. Its been talked about, but not gone anywhere. David Kemp On Wed, Sep 25, 2013 at 7:58 PM, Jesse purplecabb...@gmail.com wrote: Yeah, I have pushed some changes to mobile-spec, and when I did I also copied the tests into the plugin involved. Until we get the magic test runner happening, I think we just keep duplicating. @purplecabbage risingj.com On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill stevengil...@gmail.com wrote: We copied over tests into plugins when we first broke them out, but I don't believe they have been updated. I would say for now to just add the tests to mobile spec, and possibly in the future we go all voltron to build mobile spec and keep tests with their corresponding plugins. On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.com wrote: Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe
Re: mobile-spec and releases: How do we test?
What's the challenge of having us use the tests that come with the individual plugins ? On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com wrote: Currently, the automated test system that we have running (derived from Medic) uses only the mobilespec tests. It does not yet use tests collected from the plugins. Its been talked about, but not gone anywhere. David Kemp On Wed, Sep 25, 2013 at 7:58 PM, Jesse purplecabb...@gmail.com wrote: Yeah, I have pushed some changes to mobile-spec, and when I did I also copied the tests into the plugin involved. Until we get the magic test runner happening, I think we just keep duplicating. @purplecabbage risingj.com On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill stevengil...@gmail.com wrote: We copied over tests into plugins when we first broke them out, but I don't believe they have been updated. I would say for now to just add the tests to mobile spec, and possibly in the future we go all voltron to build mobile spec and keep tests with their corresponding plugins. On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.com wrote: Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe
Re: mobile-spec and releases: How do we test?
I like the idea can we call mobilespec now cordova-voltron and be DRY and use the tests form the plugins. Voltron by itself creates an App that tests only core, but as you use plugman to add plugins to voltron it has more test cases. It would not be a bad idea to enhance plugin.xml in the future to include information about testing (i.e. Directory containing tests files, test command, etc..) --Carlos On Thursday, September 26, 2013, Anis KADRI wrote: What's the challenge of having us use the tests that come with the individual plugins ? On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.comjavascript:; wrote: Currently, the automated test system that we have running (derived from Medic) uses only the mobilespec tests. It does not yet use tests collected from the plugins. Its been talked about, but not gone anywhere. David Kemp On Wed, Sep 25, 2013 at 7:58 PM, Jesse purplecabb...@gmail.comjavascript:; wrote: Yeah, I have pushed some changes to mobile-spec, and when I did I also copied the tests into the plugin involved. Until we get the magic test runner happening, I think we just keep duplicating. @purplecabbage risingj.com On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill stevengil...@gmail.comjavascript:; wrote: We copied over tests into plugins when we first broke them out, but I don't believe they have been updated. I would say for now to just add the tests to mobile spec, and possibly in the future we go all voltron to build mobile spec and keep tests with their corresponding plugins. On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.comjavascript:; wrote: Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe -- Carlos Santana csantan...@gmail.com
Re: mobile-spec and releases: How do we test?
Here's an off-the-top-of-my-head sketch of how we might do Voltron tests: - Add a tag to plugin.xml that names each test file: test type=automatic src=spec/foo.js name=Foo Automated / test type=manual src=spec/bar.js name=Foo Manual / - Add a new command, cordova test (maybe prepare-test), that: - Ignores the top-level www. - Instead copies in a basic testing index.html similar to the current mobile-spec's - That index reads a file akin to cordova_plugins.js (cordova_tests.js, maybe?) generated by the CLI, containing the info from the test tags. - It has navigation similar to the current mobile-spec, with buttons for the automatic and manual sections. Auto has All and then each module, manual just has the list of modules. Thoughts? Braden On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana csantan...@gmail.comwrote: I like the idea can we call mobilespec now cordova-voltron and be DRY and use the tests form the plugins. Voltron by itself creates an App that tests only core, but as you use plugman to add plugins to voltron it has more test cases. It would not be a bad idea to enhance plugin.xml in the future to include information about testing (i.e. Directory containing tests files, test command, etc..) --Carlos On Thursday, September 26, 2013, Anis KADRI wrote: What's the challenge of having us use the tests that come with the individual plugins ? On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com javascript:; wrote: Currently, the automated test system that we have running (derived from Medic) uses only the mobilespec tests. It does not yet use tests collected from the plugins. Its been talked about, but not gone anywhere. David Kemp On Wed, Sep 25, 2013 at 7:58 PM, Jesse purplecabb...@gmail.com javascript:; wrote: Yeah, I have pushed some changes to mobile-spec, and when I did I also copied the tests into the plugin involved. Until we get the magic test runner happening, I think we just keep duplicating. @purplecabbage risingj.com On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill stevengil...@gmail.com javascript:; wrote: We copied over tests into plugins when we first broke them out, but I don't believe they have been updated. I would say for now to just add the tests to mobile spec, and possibly in the future we go all voltron to build mobile spec and keep tests with their corresponding plugins. On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.com javascript:; wrote: Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe -- Carlos Santana csantan...@gmail.com
Re: mobile-spec and releases: How do we test?
I would suggest perhaps a simpler approach, which doesn't add anything new
to cordova-cli/plugman:
- Each plugin ships with a tests js-module, and we document a convention
of where they should live, and what signature it should have (i.e.,
cordova.require('plugin.name.Tests').forEach(...) ).
- Will need a common way to describe/report results (others have
mentioned TAP).
- Any app is free to run those plugin tests in any which way, but we ship a
mobile-spec app which is one opinionated way to do so.
- It attempts to require the test module for each installed plugin, runs
them, and aggregates results.
- It could report results to some shared server, allow toggling of tests,
etc, but no plugin should know or care about those features.
Using that as a generic base:
- We ship a CDVTests (or whatever) plugin which has a bunch of library
code for creating tests, and plugins can use it to register their tests.
- This makes it easier to register manual tests in a common format for core
plugins, and prevents code duplication for core auto tests.
- External plugins can chose to use our testing library, or not.
-Michal
On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson bra...@chromium.orgwrote:
Here's an off-the-top-of-my-head sketch of how we might do Voltron tests:
- Add a tag to plugin.xml that names each test file:
test type=automatic src=spec/foo.js name=Foo Automated /
test type=manual src=spec/bar.js name=Foo Manual /
- Add a new command, cordova test (maybe prepare-test), that:
- Ignores the top-level www.
- Instead copies in a basic testing index.html similar to the current
mobile-spec's
- That index reads a file akin to cordova_plugins.js (cordova_tests.js,
maybe?) generated by the CLI, containing the info from the test tags.
- It has navigation similar to the current mobile-spec, with buttons
for the automatic and manual sections. Auto has All and then each module,
manual just has the list of modules.
Thoughts?
Braden
On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana csantan...@gmail.com
wrote:
I like the idea can we call mobilespec now cordova-voltron and be DRY and
use the tests form the plugins.
Voltron by itself creates an App that tests only core, but as you
use plugman to add plugins to voltron it has more test cases.
It would not be a bad idea to enhance plugin.xml in the future to include
information about testing (i.e. Directory containing tests files, test
command, etc..)
--Carlos
On Thursday, September 26, 2013, Anis KADRI wrote:
What's the challenge of having us use the tests that come with the
individual plugins ?
On Thu, Sep 26, 2013 at 8:13 AM, David Kemp drk...@google.com
javascript:;
wrote:
Currently, the automated test system that we have running (derived
from
Medic) uses only the mobilespec tests. It does not yet use tests
collected
from the plugins. Its been talked about, but not gone anywhere.
David Kemp
On Wed, Sep 25, 2013 at 7:58 PM, Jesse purplecabb...@gmail.com
javascript:;
wrote:
Yeah, I have pushed some changes to mobile-spec, and when I did I
also
copied the tests into the plugin involved.
Until we get the magic test runner happening, I think we just keep
duplicating.
@purplecabbage
risingj.com
On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill
stevengil...@gmail.com
javascript:;
wrote:
We copied over tests into plugins when we first broke them out,
but
I
don't
believe they have been updated.
I would say for now to just add the tests to mobile spec, and
possibly in
the future we go all voltron to build mobile spec and keep tests
with
their
corresponding plugins.
On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.com
javascript:;
wrote:
Hey
Right now, I'm working on a weird file issue that requires me to
update mobile-spec, but I'm wondering where the tests should
live.
Should it all keep living in mobile-spec, or is it with the
plugins.
And if it's with the plugins, will there be scripts to assemble
mobile-spec all Voltron style?
This came up earlier, but I haven't found any fix that needed a
mobile-spec test. (Many that need native testing, like
recursive
file
copy, etc). Any thoughts?
Joe
--
Carlos Santana
csantan...@gmail.com
mobile-spec and releases: How do we test?
Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe
Re: mobile-spec and releases: How do we test?
We copied over tests into plugins when we first broke them out, but I don't believe they have been updated. I would say for now to just add the tests to mobile spec, and possibly in the future we go all voltron to build mobile spec and keep tests with their corresponding plugins. On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.com wrote: Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe
Re: mobile-spec and releases: How do we test?
Yeah, I have pushed some changes to mobile-spec, and when I did I also copied the tests into the plugin involved. Until we get the magic test runner happening, I think we just keep duplicating. @purplecabbage risingj.com On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill stevengil...@gmail.com wrote: We copied over tests into plugins when we first broke them out, but I don't believe they have been updated. I would say for now to just add the tests to mobile spec, and possibly in the future we go all voltron to build mobile spec and keep tests with their corresponding plugins. On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser bows...@gmail.com wrote: Hey Right now, I'm working on a weird file issue that requires me to update mobile-spec, but I'm wondering where the tests should live. Should it all keep living in mobile-spec, or is it with the plugins. And if it's with the plugins, will there be scripts to assemble mobile-spec all Voltron style? This came up earlier, but I haven't found any fix that needed a mobile-spec test. (Many that need native testing, like recursive file copy, etc). Any thoughts? Joe
