testing is good, no need to hide it,
it would be good though to not copy it with the rendered app
Dnia Thu Jun 19 19:11:25 2014 purplecabbage pisze:
I think we just lead by example.
Sent from my iPhone
On Jun 19, 2014, at 6:18 PM, Michal Mocny <mmo...@chromium.org> wrote:
+1 I agree, this would be awesome.
New question, should this merely be the "standard" we adhere to for core
plugins, or should we actively make it difficult for plugin devs to not
ship tests directly with plugins? (Not sure how we could accomplish that,
so I hope its just a convention that applies to our work).
-Michal
On Thu, Jun 19, 2014 at 7:48 PM, Jesse <purplecabb...@gmail.com> wrote:
My ultimate would be this:
cordova create TestFilePlugin
cd TestFilePlugin
cordova platform add android
cordova plugin add
http://git-wip-us.apache.org/repos/asf/cordova-labs.git#cdvtest:cordova-plugin-test-framework
cordova plugin add ../cordova-plugin-file/
cordova plugin add ../cordova-plugin-file/test/
cordova run android --start cdvtests/index.html
Then do this for each plugin, and for each platform
Then do this for all combinations of plugins
...
Note the run --start does not yet exist, but this would be awesome!
@purplecabbage
risingj.com
On Thu, Jun 19, 2014 at 4:15 PM, Jesse <purplecabb...@gmail.com> wrote:
Option a) was what I suggested a ways back, and I still stand by it.
I think it provides the greatest transparency, and simplicity, yet it is
still very flexible.
I don't think it would be hard to accomplish either. This is the small
re-org I was hinting at, you've already done the hard part.
@purplecabbage
risingj.com
On Thu, Jun 19, 2014 at 3:45 PM, Michal Mocny <mmo...@chromium.org>
wrote:
Andrew has raised that concern as well. My gut says that the bundling
of
a
few shorts scripts that get parsed but not run as long as they don't get
require() will not affect applications negatively (there are probably
many
more significant overheads we live with today in cordova) -- but I
understand why that may not be useful default.
In that case, some ideas: (I recall these were proposed previously but
not
sure by whom)
(a) Bundle tests as a plugin-within-the-plugin as such:
myplugin/
- plugin.xml
- src/...
- www/...
- tests/
- plugin.xml
- www/...
Which basically means the plugin tests live in the same repo/branch, and
are fetched as part of "plugin add", but are not moved into platforms/
on
cordova prepare by default, thus don't end up in your application (disk
and
network are cheap, application startup and size costs are not, right?).
Then, to run tests, we basically need to iterate all plugins looking
for a
nested tests/plugin.xml, and install those. This can be added to
CLI/Plugman, or just be a CLI hook even.
(b) add a <js-test-module> or <js-module type="test"> that is only used
if
you run prepare with --test. Similar to the above, but I think requires
more CLI/config file changes, which I'm not a fan of.
(c) Just ship tests as a second plugin in a second repo, and document
how
to install tests. Can then perhaps have a <dependency type=tests>. I
don't like this as much since its basically back to mobile-spec.
WDYT?
-Michal
On Thu, Jun 19, 2014 at 4:53 PM, Jesse <purplecabb...@gmail.com> wrote:
re:
Q: What do I do if my plugin tests must have very large assets?
- A: Don't bundle those assets with your plugin. If you can, have
your
...
My concern is I do not want to see tests added to every project that
uses a
plugin, even if the assets are not large, there are implications to
including the test framework + all the tests because they get loaded
and
processed with all of the plugins and will impact load time even if
never
run.
99.9% of the time the plugin tests will be used by us the plugin
developers, and not the people who use the plugin in there apps.
I agree, having the tester install the test harness plugin dependency
themselves is probably a better option, as I see you have wrapped all
tests
inside a exports.defineAutoTests so we don't have to worry about
describe/it/expects not being defined.
@purplecabbage
risingj.com
On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmo...@chromium.org>
wrote:
Jesee, the branch is NOT a requirement (I don't think I documented
it
as
such, except in the examples for installing plugins for initial
look).
Actually, we should delete those stale branches now that we are
moving
up-to-date tests into master. It was just for early experimentation
on
the
feature.
Jesse, I'm not seeing the benefit of using plugins-tests.xml or the
dependency on the test plugin yet, may you elaborate your thoughts?
My hope was that tests are just always installed alongside plugins.
If
that is not a good idea for some particular plugin, say because it
uses
huge assets, I elaborated my answer in the plugin FAQ (
https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq
):
FAQ
-
Q: Should I add org.apache.cordova.test-harness as a <dependancy>
of
my
plugin?
- A: No. The end-user should decide if they want to install the
test
harness, not your plugin (most users won't).
-
Q: What do I do if my plugin tests must have very large assets?
- A: Don't bundle those assets with your plugin. If you can, have
your
tests fail gracefully if those assets don't don't exist
(perhaps
log
a
warning, perhaps fail a single asset-checking test, and skip
the
rest).
Then, ideally download those assets automatically into local
storage
the
first time tests run. Or create a manual test step to download
and install
assets. As a final alternative, split those test assets into a
separate
plugin, and instruct users to install that plugin to run your
full
test
suite.
-
Q: Should I ship my app with the test harness plugin installed?
- A: Not likely. If you want, you can. Then your app could even
embed
a
link to the test page (cdvtests/index.html) from a help
section
of
your app, to give end users a way to run your test suite out
in
the feild.
That may help diagnose causes of issues within your app.
Maybe.
=============
Feel free the debate those answers -- now is certainly the time --
but I
put a lot of effort to make it super flexible and to not require
depending
on changes to CLI :P
-Michal
On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabb...@gmail.com>
wrote:
Sorry I missed providing feedback on this earlier ...
Having a deeper look at this, I am not feeling great about the
extra
requirement that every plugin have an additional branch.
Several concerns arise :
- test branch can be out of sync with master
- how do we test a specific version?
- tests are not immediately visible when looking at master
- differing versions of plugin.xml depending on the branch
The majority of the work has been done (thanks Michal!), and
mostly
any
suggestions I make will just require moving code and changing a
few
conventions.
What if we? :
1. add a plugin-test.xml file which has the exact format of
plugin.xml
2. keep tests/ and plugin-test.xml file in master branch
3. have plugman/cli support an additional flag --test so we can
install
like this:
cordova plugin add
http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git
--test
This would mean that in addition to processing the plugin.xml of
the
plugin, we would also process the plugin-test.xml file ( identical
processing logic )
4. have all plugin-test.xml files declare a dependency on
cordova-plugin-test-framework
The above suggestions could also be used in conjuction with the
cordova
run
--tests platform mentioned by Michal, but without the need to
manage
the
switching of branches.
@purplecabbage
risingj.com
On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny <
mmo...@chromium.org>
wrote:
Piotr: Actually I'm not sure how running tests in the harness
would
work,
since the path to the resource may be different. However, in
general,
with
development using the harness you aren't making any changes to
plugins.
The whole point is for app developers who want to modify only
web
application bits and not deal with native compiles.
In theory the app harness could support working on the
js-modules
of
plugins, but that sounds like a really niche idea. I'd not be
opposed
to
someone working on it but I'm not sure you'll have luck finding
volunteers.
-Michal
On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny <
mmo...@chromium.org>
wrote:
At the time I went through my design iterations I just didn't
want
to
necessarily depend on cordova tooling changes / documentation.
In
other
words, someone else may have a different strategy for
testing..
My personal opinion would be have the test plugin ship with a
plugin
hook
(those are in, right? or at least on their way), that will
automatically
update the start page if you pass a flag to run command. Ie,
in an
ideal
world: `cordova run --tests` runs a plugin hook passing in
--tests
flag
which changes the start page, in a way that isn't overwritten
by
the
top-level config.xml.
My 2 cents, since I don't want "our way" of testing mobile
spec
to
be
"the
only way" to test. Frameworks and opinions on testing
change.
-Michal
On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa <
pzal...@mozilla.com
wrote:
One thing more - it would be great if user could create a
test
using
test
harness app as well. Is it also considered?
Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez pisze:
It would be a nice to have in the cli, aimed to just setup
the
right
path
in the config.xml, maybe along with an another argument to
build,
run/emulate as well.
It sounds great.
2014-06-17 15:21 GMT-05:00 Piotr Zalewa <
pzal...@mozilla.com
:
Thanks Martin,
Has it been considered to create a separate command
"testrun" or
similar
which would remove the need to edit the config.xml?
Dnia Tue Jun 17 11:58:33 2014 Michal Mocny pisze:
Martin, thanks for posting those links.
And I'll look into the INFRA tickets I need to file to set
up a
repo
for
that plugin, since its ready to come out of labs.
On Tue, Jun 17, 2014 at 2:06 PM, Martin Gonzalez <
martin.c.glez.g...@gmail.com> wrote:
This is the Cordova Plugin Test Framework readme.md,
you
can
catch
up
with
the functionality by reading some of the content:
Repository:
https://github.com/apache/cordova-labs
Docs:
https://github.com/apache/cordova-labs/blob/master/README.md
https://github.com/apache/cordova-labs/blob/cdvtest/
cordova-plugin-test-framework/README.md
2014-06-17 12:56 GMT-05:00 Piotr Zalewa <
pzal...@mozilla.com
:
a documentation explaining how it's gonna work
Dnia Tue Jun 17 10:51:58 2014 Michal Mocny pisze:
What do you mean?
On Tue, Jun 17, 2014 at 1:50 PM, Piotr Zalewa <
pzal...@mozilla.com>
wrote:
Is there any predev document?
Dnia Mon Jun 16 18:30:46 2014 Andrew Grieve pisze:
Yeah, really exciting. Thanks for taking this on.
On Mon, Jun 16, 2014 at 3:42 PM, Michal Mocny <
mmo...@chromium.org>
wrote:
Fantastic!
I'll try to keep an eye out on the PR's, and please
ping
me
if
you
would
like any help.
-Michal
On Mon, Jun 16, 2014 at 3:25 PM, Marcel Kinard <
cmarc...@gmail.com>
wrote:
Hi, after some discussions here with IBM
management,
we’re
going
to
bring
in a couple extra interns for a week to jumpstart
the
migration
of
the
tests out of mobile-spec into the new plugin test
framework.
Staci
Cooper
will be leading this effort, and Martin Gonzalez
will
be a
part
of
it.
So if you see a bunch of pull requests, this is what it
is
for.
We’ll
get
the interns to submit an ICLA asap.
--
Piotr Zalewa
Mozilla
--
Piotr Zalewa
Mozilla
--
Regards,
Martin Gonzalez
--
Piotr Zalewa
Mozilla
--
Piotr Zalewa
Mozilla
--
Piotr Zalewa
Mozilla
