[
https://issues.apache.org/jira/browse/CB-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063534#comment-16063534
]
Filip Maj commented on CB-12950:
--------------------------------
Note that the e2e tests _require_ that the Android SDK be installed and usable
- thus why these tests are expensive (and maybe why we don't run them on
appveyor?).
>From what I can tell:
- the create e2e tests exercise whether we can create and compile a project
with a combination of ascii, unicode, and space characters using the CLI
scripts. Is this a good use of compute time on VMs with the SDK? i have a
feeling that the specific logic being exercised by these tests (the
unicode/ascii/spaces-in-names) could be covered via unit tests. That said, the
fact that a newly-created project _compiles_ is probably a good test.
- the plugin e2e test (there's only one) exercises whether we can add a "fake"
plugin. Is this test valuable? I think a true integration test of
cordova-android in combination with core cordova-plugins would be a much more
valuable test (and IMO would need to live _outside_ of the cordova-android
repository)
- the update e2e test (again, only one) exercises whether the latest update
script works when upgrading from cordova-android@4.0.0, and then whether the
`cordova/version` CLI script returns the proper version post-upgrade. With the
way the upgrade command is going and the direction the discussion is heading
for that command, it sounds like this test will not be needed in the near
future.
All in all, these tests take 600+ seconds on my mac laptop.
I think reviewing these tests, and identifying the critical use cases within
cordova-android that need testing on specific environments, would be healthy.
Off the top of my head: creating a project on windows+unix, compiling a project
on windows+unix (would require Android SDK) would likely be the two initial use
cases I would suggest to use for e2e tests. These would be especially valuable
if we could have these tests run on CI on Linux, Mac and Windows, but, we'd
need to find a CI service we could leverage that would provide that. Travis
gives us what we need to have Linux covered, but I'm not sure if appveyor is
able to give us the Android SDK requirements.
> test-build e2e tests in cordova-android not running in CI
> ---------------------------------------------------------
>
> Key: CB-12950
> URL: https://issues.apache.org/jira/browse/CB-12950
> Project: Apache Cordova
> Issue Type: Bug
> Components: cordova-android
> Reporter: Filip Maj
> Assignee: Filip Maj
>
> Looks like because jasmine removed support for the CLI flag
> {{--captureExceptions}} in a minor version bump, the CI is no longer
> executing the {{test-build}} npm task, which runs e2e tests.
> Should probably:
> - fix this, but probably more importantly
> - look at whether these tests are needed? Running them is in the travis job
> description, but not in the appveyor one, which means we are not getting this
> test coverage on Windows in CI anyways.
> - perhaps the test coverage here can be provided by unit testing the CLI
> scripts more fully, rather than relying on e2e tests?
> Ping [~bowserj].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@cordova.apache.org
For additional commands, e-mail: issues-h...@cordova.apache.org
