Hi Harbs, Fully agree what you are saying! Sometimes I have found myself difficult without deeper knowledge how something is working when I check the code. One thing which I can mention is that we already have Selenium integrated. If you are going to spend some time on that I would start from the point [1] and see whether we can use it.
As for your coming to US - Wish I could be there! [1] https://github.com/apache/royale-asjs/tree/develop/examples/examples-integrationtests Piotr 2017-11-03 12:19 GMT+01:00 Harbs <harbs.li...@gmail.com>: > One topic which keeps coming up is better test coverage for Royale. > > I think this is becoming a critical issue for a few reasons: > 1. As we get close to version 1.0 it’s necessary to have good test > coverage for confidence of quality and confidence that we don’t introduce > recessive bugs. > 2. It’s really hard to accept Github pull requests without examining the > code VERY well that it does not introduce recessive bugs. CI which runs > automated tests could give a preliminary test on pull requests to ensure > that they don’t break anything. If the pull requests do break something, it > allows the requester to fix the problem with confidence without taking > others’ time. > > I think we need to break up testing into pieces and figure out a strategy > to implement automated tests in a way that they are maintainable. > > Some points: > 1. I think integration into something like Travis would be very helpful. > 2. I think there’s a Jenkins plugin for building pull requests. Not sure > how useful it is.[1] > 3. Josh has created a Node.js-compatible test-runner architecture which > could be useful for unit tests on parts of the framework which don’t rely > on browser features. (i.e. models and the like) He mentioned the > possibility of donating it, and I think that would be a very useful feature. > 4. For UI and integration tests, there seem to be some pretty cool > integrations using Selenium.[2][3] > 5. I think the main testing effort needs to be using JS and something like > Josh’s testing framework for non-UI pieces and some easy-to-use Selenium > integration for visual pieces and integration tests. > 6. We probably also want some API endpoints we can test off of for > integration testing. > > I’m willing to invest time into this. > > It’s going to be a lot of work building out the tests and I think we need > a plan for that. My thoughts: > > 1. Step one is to make it easy to write meaningful automated tests and > establish a clear pattern. > 2. Step two is to start writing tests starting from the most-used/easiest > to beak pieces and work out from there. > 3. Once the pattern is established, any new pieces MUST have testing > coverage. > 4. When fixing bugs, attention should be paid to adding testing for that > component. > 5. When a pull request comes in on a piece which does not have unit test, > a test must be written before accepting the pull request. The test does not > need to be written by the requester. Before examining the request, the test > should be written to pass for expected behavior and fail for the bug that > the pull request is attempting to fix (assuming the pull request is to fix > a bug). > > Thoughts? > Harbs > > P.S. I’m thinking of coming to the US in late December/early January. I > would be interested in getting together for a hacking session with folks > who are available. > > [1]https://wiki.jenkins.io/display/JENKINS/GitHub+pull+ > request+builder+plugin <https://wiki.jenkins.io/ > display/JENKINS/GitHub+pull+request+builder+plugin> > [2]https://docs.travis-ci.com/user/gui-and-headless-browsers/ < > https://docs.travis-ci.com/user/gui-and-headless-browsers/> > [3]https://saucelabs.com/products/integrations <https://saucelabs.com/ > products/integrations> -- Piotr Zarzycki mobile: +48 880 859 557 skype: zarzycki10 LinkedIn: http://www.linkedin.com/piotrzarzycki <https://pl.linkedin.com/in/piotr-zarzycki-92a53552> GitHub: https://github.com/piotrzarzycki21
