Please feel free to use my test runner. I'm happy to commit it somewhere to make it official Apache code, if necessary.
- Josh On 2017-11-03 04:19, Harbs <harbs.li...@gmail.com> wrote: > One topic which keeps coming up is better test coverage for Royale. > > I think this is becoming a critical issue for a few reasons: > 1. As we get close to version 1.0 itâs necessary to have good test coverage > for confidence of quality and confidence that we donât introduce recessive > bugs. > 2. Itâs really hard to accept Github pull requests without examining the > code VERY well that it does not introduce recessive bugs. CI which runs > automated tests could give a preliminary test on pull requests to ensure that > they donât break anything. If the pull requests do break something, it > allows the requester to fix the problem with confidence without taking > othersâ time. > > I think we need to break up testing into pieces and figure out a strategy to > implement automated tests in a way that they are maintainable. > > Some points: > 1. I think integration into something like Travis would be very helpful. > 2. I think thereâs a Jenkins plugin for building pull requests. Not sure > how useful it is.[1] > 3. Josh has created a Node.js-compatible test-runner architecture which could > be useful for unit tests on parts of the framework which donât rely on > browser features. (i.e. models and the like) He mentioned the possibility of > donating it, and I think that would be a very useful feature. > 4. For UI and integration tests, there seem to be some pretty cool > integrations using Selenium.[2][3] > 5. I think the main testing effort needs to be using JS and something like > Joshâs testing framework for non-UI pieces and some easy-to-use Selenium > integration for visual pieces and integration tests. > 6. We probably also want some API endpoints we can test off of for > integration testing. > > Iâm willing to invest time into this. > > Itâs going to be a lot of work building out the tests and I think we need a > plan for that. My thoughts: > > 1. Step one is to make it easy to write meaningful automated tests and > establish a clear pattern. > 2. Step two is to start writing tests starting from the most-used/easiest to > beak pieces and work out from there. > 3. Once the pattern is established, any new pieces MUST have testing coverage. > 4. When fixing bugs, attention should be paid to adding testing for that > component. > 5. When a pull request comes in on a piece which does not have unit test, a > test must be written before accepting the pull request. The test does not > need to be written by the requester. Before examining the request, the test > should be written to pass for expected behavior and fail for the bug that the > pull request is attempting to fix (assuming the pull request is to fix a bug). > > Thoughts? > Harbs > > P.S. Iâm thinking of coming to the US in late December/early January. I > would be interested in getting together for a hacking session with folks who > are available. > > [1]https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin > <https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin> > [2]https://docs.travis-ci.com/user/gui-and-headless-browsers/ > <https://docs.travis-ci.com/user/gui-and-headless-browsers/> > [3]https://saucelabs.com/products/integrations > <https://saucelabs.com/products/integrations>
