Sorry for the late response. Yes, go ahead! - Josh
On 2017-11-04 11:30, Harbs <harbs.li...@gmail.com> wrote: > Awesome! > > I assume that this means I can code the code to the Royale repo under Apache > 2. Right? > > Harbs > > > On Nov 3, 2017, at 5:37 PM, Josh Tynjala <joshtynj...@apache.org> wrote: > > > > Please feel free to use my test runner. I'm happy to commit it somewhere to > > make it official Apache code, if necessary. > > > > - Josh > > > > On 2017-11-03 04:19, Harbs <harbs.li...@gmail.com> wrote: > >> One topic which keeps coming up is better test coverage for Royale. > >> > >> I think this is becoming a critical issue for a few reasons: > >> 1. As we get close to version 1.0 itâÂÂs necessary to have good test > >> coverage for confidence of quality and confidence that we donâÂÂt > >> introduce recessive bugs. > >> 2. ItâÂÂs really hard to accept Github pull requests without examining > >> the code VERY well that it does not introduce recessive bugs. CI which > >> runs automated tests could give a preliminary test on pull requests to > >> ensure that they donâÂÂt break anything. If the pull requests do break > >> something, it allows the requester to fix the problem with confidence > >> without taking othersâ time. > >> > >> I think we need to break up testing into pieces and figure out a strategy > >> to implement automated tests in a way that they are maintainable. > >> > >> Some points: > >> 1. I think integration into something like Travis would be very helpful. > >> 2. I think thereâÂÂs a Jenkins plugin for building pull requests. Not > >> sure how useful it is.[1] > >> 3. Josh has created a Node.js-compatible test-runner architecture which > >> could be useful for unit tests on parts of the framework which donâÂÂt > >> rely on browser features. (i.e. models and the like) He mentioned the > >> possibility of donating it, and I think that would be a very useful > >> feature. > >> 4. For UI and integration tests, there seem to be some pretty cool > >> integrations using Selenium.[2][3] > >> 5. I think the main testing effort needs to be using JS and something like > >> JoshâÂÂs testing framework for non-UI pieces and some easy-to-use > >> Selenium integration for visual pieces and integration tests. > >> 6. We probably also want some API endpoints we can test off of for > >> integration testing. > >> > >> IâÂÂm willing to invest time into this. > >> > >> ItâÂÂs going to be a lot of work building out the tests and I think we > >> need a plan for that. My thoughts: > >> > >> 1. Step one is to make it easy to write meaningful automated tests and > >> establish a clear pattern. > >> 2. Step two is to start writing tests starting from the most-used/easiest > >> to beak pieces and work out from there. > >> 3. Once the pattern is established, any new pieces MUST have testing > >> coverage. > >> 4. When fixing bugs, attention should be paid to adding testing for that > >> component. > >> 5. When a pull request comes in on a piece which does not have unit test, > >> a test must be written before accepting the pull request. The test does > >> not need to be written by the requester. Before examining the request, the > >> test should be written to pass for expected behavior and fail for the bug > >> that the pull request is attempting to fix (assuming the pull request is > >> to fix a bug). > >> > >> Thoughts? > >> Harbs > >> > >> P.S. IâÂÂm thinking of coming to the US in late December/early January. > >> I would be interested in getting together for a hacking session with folks > >> who are available. > >> > >> [1]https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin > >> > >> <https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin> > >> [2]https://docs.travis-ci.com/user/gui-and-headless-browsers/ > >> <https://docs.travis-ci.com/user/gui-and-headless-browsers/> > >> [3]https://saucelabs.com/products/integrations > >> <https://saucelabs.com/products/integrations> > >
