Wow. I don’t think I ever noticed the tutorial for FlexUnit on the Flex site: https://flex.apache.org/flexunit/tutorial/ <https://flex.apache.org/flexunit/tutorial/>
Who put that together? Looks like a lot of work went into that. Harbs > On Nov 4, 2017, at 6:59 PM, Harbs <harbs.li...@gmail.com> wrote: > > By “Unit Tests”, I really meant tests which are UI agnostic. I’m not > concerned by size of “Unit” tests, I’m more concerned about feature coverage. > When I think of unit tests in this context I mean FlexUnit-type tests. The > problem with FlexUnit, is that it only tests in Flash. Josh’s work solves > that problem. Ideally, we’d have tests which are run on both platforms. I > think We can use FlexUnit on Flash and Josh’s implementation to run the exact > same tests in node. > > To me, it’s easier to tackle this problem if we first get “unit” tests > working that are platform independent. > > Once that’s working, we can figure out how much the Unit-type tests can be > extended to test UI features and/or Mustella can be used. Mustella is a big > mystery to me. > > Whatever the strategy, the tests need to be easy to write. Maybe it already > is easy to write mustella tests, but we need more documentation. I don’t know. > > Either way, I will try to add Josh’s code and write some tests and see how it > goes. > > Harbs > >> On Nov 3, 2017, at 6:26 PM, Alex Harui <aha...@adobe.com.INVALID> wrote: >> >> I was unable to tell what it is Josh has from that link. >> >> Right now, we have 3 test subsystems: >> 1) Java-based tests using Selenium that Chris Dutz wrote >> 2) FlexUnit tests (written in AS3) that only work in Flash (in the Basic >> project) >> 3) Mustella tests that are written in MXML that run on both Flash and in >> the browser using Selenium. >> >> This thread is titled "Unit Tests" Do we have agreement on what the size >> of the unit is? >> >> Is there any advantage to trying to get FlexUnit to work without Flash? >> For example, would folks migrating have old FlexUnit tests they want to >> run? >> >> I agree with Piotr that it would be better if tests didn't have to be >> written in Java. Using a declarative language like MXML makes you more >> output-language agnostic. Mustella works by having each Mustella command >> implemented in Java for Selenium. It constrains what your test can do to >> things we know we can do on all platforms, and is agnostic about when the >> next command run, which is helpful when there are differences in the way >> the runtimes execute code and make results available. I'd have to dig >> into it again, but I think there are some issues in failure reporting in >> Selenium of Selenium thinks there is only one test step which is Java code >> that shoved a pile of JS into the browser to be executed. >> >> It would be nice if we can leverage what we have instead of building a >> whole new test system, but maybe Josh has something that is closer to what >> we are looking for. >> >> My 2 cents, >> -Alex >> >> On 11/3/17, 8:37 AM, "Josh Tynjala" <joshtynj...@apache.org> wrote: >> >>> Please feel free to use my test runner. I'm happy to commit it somewhere >>> to make it official Apache code, if necessary. >>> >>> - Josh >>> >>> On 2017-11-03 04:19, Harbs <harbs.li...@gmail.com> wrote: >>>> One topic which keeps coming up is better test coverage for Royale. >>>> >>>> I think this is becoming a critical issue for a few reasons: >>>> 1. As we get close to version 1.0 it’s necessary to have good test >>>> coverage for confidence of quality and confidence that we don’t >>>> introduce recessive bugs. >>>> 2. It’s really hard to accept Github pull requests without examining >>>> the code VERY well that it does not introduce recessive bugs. CI which >>>> runs automated tests could give a preliminary test on pull requests to >>>> ensure that they don’t break anything. If the pull requests do break >>>> something, it allows the requester to fix the problem with confidence >>>> without taking others’ time. >>>> >>>> I think we need to break up testing into pieces and figure out a >>>> strategy to implement automated tests in a way that they are >>>> maintainable. >>>> >>>> Some points: >>>> 1. I think integration into something like Travis would be very helpful. >>>> 2. I think there’s a Jenkins plugin for building pull requests. Not >>>> sure how useful it is.[1] >>>> 3. Josh has created a Node.js-compatible test-runner architecture which >>>> could be useful for unit tests on parts of the framework which don’t >>>> rely on browser features. (i.e. models and the like) He mentioned the >>>> possibility of donating it, and I think that would be a very useful >>>> feature. >>>> 4. For UI and integration tests, there seem to be some pretty cool >>>> integrations using Selenium.[2][3] >>>> 5. I think the main testing effort needs to be using JS and something >>>> like Josh’s testing framework for non-UI pieces and some easy-to-use >>>> Selenium integration for visual pieces and integration tests. >>>> 6. We probably also want some API endpoints we can test off of for >>>> integration testing. >>>> >>>> I’m willing to invest time into this. >>>> >>>> It’s going to be a lot of work building out the tests and I think we >>>> need a plan for that. My thoughts: >>>> >>>> 1. Step one is to make it easy to write meaningful automated tests and >>>> establish a clear pattern. >>>> 2. Step two is to start writing tests starting from the >>>> most-used/easiest to beak pieces and work out from there. >>>> 3. Once the pattern is established, any new pieces MUST have testing >>>> coverage. >>>> 4. When fixing bugs, attention should be paid to adding testing for >>>> that component. >>>> 5. When a pull request comes in on a piece which does not have unit >>>> test, a test must be written before accepting the pull request. The test >>>> does not need to be written by the requester. Before examining the >>>> request, the test should be written to pass for expected behavior and >>>> fail for the bug that the pull request is attempting to fix (assuming >>>> the pull request is to fix a bug). >>>> >>>> Thoughts? >>>> Harbs >>>> >>>> P.S. I’m thinking of coming to the US in late December/early January. >>>> I would be interested in getting together for a hacking session with >>>> folks who are available. >>>> >>>> >>>> [1]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki. >>>> jenkins.io%2Fdisplay%2FJENKINS%2FGitHub%2Bpull%2Brequest%2Bbuilder%2Bplug >>>> in&data=02%7C01%7C%7C36d681ea88ba4315312f08d522d0bf50%7Cfa7b1b5a7b3443879 >>>> 4aed2c178decee1%7C0%7C0%7C636453202314829220&sdata=dqT0Nao8HHzlkWmk6zy%2F >>>> eDEGi6nXYkM8P0%2Bbibe4NA4%3D&reserved=0 >>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.je >>>> nkins.io%2Fdisplay%2FJENKINS%2FGitHub%2Bpull%2Brequest%2Bbuilder%2Bplugin >>>> &data=02%7C01%7C%7C36d681ea88ba4315312f08d522d0bf50%7Cfa7b1b5a7b34438794a >>>> ed2c178decee1%7C0%7C0%7C636453202314829220&sdata=dqT0Nao8HHzlkWmk6zy%2FeD >>>> EGi6nXYkM8P0%2Bbibe4NA4%3D&reserved=0> >>>> >>>> [2]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs. >>>> travis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%7C01%7C%7C36d >>>> 681ea88ba4315312f08d522d0bf50%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0% >>>> 7C636453202314829220&sdata=9D9riTJ29rqNonT81v%2FRtdpC6J8titW7LIUb07LLXa4% >>>> 3D&reserved=0 >>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.tr >>>> avis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%7C01%7C%7C36d68 >>>> 1ea88ba4315312f08d522d0bf50%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C >>>> 636453202314829220&sdata=9D9riTJ29rqNonT81v%2FRtdpC6J8titW7LIUb07LLXa4%3D >>>> &reserved=0> >>>> >>>> [3]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsauce >>>> labs.com%2Fproducts%2Fintegrations&data=02%7C01%7C%7C36d681ea88ba4315312f >>>> 08d522d0bf50%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636453202314829 >>>> 220&sdata=WNzEofopPA6YaHS51O%2BCx6Zrsc2qLOFcSWMUUXssD3M%3D&reserved=0 >>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsaucela >>>> bs.com%2Fproducts%2Fintegrations&data=02%7C01%7C%7C36d681ea88ba4315312f08 >>>> d522d0bf50%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63645320231482922 >>>> 0&sdata=WNzEofopPA6YaHS51O%2BCx6Zrsc2qLOFcSWMUUXssD3M%3D&reserved=0> >> >
