> Hi Tigran, > > Thank you for your offer of a special license of RIATest for Apache > Flex. > I'm not familiar with RIATest at all. It appears to be more like what > I would call an application testing framework like QTP. Is that > correct?
Correct. RIATest does functional testing of GUI just like QTP does. > One of the reasons Adobe wrote Mustella was to get exact > control over the timing of frame events relative to other faked mouse > and keyboard events. Can RIATest do something like that? We may still > need that for testing the AS-side of the framework. No, RIATest cannot do that, so it is certainly not a replacement for Mustella. RIATest simulates user actions and is not aware of frame timing. For Flex applications it uses Flex Automation framework to simulate the actions. For HTML pages it simulates DOM events directly on HTML DOM elements. It can also simulate system-level mouse and keyboard events on any Windows GUI object if needed. > Regardless, I am still interested in RIATest in two ways: > 1) RIATest as the way to test sample apps. I believe that would be the use for RIATest. It can simulate actions on sample Flex applications compiled to SWF and HTML/JS targets loaded in a browser (Chrome, Firefox and IE) to ensure they are functioning as expected. > 2) I would love to see what changes we would need to make to FlexJS to > get RIATest to work in our workflow so that folks can use RIATest to > test their applications. Actually strictly speaking no changes are mandatory. See sample test I attached to https://issues.apache.org/jira/browse/FLEX-33489 and which tests the output of FlexTestAgain example compiled with current FalconJS compiler. However see below for desirable changes. > I was not directly involved in the QTP automation capability of the > current Flex SDK, but I think there were two main complaints about the > way Adobe implemented it: > A) Lots of automation code is baked into the framework whether you use > it or not > B) You have to choose at compile time whether to add in the rest of the > automation code. Correct. You need to compile the application with a few SWCs that contain automation delegate classes for all Flex components and the core Flex Automation framework classes. But see below on another option. > I'm hoping that the "beads" plug-in model will solve these two issues, > but I can't see myself having time to actually try it any time soon. > It appears that RIATest doesn't require additional properties on the > component (other > than the id property you mentioned). Does RIATest require other > properties > on the components? RIATest does not require additional properties and it is possible to write tests on objects which are missing ids however having uniquely identifying and descriptive text assigned to the 'id' or other publicly accessible property greatly simplifies test automation and makes test scripts more descriptive and less sensitive to application changes. This is generally true for any GUI test automation not just for RIATest. RIATest scripts can use any public property (or a combination of properties) to identify the acting object however the 'id' property when present is used by default for the HTML elements. Flex Automation framework also supports 'automationName' and 'automationValue' property. You can either explicitly set values of these properties in your MXML or ActionScript for all UI components derived from UIComponent. Alternatively 'automationName' or 'automationValue' will be derived from other public properties such as 'text' or 'label'. The 'automationName' is the default property that is used by RIATest to identify Flex objects during tests (although it can use any combination of public properties and it is possible to perform strict or regex matching). It would be great if FalconJS honored 'automationName' and emitted corresponding 'data-automation-name' property on DOM elements. If we go a step further and also emit Flex class name in another property (perhaps in the 'class' property of DOM elements) we could make RIATest automatically recognize which Flex component does each DOM element represent and even aim to have a universal syntax for test scripts that can be run against both SWF target and JS target of the same Flex application. What this basically means is that you can write a test script on a SWF target and then make sure the same script runs exactly the same way on the JS target. That coupled with the ability to run the tests on multiple browsers and 2 operating systems will give a very powerful way to validate the JS output and ensure it works exactly the same way as the SWF output of the same Flex app. This will require modifications to RIATest but the end result will be really exciting so I think we can consider adding such capability to an upcoming RIATest version. > Does it run against a production SWF so it doesn't > have issue B? It doesn't have that issue. One of the options in RIATest is running against a production SWF, so it is not necessary to make the application ready for automation at compile time. Technically RIATest first loads a trivial application containing one full screen SWFLoader component and compiled with necessary automation libraries. This SWFLoader in turn loads the end user's SWF application to test. This way the loaded SWF application becomes fully exposed to Flex Automation framework and testable by RIATest. This option can be used as long as the Flex application is loadable by SWFLoader and does not change its behavior when loaded in a SWFLoader. I believe this will be true for the majority of trivial sample applications that will be part of the test bed. A couple more pros for using RIATest: The scripts in RIATest are written in simplified ECMAScript with minor syntax extensions so it is a good match for Flex devs who already know ActionScript. RIATest can also be invoked from Hudson/Jenkins and generates JUnit compatible results allowing to integrate it with continuous builds. Thanks Tigran. P.S. Sorry for a long email, I can't stop when I start talking about RIATest. :-)
