Hi Hiranya et al, Thanks a lot for all the guidance. Sorry for the delay, I've been working on logging and reporting aspects. I've submitted the complete code as a patch to the JIRA.
As per the discussion, I added a separate module called "integration". A total of 64 samples are automated in this patch. Just executing "mvn site" inside the integration module should work now :-). Executing mvn site without any parameters will run all the samples, create the logs and the html report. I've configured logging so that the complete log from a test session will goto SYNAPSE_HOME/sampleAutomationLogs/. A separate log named log-<timestamp>.log will be created per test session. I've created suites according to the categories in samples, to run a suite, -Dsuite=<suitename> (e.g. -Dsuite=message) can given. Also, any selected number of samples can be run by specifying -Dtests=<sample_numbers> (e.g. -Dtests=1,2,51,152,430). The samples can be also run by importing the project into an IDE (such as IDEA). I wrote short articles on usage and extending the framework, will update soon with the links. Detailed status of the framework can be found at [1]. There are some concerns / fails regarding some samples. Will discuss and try to fix them. As Hiranya mentioned, there are many things to improve. Although GSoC is finished, I hope to keep contributing to the framework and Synapse. [1] - https://docs.google.com/spreadsheet/ccc?key=0ArPKF5Gw4b0qdExSS1ZNb0RIeVJFNHZfZjYxUWFnOWc&hl=en_US On Thu, Aug 18, 2011 at 10:10 AM, Hiranya Jayathilaka <[email protected]>wrote: > Hi Folks, > > I just went through a demo of the new sample automation framework developed > by Amila for his GSoC project. Have to admit that it's a brilliant piece of > work. There is lot of room for improvement but the idea and the > implementation is pretty good for a first cut version. He has automated a > lot of samples and adding new samples to the framework is easy. I've > instructed him to carry out the following action items in the next few days > before we start off the GSoC final evaluations: > > 1. Submit all the code as patches to the JIRA (He has already submitted > some patches during the mid term evaluations) > 2. Write up some wiki pages explaining how to use and extend the framework > > Soon I'll start checking in this work to the Synapse trunk. I think the > best approach is to introduce a top level module named "integration" and add > all the automated samples (implemented as test cases) under that. That way > our samples become a suite of integration tests for Synapse and they will be > included in the nightly builds as well. Appreciate your thoughts on this. > > Here are some points that we can improve in the test framework: > > 1. Make it into a separate binary distro so we can download it and run > against a given Synapse instance > 2. Improved logging and reporting > 3. Code coverage > > Thanks > -- > Hiranya Jayathilaka > Associate Technical Lead; > WSO2 Inc.; http://wso2.org > E-mail: [email protected]; Mobile: +94 77 633 3491 > Blog: http://techfeast-hiranya.blogspot.com > -- Regards, Amila Manoj blog: http://amilamanoj.blogspot.com/
