[
https://issues.apache.org/jira/browse/MUSE-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517020
]
Kam K. Yee commented on MUSE-244:
---------------------------------
======================================
PROPOSED IMPLEMENTATION - HIGH LEVEL
======================================
Following are some initial thoughts/proposal for the implementation of the
various tests and framework.
(This proposal incorporates and adds to the input from Andrew Eberbach's
previous postings and email exchanges (Thanks! Andrew).
- The unit, functional, and stress tests will be JUnit based.
- ANT scripts will be created to setup and invoke these JUnit test suites.
- Conformance tests:
- The WS-I Organization (http://ws-i.org) has developed and made available
the WS-I Interoperability Testing Tools to help developers determine whether
their Web Services
are conformant with WS-I profile guidelines. (More details on these tools and
how they work can be found at
http://ws-i.org/deliverables/workinggroup.aspx?wg=testingtools
and http://www-128.ibm.com/developerworks/webservices/library/ws-wsitest/).
The latest version is designed to test the Basic Profile 1.1 and SSBP 1.0. We
will extended
the testing tools to test Muse for conformance against the latest supported
specification levels (WSRF 1.2, WSN 1.3, WSDM 1.1, and WS-MetadataExchange).
- End To End Test Automation:
- For end to end test automation, STAF/STAX seems to be a good automation
framework to use and build upon. (More details on STAF/STAX can be found at
http://staf.sourceforge.net/index.php).
The Software Testing Automation Framework (STAF) is an open source,
multi-platform, multi-language framework designed around the idea of services
(such as process
invocation, resource management, logging, and monitoring). STAF removes the
tedium of building an automation infrastructure, thus enabling testers to focus
on building the
automation solution. The STAF framework provides the foundation upon which to
build higher level solutions, and provides a pluggable approach supported
across a large
variety of platforms and languages.
STAX is an execution engine which can help with thoroughly automating the
distribution, execution, and results analysis of testcases. STAX builds on top
of three
existing technologies, STAF, XML, and Python, to place great automation power
in the hands of testers. STAX also provides a GUI monitoring application which
allows one to
interact with and monitor the progress of jobs.
STAF/STAX will be used to automatically invoke and orchestrate the
invocation of the ANT scripts upon receiving a notification (such as an email
notifiying that a build
has completed).
UNIT TESTING:
- Each top-level muse component (such as muse-core, muse-tools, etc.) will
have a corresponding "-test" directory (muse-core-test, muse-tools-test, etc).
This
directory will have a structure mirroring the component it is testing (as is
common practice with JUnit). Each non-trivial class will have a corresponding
test suite and
the test suites would be self-contained within the project. This is so that
users don't have to check out several "-test" directories in order to run one
(dependencies
become very tedious, very quickly).
- A new target will be introduced to the build.xml to build a new archive
(muse-test-unit-bin) containing only the unit test code and needed artifacts
(e.g., WSDLs &
XSDs).
- The identified steps for unit testing are as follows:
1. Download or locally build muse-bin runtime archive
2. Download or locally build muse-test-unit-bin archive
3. Unzip muse-bin
4. Unzip muse-test-unit-bin
5. Run muse-bin code generation tool to generate an endpoint with all
capabilities for the target container
6. Deploy generated endpoint to the target container
7. Startup target container with generated endpoint
8. Invoke execution of JUnit unit test suites
- A new unit test ANT script will be created to invoke and execute the
different JUnit test suites.
Later on when full automation is introduced, the automation framework will be
invoking these ANT script target to automatically execute the test suites.
- This design also allows for developers to do a sanity check by running these
tests locally on their machines prior to checking code into the repository.
FUNCTIONAL TESTING:
- This would loosely follow Dan's current approach of top-level directories
that are related to specific capabilties (wsrf, wsn, etc). The structure here
isn't as easy to
generalize, but some directory/hierarchical organization that seems sane would
suffice. Again, the goal here is self-containment as to avoid an overly complex
test
environment.
- A new target will be introduced to the build.xml to build a new archive
(muse-test-fvt-bin) containing only the functional test code and needed
artifacts (e.g., WSDLs &
XSDs).
- The identified steps for functional testing are as follows:
1. Download or locally build muse-bin runtime archive
2. Download or locally build muse-test-fvt-bin archive
3. Unzip muse-bin
4. Unzip muse-test-fvt-bin
5. Run muse-bin code generation tool to generate endpoint(s) to be deployed
to target container
6. Deploy generated endpoint(s) to the target container
7. Startup target container with generated endpoint(s)
8. Invoke execution of JUnit functional test suites
- A new functional test ANT script will be created to invoke and execute the
different JUnit test suites.
Later on when full automation is introduced, the automation framework will be
invoking these ANT script target to automatically execute the test suites.
- This design also allows for developers to do a sanity check by running these
tests locally on their machines prior to checking code into the repository.
Finally, since modifying something simple is usually easier than generating
something simple from scratch, we will create two "read only" top-level
directories which would lay out a reasonable skeleton for users contributing
their own junit and functional tests for top-level directories.
> Test/compliance framework for Muse
> ----------------------------------
>
> Key: MUSE-244
> URL: https://issues.apache.org/jira/browse/MUSE-244
> Project: Muse
> Issue Type: New Feature
> Components: Other
> Reporter: Balan Subramanian
> Assignee: Dan Jemiolo
> Fix For: 2.3.0
>
>
> A test framework is required for Muse. This framework will:
> 1. help in testing - individually and in combination - the various WSDM
> capabilities implemented in Muse
> 2. help users test their own endpoints
> 3. help run stress tests
> 4. Manage deployments on various target platforms and their combinations
> (OSGi (Equinox, others), J2EE (WAS or Tomcat) each with either axis2 or mini)
> for different operating systems and JDK versions
> 5. Provide ability to automatically executing test suites aimed at 1)
> endpoint function 2) standard implementation testing 3) compliance testing
> using the SOAP messages
> There have been a number of conversations about such a framework on the
> mailing list led primarily by Andrew Eberbach. Those will be collected and
> consolidated under this feature requirement.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
