Re: Integration testing - we need some :-)
By making the test classloader a child of the plugin's we are able to
run the tests - at least the scenario I had below. That's not ideal
as it makes the plugin's classes visible to the tests but at least we
can make progress on writing cases.
So, with that thought in mind, where should we start?
--
Jeremy
On Oct 18, 2006, at 10:28 PM, Jeremy Boynes wrote:
I manage to get most of this working but I am having problems with
Surefire's classloader isolation. The StartMojo deploys the
application and binds the CompositeContext to the thread. However,
the SCA class is loaded from two different classloaders so the
value returned from getContext() in the test code is null.
Suggestions on how to push this over without massive use of
reflection would be welcome. All the latest code is in the sandbox.
https://svn.apache.org/repos/asf/incubator/tuscany/sandbox/testing
Cheers
--
Jeremy
On Oct 18, 2006, at 9:57 AM, Jeremy Boynes wrote:
On Oct 18, 2006, at 8:53 AM, Raymond Feng wrote:
+1 from me.
I can help as well. I already have a simple mortage approval
application in my sandbox and I'll convert it to whatever
integration test environment we come out when it's ready.
What I'm messing with is a maven plugin that can be attached to
the integration-test phase of the build. I added Mojo that starts/
stops the server and managed to bind the surefire plugin to the
integration-test phase to get it to run integration tests - this
would allow us to write the test clients using JUnit or TestNG.
However, I'm wrestling with the issue about how to make sure the
test application and the client run in compatible classloaders - I
am afraid at the moment that they will end up using different ones
which would make life awkward. I've had a look at Surefire and I
think it should be fairly easy to mimic the surefire plugin's
behaviour but tweak it so that it uses the application classloader.
I hope to get back into it this PM and should have something today.
That would provide a harness to run them but we will need to write
the tests themselves. As you probably all know I don't think that
using samples is the right way to go about testing - samples are
about illustrating things, testing is about trying to break stuff
and that involves lots of edge cases and other things that you
don't want to encourage users to do.
The rough idea I have is that we write the test cases as simple
SCA apps that run in a composite embedded in the maven process.
These can connect to the composites under test using SCA wiring -
either in-process for simple tests, or using bindings to connect
to ones running in a different (server) environment. Something like:
class HelloITest extends TestCase {
private HelloService service;
public testHello() {
assertEquals(Hello, service.getGreeting());
}
protected void setUp() {
super.setUp();
service = CurrentCompositeContext.locateService(HelloService);
}
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
On Oct 18, 2006, at 11:12 PM, Jeremy Boynes wrote:
By making the test classloader a child of the plugin's we are able
to run the tests - at least the scenario I had below. That's not
ideal as it makes the plugin's classes visible to the tests but at
least we can make progress on writing cases.
I think that is acceptable.
So, with that thought in mind, where should we start?
One thing I'd like to do is be able to verify bootstrap on various
hosts, basically that system scdl contributions define a complete/
valid runtime for a particular host. Another area I want to start
testing is specific invocation scenarios such as callback sequences.
One other area is interactions between various subsystems, such as
between bindings and component impl types.
Do you want to move the stuff out of sandbox so we can start?
Jim
--
Jeremy
On Oct 18, 2006, at 10:28 PM, Jeremy Boynes wrote:
I manage to get most of this working but I am having problems with
Surefire's classloader isolation. The StartMojo deploys the
application and binds the CompositeContext to the thread. However,
the SCA class is loaded from two different classloaders so the
value returned from getContext() in the test code is null.
Suggestions on how to push this over without massive use of
reflection would be welcome. All the latest code is in the sandbox.
https://svn.apache.org/repos/asf/incubator/tuscany/sandbox/testing
Cheers
--
Jeremy
On Oct 18, 2006, at 9:57 AM, Jeremy Boynes wrote:
On Oct 18, 2006, at 8:53 AM, Raymond Feng wrote:
+1 from me.
I can help as well. I already have a simple mortage approval
application in my sandbox and I'll convert it to whatever
integration test environment we come out when it's ready.
What I'm messing with is a maven plugin that can be attached to
the integration-test phase of the build. I added Mojo that starts/
stops the server and managed to bind the surefire plugin to the
integration-test phase to get it to run integration tests - this
would allow us to write the test clients using JUnit or TestNG.
However, I'm wrestling with the issue about how to make sure the
test application and the client run in compatible classloaders -
I am afraid at the moment that they will end up using different
ones which would make life awkward. I've had a look at Surefire
and I think it should be fairly easy to mimic the surefire
plugin's behaviour but tweak it so that it uses the application
classloader.
I hope to get back into it this PM and should have something today.
That would provide a harness to run them but we will need to
write the tests themselves. As you probably all know I don't
think that using samples is the right way to go about testing -
samples are about illustrating things, testing is about trying to
break stuff and that involves lots of edge cases and other things
that you don't want to encourage users to do.
The rough idea I have is that we write the test cases as simple
SCA apps that run in a composite embedded in the maven process.
These can connect to the composites under test using SCA wiring -
either in-process for simple tests, or using bindings to connect
to ones running in a different (server) environment. Something like:
class HelloITest extends TestCase {
private HelloService service;
public testHello() {
assertEquals(Hello, service.getGreeting());
}
protected void setUp() {
super.setUp();
service = CurrentCompositeContext.locateService
(HelloService);
}
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
I actually set up a Continuum server to run Tuscany builds here in my office. I just wanted to check it out. It is quite easy to set up. Unfortunately, I do not have a machine I can donate to do this full time or that would be accessible externally. If you'd like I could write up some quick instructions on how to get Continuum set up to run Tuscany builds if that would be helpful. The next step, I would imagine, would be to create a maven profile that, when enabled, would run some integration tests as part of the build. -Bert On 10/17/06, Jeremy Boynes [EMAIL PROTECTED] wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
That would be great - I have a remote machine we can use. I'm working on some itest mojo that starts/stop Tuscany inside Maven. I was thinking that could be used for in-VM testing or for running test clients that could use SCA to connect to a remote test server. -- Jeremy On Oct 18, 2006, at 5:16 AM, Bert Lamb wrote: I actually set up a Continuum server to run Tuscany builds here in my office. I just wanted to check it out. It is quite easy to set up. Unfortunately, I do not have a machine I can donate to do this full time or that would be accessible externally. If you'd like I could write up some quick instructions on how to get Continuum set up to run Tuscany builds if that would be helpful. The next step, I would imagine, would be to create a maven profile that, when enabled, would run some integration tests as part of the build. -Bert On 10/17/06, Jeremy Boynes [EMAIL PROTECTED] wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
On 10/17/06, Jeremy Boynes [EMAIL PROTECTED] wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. +1, and I'd help. Probably we'll all end up needing to be some help wont we to get tests done for things like each extension? ...ant
Re: Integration testing - we need some :-)
+1 from me. I can help as well. I already have a simple mortage approval application in my sandbox and I'll convert it to whatever integration test environment we come out when it's ready. Thanks, Raymond - Original Message - From: Jeremy Boynes [EMAIL PROTECTED] To: tuscany-dev@ws.apache.org Sent: Wednesday, October 18, 2006 6:16 AM Subject: Re: Integration testing - we need some :-) That would be great - I have a remote machine we can use. I'm working on some itest mojo that starts/stop Tuscany inside Maven. I was thinking that could be used for in-VM testing or for running test clients that could use SCA to connect to a remote test server. -- Jeremy On Oct 18, 2006, at 5:16 AM, Bert Lamb wrote: I actually set up a Continuum server to run Tuscany builds here in my office. I just wanted to check it out. It is quite easy to set up. Unfortunately, I do not have a machine I can donate to do this full time or that would be accessible externally. If you'd like I could write up some quick instructions on how to get Continuum set up to run Tuscany builds if that would be helpful. The next step, I would imagine, would be to create a maven profile that, when enabled, would run some integration tests as part of the build. -Bert On 10/17/06, Jeremy Boynes [EMAIL PROTECTED] wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
I can take a look at doing some of this, perhaps starting with getting a simple sca application automated and going from there. Any direction from the list on specific areas that need integration test coverage would be helpful. Brent On 10/17/06, Jeremy Boynes [EMAIL PROTECTED] wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
I did a quick write up on how I set my server up here: http://wiki.apache.org/ws/Tuscany/TuscanyJava/ContinuumForTuscany Hope this helps! -Bert On 10/18/06, Jeremy Boynes [EMAIL PROTECTED] wrote: That would be great - I have a remote machine we can use. I'm working on some itest mojo that starts/stop Tuscany inside Maven. I was thinking that could be used for in-VM testing or for running test clients that could use SCA to connect to a remote test server. -- Jeremy On Oct 18, 2006, at 5:16 AM, Bert Lamb wrote: I actually set up a Continuum server to run Tuscany builds here in my office. I just wanted to check it out. It is quite easy to set up. Unfortunately, I do not have a machine I can donate to do this full time or that would be accessible externally. If you'd like I could write up some quick instructions on how to get Continuum set up to run Tuscany builds if that would be helpful. The next step, I would imagine, would be to create a maven profile that, when enabled, would run some integration tests as part of the build. -Bert On 10/17/06, Jeremy Boynes [EMAIL PROTECTED] wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
On Oct 18, 2006, at 9:20 AM, Bert Lamb wrote: I did a quick write up on how I set my server up here: http://wiki.apache.org/ws/Tuscany/TuscanyJava/ContinuumForTuscany Hope this helps! Thanks - I'll take a look and see if I can get it running on my server later today. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
On Oct 18, 2006, at 8:53 AM, Raymond Feng wrote:
+1 from me.
I can help as well. I already have a simple mortage approval
application in my sandbox and I'll convert it to whatever
integration test environment we come out when it's ready.
What I'm messing with is a maven plugin that can be attached to the
integration-test phase of the build. I added Mojo that starts/stops
the server and managed to bind the surefire plugin to the integration-
test phase to get it to run integration tests - this would allow us
to write the test clients using JUnit or TestNG. However, I'm
wrestling with the issue about how to make sure the test application
and the client run in compatible classloaders - I am afraid at the
moment that they will end up using different ones which would make
life awkward. I've had a look at Surefire and I think it should be
fairly easy to mimic the surefire plugin's behaviour but tweak it so
that it uses the application classloader.
I hope to get back into it this PM and should have something today.
That would provide a harness to run them but we will need to write
the tests themselves. As you probably all know I don't think that
using samples is the right way to go about testing - samples are
about illustrating things, testing is about trying to break stuff and
that involves lots of edge cases and other things that you don't want
to encourage users to do.
The rough idea I have is that we write the test cases as simple SCA
apps that run in a composite embedded in the maven process. These can
connect to the composites under test using SCA wiring - either in-
process for simple tests, or using bindings to connect to ones
running in a different (server) environment. Something like:
class HelloITest extends TestCase {
private HelloService service;
public testHello() {
assertEquals(Hello, service.getGreeting());
}
protected void setUp() {
super.setUp();
service = CurrentCompositeContext.locateService(HelloService);
}
}
--
Jeremy
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
I manage to get most of this working but I am having problems with
Surefire's classloader isolation. The StartMojo deploys the
application and binds the CompositeContext to the thread. However,
the SCA class is loaded from two different classloaders so the value
returned from getContext() in the test code is null.
Suggestions on how to push this over without massive use of
reflection would be welcome. All the latest code is in the sandbox.
https://svn.apache.org/repos/asf/incubator/tuscany/sandbox/testing
Cheers
--
Jeremy
On Oct 18, 2006, at 9:57 AM, Jeremy Boynes wrote:
On Oct 18, 2006, at 8:53 AM, Raymond Feng wrote:
+1 from me.
I can help as well. I already have a simple mortage approval
application in my sandbox and I'll convert it to whatever
integration test environment we come out when it's ready.
What I'm messing with is a maven plugin that can be attached to the
integration-test phase of the build. I added Mojo that starts/stops
the server and managed to bind the surefire plugin to the
integration-test phase to get it to run integration tests - this
would allow us to write the test clients using JUnit or TestNG.
However, I'm wrestling with the issue about how to make sure the
test application and the client run in compatible classloaders - I
am afraid at the moment that they will end up using different ones
which would make life awkward. I've had a look at Surefire and I
think it should be fairly easy to mimic the surefire plugin's
behaviour but tweak it so that it uses the application classloader.
I hope to get back into it this PM and should have something today.
That would provide a harness to run them but we will need to write
the tests themselves. As you probably all know I don't think that
using samples is the right way to go about testing - samples are
about illustrating things, testing is about trying to break stuff
and that involves lots of edge cases and other things that you
don't want to encourage users to do.
The rough idea I have is that we write the test cases as simple SCA
apps that run in a composite embedded in the maven process. These
can connect to the composites under test using SCA wiring - either
in-process for simple tests, or using bindings to connect to ones
running in a different (server) environment. Something like:
class HelloITest extends TestCase {
private HelloService service;
public testHello() {
assertEquals(Hello, service.getGreeting());
}
protected void setUp() {
super.setUp();
service = CurrentCompositeContext.locateService(HelloService);
}
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Integration testing - we need some :-)
In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Integration testing - we need some :-)
On Oct 17, 2006, at 12:53 PM, Jeremy Boynes wrote: In cutting M2 the lack of integration testing has been quite painful. We need to start adding the post-build functional/ integration testing we've talked about before but which we've not actually implemented. I'm going to start work on some projects in the sandbox to try and automate integration testing in a way that lends itself to execution using something like Continuum. I see the goals as being: * simple to maintain * simple to run * quick to run * able to be broken down into sub-suites that can run in different environments I welcome anyone interested joining in. I'm willing to help - let me know what needs to be done. -- Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
