I've been doing some further thinking around this over the last
couple of days and have come up with 3 possible structures (there are
undoubtedly more but that is as far as my brain got!):
1) The original idea below. ie.
\testsuite
\api
\internals
\smoke
This setup requires each container, and version there of, to be a
separate profile. Container specific tests can be excluded with
surefire.
This does get quite messy quickly if there are lots of containers,
and lots of container specific tests that need to be written (ie.
lots of exclusions in profiles for tests you don't want to run)
2) More along the lines of Seam Persistence. ie.
\testsuite
\base
\jboss
\weld-embedded
\jetty
Each version of a container will have their own profile within a
container module.
This nicely breaks down the containers, but makes it difficult to
know what pieces of a module (ie. API, internals, etc) are being
tested on each container.
3) A hybrid approach ie.
\testsuite
\base or common
\internals
\base
\jboss
\jetty
We could either go with a base/common module at the root of
testsuite, which would hold utilities for artifact creation and all
common tests or base tests for all modules, and/or a base/common for
each submodule (ie. api, internals). Could take either approach, but
a single base/common at the testsuite root may make it simpler.
Each container module would then have a profile for each version,
with the version of a container that you want to test as part of
builds set to be active by default.
I'm personally leaning towards 3 as it gives us a breakdown of types
of testing, api, internals, smoke, cluster, etc while also providing
a breakdown of container specific test requirements.
Note: All pure unit tests that do not require a container would still
remain in the appropriate module, ie. api or impl, of the project.
Any other options anyone can think of, or tweaks to any of the above
options?
What does everyone think?
Ken
On Wed, Jul 6, 2011 at 1:43 PM, Jason Porter <lightguard.jp
<http://lightguard.jp>@gmail.com <http://gmail.com>> wrote:
Adding the rest of the list as I think we're nearing the point we
want some more feedback.
On Wed, Jul 6, 2011 at 10:35, Ken Finnigan <[email protected]
<mailto:[email protected]>> wrote:
Nasty, read the irc log, not nice at all on AS7 part!
At the moment I have a structure similar to persistence, but
more following AS7. ie:
\module_parent
\api
\impl
\testsuite
\api
\benchmark
\clustering
\internals
\smoke
\stress
I like the ideas, but I think the structure should be along the
lines of the containers (perhaps in addition to, if we're going
to test things like clustering and perf).
Idea being that pure unit tests (ie. no container) remain
within either api/impl modules, then any container testing
falls within the testsuite somewhere. Default test only runs
api, internals and smoke, others are picked up by a profile.
Having the different containers be different test modules also
gets us away from having to rely on a profile (so you could run
each set of tests in one go instead of multiple invocations or
lots of profiles).
At present most of these folders are just ideas about the
kinds of tests we could do (along the AS7 lines), but they
seemed appropriate. Existing i18n module tests are now in
internals for testing the implementation. I see smoke as
being more comprehensive tests covering combined use of
features in the module, and quite possibly integration with
other modules too.
I do kind of like the idea of all the tests being inside the test
suite, but I'm also okay with basic unit tests living in api and
impl. I think we probably need to do some lifecycle tweaking and
have those tests under the testsuite run under the integration
phase like Solder.
Which container is run is then all down to profile. Not sure
whether to continue with embedded weld being the default
container that is run when no profile is specified, or to
force a profile to be specified for any container testing.
Also had another thought today as to whether the poms for the
test modules should define dependencies, or whether dependent
libraries should be added directly to deployment artifact as
library to prevent unintended libraries from appearing on
test classpath.
If we go with test modules instead of profiles this becomes a non
issue as they'd just be test deps for the module.
Thoughts?
On Wed, Jul 6, 2011 at 11:35 AM, Jason Porter <lightguard.jp
<http://lightguard.jp>@gmail.com <http://gmail.com>> wrote:
Very cool. I got stuff working for catch last night too,
but there is a problem if you're testing deployment
errors. It's actually an AS7 and Arquillian issue. I've
opened JIRA tickets about both. I think what we'll want
to do is set up testing similar to what Stuart has done
in Persistence with multiple modules. We'll eventually
want to test in (probably) embedded weld (it's runs
quickly, could probably be like a smoke test), JBoss AS7,
AS6, GF 3.1, Resin and an OWB container or embedded OWB.
Not all of those containers work right now but then we
really could say we know it all works on each CDI impl.
Sent from my iPhone
On Jul 6, 2011, at 7:44, Ken Finnigan <[email protected]
<mailto:[email protected]>> wrote:
I actually "stole" some day job time to try at work as I
thought the problem might be caused by the local build
of as7 I had in my repo, but that wasn't it.
Figured out the problem I was having with the aether
class not being found was caused by the arquillian junit
container dependency being specified in a parent pom,
and then the managed container being specified in the
pom running the tests, which meant Maven decided the
versions of the jars in the pom running the tests should
take precedence.
Adding the junit container dependency to the pom running
the tests fixed that problem.
Should be able to finish up the test setup some time
tonight to then forward to the mailing list for review.
Ken
On Tue, Jul 5, 2011 at 11:13 PM, Jason Porter
<lightguard.jp <http://lightguard.jp>@gmail.com
<http://gmail.com>> wrote:
Sure, you let me know a time. I'm working on getting
Catch up to date as well. BTW, Arquillian
1.0.0.Final will be out tomorrow, not sure about
container support though.
On Tue, Jul 5, 2011 at 20:22, Ken Finnigan
<[email protected] <mailto:[email protected]>> wrote:
Jason,
If you have some time tomorrow it would be
appreciated if you could take a look at the new
testsuite setup on i18n. Here's the branch:
git://github.com/kenfinnigan/international.git
<http://github.com/kenfinnigan/international.git>
I thought I was making progress but now I seem
to have managed to move backwards! For some
reason I keep getting a NoClassDef errors on
aether classes. It appears that the classes are
brought in correctly from the arquillian junit
container (from shrinkwrap beta 3), but then the
jboss as managed container brings in an older
version of shrinkwrap that doesn't include aether.
I'm sure it's a simple fix, as I've mirrored it
off confbuzz, servlet and rest, as well as
jbossas, that all run as7 arquillian tests, but
can't see the wood for the trees tonight. Will
be looking at it again tomorrow evening, but if
you can spot an easy fix that would be great!
Thanks
Ken
On Tue, Jul 5, 2011 at 1:39 PM, Jason Porter
<lightguard.jp <http://lightguard.jp>@gmail.com
<http://gmail.com>> wrote:
Okay, great. Let me or Dan know if you need
some help.
On Tue, Jul 5, 2011 at 11:37, Ken Finnigan
<[email protected]
<mailto:[email protected]>> wrote:
Pretty good.
Have the basic structure for the module
in terms of what needs to go where.
Currently going through and getting the
existing i18n tests working in AS7,
which is proving more of a challenge
than I expected!
Of all the tests that pass in
weld-ee-embedded, about a third actually
pass in AS7! Most of it, I think, is
down to differing classpaths, but should
be in a position to push the feature
branch to my fork of i18n in a couple of
days.
Once I've pushed it to my fork I was
going to send an email to everyone on
seam-dev to get their thoughts on the
layout/setup for testing.
btw, you don't need to worry about
cc'ing my sorstech email in the future
as I don't use that much anymore.
Ken
On Tue, Jul 5, 2011 at 1:20 PM, Jason
Porter <lightguard.jp
<http://lightguard.jp>@gmail.com
<http://gmail.com>> wrote:
How goes things with i18n and the
new Arquillian?
--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next
Generation Java Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net
<http://keyserver.net>, pgp.mit.edu
<http://pgp.mit.edu>
--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java
Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net
<http://keyserver.net>, pgp.mit.edu
<http://pgp.mit.edu>
--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java
Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net
<http://keyserver.net>, pgp.mit.edu <http://pgp.mit.edu>
--
Jason Porter
http://lightguard-jp.blogspot.com
http://twitter.com/lightguardjp
Software Engineer
Open Source Advocate
Author of Seam Catch - Next Generation Java Exception Handling
PGP key id: 926CCFF5
PGP key available at: keyserver.net <http://keyserver.net>,
pgp.mit.edu <http://pgp.mit.edu>
