Thanks for the thorough explanation.
Pardon my density, but I'm missing something fundamental - how will
NMaven work in any of the cases below at runtime, without a settings
file and capability matching, given that you are saying the IT test
might need some special handling to specify the implementation?
- Brett
On 17/12/2007, at 9:58 AM, Shane Isbell wrote:
Hi Brett,
The trunk integration tests are set up the same way as Maven and my
intention for the first release was just to test out the latest
version of
Mono using .NET 2.0, as well as Microsoft 2.0. This would involve just
changing the environment on a limited number of configurations. Just
to
note, a simple changing out of the path will not completely work on
all
configurations, as Mono contains csc.exe (for .NET 1.1) and gmcs.exe
(for
.NET 2.0). Microsoft, on the other hand, has the versions in
different
directories, which makes swapping out the paths easier. But you have
to keep
in mind that for Microsoft .NET 3.0, the framework uses the same
compiler as
.NET 2.0, but a different one for 1.1 and 3.5. Configurations can
get a
little funky.
If we were only dealing with Mono or only Microsoft, I would be much
more
confident that would could pull off doing the IT setup exactly the
same as
Maven for all the needed configurations. I'm hoping that this is the
case.
However, it's a little premature for me to have a good idea of how the
integration tests will work under multiple platforms now that we
don't have
capability matching. Right now we have options of creating scripts
setting
up the paths, with modifications for each permutation of
installations.
Another approach is something like the nmaven-settings file, which
contains
the parameters and lets some component handle the paths. One thing
that I do
like about a settings approach is that, on Windows, we can
autogenerate the
settings file by inspecting the registry, meaning we only test what
the
platform is capable of testing. We'll need to open up that to design
and
comments when we get to that point.
If anyone on the list wants to take up on how IT testing should be
done, go
for it. The issue: http://jira.codehaus.org/browse/NMAVEN-14 has
been out
there a long time.
As part of the IT testing, we will need to create a .NET assembly
(or find a
way to do it through Java) that handles the inspection of meta-data
within
the project assembly. This is the only way to verify things like
resource
generation, signing of assemblies, proper dependencies/references,
and so
on. We may even be able to write this under the verifier.
We haven't yet addressed within the compiler interface implementation
the specifying of framework version for Mono, so I have even less of
an idea
of exactly how it will work with the IT tests.
Shane
On Dec 16, 2007 1:32 PM, Brett Porter <[EMAIL PROTECTED]> wrote:
On 17/12/2007, at 8:19 AM, Shane Isbell wrote:
I agree on all the points. Can you post a bug about what is
breaking?
Will do.
This was the original motivation for the nmaven-settings file. It
allowed
changing the platform configuration to replace vendors, vendor
versions and
framework versions. I think that the general nmaven-settings file
concept is
the right approach for integration testing, it should just be used
for
integration tests and should be non-obstrusive. This will likely
require
adding some component extensions that will allow modifying of the
working
directory of executables. This approach would avoid having to bring
in all
the capability matching components to support it.
At this stage I would be happy with the integration tests just
running
under whatever the current execution environment is, like the normal
NMaven execution would do. This is basically what the Maven ones do
for now, based on the version of Maven in the path. You then switched
your execution environment and re-run the test suite. Some tests are
excluded on environments they are not suitable for. The integration
test tools that are used for Maven should be able to be re-used in
NMaven.
Beyond that, I would just use whatever the toolchain capabilities are
in Maven at the time to go towards the next step rather than adding
anything specific for it in either NMaven or the integration tests.
Is that in line with what you were thinking?
- Brett
