On Jan 26, 2007, at 6:50 AM, Vladimir Ivanov wrote:
Hello everyone,
while we have more or less stable CC it is time to do work to
extend Harmony
testing :)
Yes, I know that we have some gap in stability on x86_64 platform
and on Red
Hat Linux so CC runs tests in the 'perTest' mode but anyway I want to
summarize our discussions about testing and continue extend testing
capability.
The current buildtest infrastructure includes only one case for
testing:
cyclic run of Harmony pre-integration tests. Also we have jira
issues with
iterative runs, eclipse "hello world" and JEDIT scenarios. And
community
members time to time runs applications and report their execution
status.
The idea was to collect all test-run scripts into buildtest
directory and
implement configurable CC.
Yep!
Created jira should be integrated to the buildtest infrastructure and
buildtest module should be extended by different application
scenario tests,
reliability tests etc. In general case we will have a long CC cycle
(up to 1
week J ) and results of this runs should be processed with other
procedure
(for example, results should be upload to harmonytest.org and than
jira
issues should be created for all failures).
Right - we should probably try to define what we mean by "short",
"medium" and "long" scenarios, and I suggest:
1) what we have now (build and unit test drlvm, classlib) as the
"short" cycle for fast failure notification
2) test classlib w/ drlvm and J9 + the iterative test cycle as the
"medium" cycle
3) everything you talk about above as the "long" cycle
It requires some 'standard' interface for all integrated scripts. I
like
classlib interface so how about:
- call of "ant setup;ant" will run all available scripts;
- call of "ant -Dmodule=hit setup;ant" will run current version of
CC –
Harmony integration tests;
- call of "ant -Dmodule=eut setup;ant" will run Eclipse unit test
etc.
Note, in this case each module should implement proper 'setup'
target and
has configuration for CC. The root-script will iterate over all
modules to
call their 'setup' and this setup should include whole test setup
(downloading software, adding modules cc-configuration to working
configuration etc).
Is it OK?
I dunno - this sounds like disjoint and separate CC runs, rather than
a CC run with multiple projects.
For example, I'd like to have a set of "modules", which would be
incomplete cc config files, that somehow get glommed into a bigger cc
file - maybe the config.xml would have some kind of include mechanism.
Suppose that we have :
trunk/cc/
config.xml
modules/
default_module.xml
hut_module.xml
eut_module.xml
iterative_module.xml
dacapo_module.xml
specjbb_module.xml
short_module.xml
medium_module.xml
long_modules.xml
so then I could do :
$ ant
to get what we have now - runs the default module - or
$ ant -Dmodules=hut,eut,dacapo
to run those...
Something like "medium_module.xml" would look like (the following is
'psuedo-code') :
<includeconfig name="default_module.xml"/>
<includeconfig name="dacapo_module.xml"/>
so that you can nest this as you want.
If nobody objects I'll start restructuring of buildtest module and
will try
to integrate one from extensions.
Please describe how you want to do it.
Thanks, Vladimir
PS I think the resulting structure should be easy to extend and may
looks
like this:
buildtest
|--config (default CC configuration to build classlib and DRLVM)
|--hit (CC configuration to run Harmony classlib&DRLVM tests)
|--eclipse
|-- eut (setup and CC configuration to run eclipse non-interactive
tests)
|-- eclipse3.1.1
|-- some scenario
|-- build.xml (common setup + call of module's 'setup')
Interesting. I think one issue is that it seems like heavy lifting
to add a new module - each module becomes a "peer". What do you
think of the other approach above?
Either way, we don't want hit, eclipse, etc as peers. If anything,
they should go in a modules/ directory...
geir
