Hi Christoph,
Christoph Neumann wrote:
Hi,
Thorsten Ziehm schrieb:
Hi Mathias,
Do you think it's worth it?
I think it's not primarly the matter of running the regression-suite
before QA approval but to have a small set of meaningful regression
tests available ?
Exactly, and I would prefer to have regression tests based on the API or
complex test framework and not based on the GUI testtool. We shouldn't
raise even more barriers to contribution.
I'm really on your side! But how many complex tests do we have? How high
is the code coverage with API tests in complex scenarios. I do not think
that we have enough test scripts on coding level. If we have it, then we
should make them mandatory quickly. Perhaps then the tests on GUI level
with TestTool could be unnecessarily.
how high is the code coverage of TestTool? Both can be extended.
The code coverage of TestTool tests is more than 50% overall. In some
applications we are testing more than 70%. But this isn't the point
where I want to point my finger on. I see, that we have a stable API in
most cases. I do not see so many issues about broken APIs. But I see a
lot of issues about regressions. My question is here, do we find the
regressions with API tests only? Do we need more complex tests, which
test the API in complex scenarios? Will they find the regressions, which
hinder the user to work with the product? I do not know this, therefore
I ask this question.
But everybody shouldn't forget that API testing is another level of
testing. It is testing on code level. GUI testing is the highest level
and it tests against the specification and the functionality on user
level. It will find completely different problems as UNIT-, API- or
Complex-Tests.
But do not misunderstand me. I want to have as much testing and quality
assurance on a CWS as I can get. So I'm friend of UNIT-, API- and
Compex-Tests. And when they should be mandatory for approving a CWS, I
will vote for it.
I have planned to establish an automated UNO-API testing for CWS and
Master. The CWS tests should be started in EIS like ConvWatch, the
Master would be tested automatically as soon as they are available. This
should be done until end of summer (hopefully).
Super !
Thorsten
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
