Thorsten Ziehm wrote:
Hi Mathias,
Mathias Bauer wrote:
[...]
You are right, there were not found all regressions in Master by the
automated tests. But some of them were found, when some more tests are
mandatory. In the past only 2 smaller tests are mandatory for approving
a CWS. Many testers run more than these tests, but not all. Therefore
some regressions went into the Master, which could be identified by
the test cases.
I didn't talk about tests on a CWS (I know that we only have a few
mandatory tests), I was talking about the regressions that haven't been
detected by the "release testing" on the master.
We will never find all regressions with TestTool or any other tooling
or human testing. This has nothing to do with 'release testing' or so.
We do not have test cases which identify problems in displaying
documents or something similar. Perhaps intensive usage of ConvWatch
could help a little bit. But to make this mandatory is another story!
When we talked about the last big regressions with CWS aw024 or the
CWS with AutomaticStyles in Writer. I do not have any solution until
now, how such regressions could be minimized in the near future. Our
test mechanisms do not find such regressions. Only a mass of human
testers could help here. So we are trying to bring more QA members of
the community to the relevant master builds for intensive testing of the
new feature.
Hi Thorsten,
it's not about detecting EVERY regression - it's about finding at least
a few regressions.
What you wrote is an argument against automated regression tests on CWSs
. If we are not able to detect regression on whatever workspace
(MWS/CWS), we don't even need to think about it.
Anyway - if there are such meaningful regression tests that could have
prevented a regression in the past - O.K. that's fine and it will help
to do such test on a CWS.
But don't make everything mandatory. If I change a string in the setup
or change platform dependend code for systemintegration I don't want to
do a mandatory test that tests whether all dialogs in the Calc still work.
- Hennes
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
