On 16.08.2013 21:37, Regina Henschel wrote:
Rob Weir schrieb:
Moving this topic to its own thread.
It should be possible to code a very thorough set of test cases in a
spreadsheet, without using macros or anything fancy. Just careful
reading of the ODF 1.2 specification and simple spreadsheet logic.
Following the spec is not enough. For example, if the accuracy decreases
from 14 digits to 7 digits, that is not covered by the spec.
<skip test case example description>
If we used an approach like this on the other spreadsheet functions,
we could have a semi-automated test suite that would practically
guarantee that Calc is free of calculations errors. Once we're
written the test cases, a modest upfront investment
"modest"? One function a day and you need more than a year.
, it will benefit
us with every release we do. Heck, it would benefit LibreOffice,
Gnumeric, Calligra as well, maybe even Microsoft and Google, though
they might already have such test cases defined internally.
I see a problem in how such a test suite is made available. And how the
results for a special release are collected.
The problem with the current test cases is, that I do not know where
they are, how they are to use and how to generate new ones. It is a
closed book, only for insiders.
An example of a test case where a formula ("addition") is checked is in
[1]. This file is clean and easy enough that it could be used as a
template for more formula checks.
[1]
http://svn.apache.org/repos/asf/openoffice/trunk/test/testuno/source/fvt/uno/sc/formula/AddtionOperatorInFormula.java
The general topic on getting started with test automation is covered in [2].
[2] http://wiki.openoffice.org/wiki/QA/test_automation_guide
Anyone interesting in helping with this kind of test case development?
There exist some files already in Bugzilla. I used to make test
documents, when working on functions. I think, that they can be extended
to work in a way, that a simple look on it will tell errors. But I have
no ready collection on my PC and most will be already deleted from my PC
in the meantime.
One problem is, that comparisons with constants have to be written in a
way, that they are independent from local. Eike has once corrected one
of my test spreadsheets that way.
Any ideas on how to fully automate this? ODF 1.2 is very strict, so
we're not starting from a perfect score. But we should find an easy
way to report on regressions.
If you will automate this, you will need to develop a frame. But
automation is not the total solution. Testing can be a way to bring user
into the community. And tests have to cover different languages and
scripts. I remember errors reported to LibreOffice, where a time
calculation was wrong only in special locals. To extend a testing frame
to consider this would be very expensive.
There is no need to develop a new framework. Please check Zhe Liu's
wonderful work on test automation that I referenced above [2] that is
already available in our "test/" directory. It is very powerful, clean
and relatively easy to use.
Herbert
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org
For additional commands, e-mail: dev-h...@openoffice.apache.org
