On 16 August 2013 21:37, Regina Henschel <rb.hensc...@t-online.de> wrote:
> Hi Rob, > > Rob Weir schrieb: > >> Moving this topic to its own thread. >> >> It should be possible to code a very thorough set of test cases in a >> spreadsheet, without using macros or anything fancy. Just careful >> reading of the ODF 1.2 specification and simple spreadsheet logic. >> >> > Following the spec is not enough. For example, if the accuracy decreases > from 14 digits to 7 digits, that is not covered by the spec. > > <skip test case example description> > > If we used an approach like this on the other spreadsheet functions, >> we could have a semi-automated test suite that would practically >> guarantee that Calc is free of calculations errors. Once we're >> written the test cases, a modest upfront investment >> > > "modest"? One function a day and you need more than a year. > I think that a bit over the top, you can do quite a lot with an editor :-) And dont forget, if it really takes that long, how long does it then take to test it manually, no less I assume. so its a win-win situation, first person that tests functions write the first macros and so on. > > , it will benefit > >> us with every release we do. Heck, it would benefit LibreOffice, >> Gnumeric, Calligra as well, maybe even Microsoft and Google, though >> they might already have such test cases defined internally. >> > > I see a problem in how such a test suite is made available. And how the > results for a special release are collected. > > The problem with the current test cases is, that I do not know where they > are, how they are to use and how to generate new ones. It is a closed book, > only for insiders. > Thats a matter of documentation, I did not know where build.pl was stored or how the build system worked before I invested time. Everything is a closed book and for insider until you know it. > > >> Anyone interesting in helping with this kind of test case development? >> > > There exist some files already in Bugzilla. I used to make test documents, > when working on functions. I think, that they can be extended to work in a > way, that a simple look on it will tell errors. But I have no ready > collection on my PC and most will be already deleted from my PC in the > meantime. > ODF must also as such have test suites to test the specification. > > One problem is, that comparisons with constants have to be written in a > way, that they are independent from local. Eike has once corrected one of > my test spreadsheets that way. > We can simply assume for a start (that would be huge) that automated testing runs in the en-US environment, then if there are problems it is very likely in the translation. > > >> Any ideas on how to fully automate this? ODF 1.2 is very strict, so >> we're not starting from a perfect score. But we should find an easy >> way to report on regressions. >> > > If you will automate this, you will need to develop a frame. But > automation is not the total solution. Testing can be a way to bring user > into the community. And tests have to cover different languages and > scripts. I remember errors reported to LibreOffice, where a time > calculation was wrong only in special locals. To extend a testing frame to > consider this would be very expensive. > > I simply disagree, I come from a company where every bug was turned into at least one test case in the framework, that was not all expensive, merely giving the developer a slightly different job. Using this philosophy we guarantee that a bug never reoccurs and that we test more and more with every regression. Testing AOO is actually quite simple than testing live systems, where the history changes the behavior of the system. Apart from very few test cases, we can simply restart every time. We do need a test framework, but that is much more documentation and then use what we have. In the first line macros, later perhaps scripting through the extension framework. But alone with macros we have come a long way. > Let me not be misunderstood, I like the idea of collecting test cases. > I could have misunderstood what you wrote, but I know we all have the same goal, a high quality. rgds jan I > Kind regard > Regina > > > > > ------------------------------**------------------------------**--------- > To unsubscribe, e-mail: > dev-unsubscribe@openoffice.**apache.org<dev-unsubscr...@openoffice.apache.org> > For additional commands, e-mail: dev-h...@openoffice.apache.org > >
