On 28/07/10 23:46, Steve Hall wrote:
From: Dominique_Pellé, Wed, July 28, 2010 5:18 pm
1/ Human tests are certainly complementary with automated tests.
Having said that, I'm mostly in favor of adding more automated
tests (make test) or improving coverage of existing ones. I've
never looked at how to write those tests myself, but I can give
it a try.
Are we discussing strictly build tests or also usage testing? I see a
big difference between the two but the importance of both.
As for the latter, Cream uses an extremely large number of Vim
features just to start and in doing so tests a wide range of code and
conditions. (I think it has uncovered a Vim bug only a couple of times
in the last 10 years. So far, no problems with 7.3c BETA.)
Couldn't a script/test suite be created that methodically tests all
the features, with new tests added for each new feature? I would think
a collection of statements could be added to a log file. Here's one
method of data collection we use:
let s:debug = "\n INITIALIZATION:\n\n"
function! Foo_function(dir)
" collect the passed {dir} variable as test data...
let s:debug = s:debug . "Foo_function(): {dir} == \"" . a:dir .
"\"\n"
[... rest of script ...]
function! Collect_debug_info_local()
return s:debug
endfunction
This could easily be extended to test any Vim feature.
4/ Some bugs can also be found if we test Vim on different OS.
How about keeping track of whether Vim successfully compiles and
passes all tests on:
[...]
We can ask users to submit short summaries like this and collect
them somewhere:
Vim version: Vim-7.3c (2427:33148c37f3c9)
OS: Ubuntu-10.04, x86
configure: --with-features=huge --enable-gui=gnome2
compiler: gcc-4.4.3
compilation: OK, no warnings.
make test: ALL DONE
Tested by: Dominique Pelle
The file data collection and formatting should be automated so the
user just has to do something like:
:vimtests
or similar and email the resulting data file or output. I do this with
Cream and it because the user only has to run one command, I can
collect an exhaustive amount of data that would otherwise take hours
to run. Our script labels, formats, and even inserts folds in the log
for easy navigation.
There is already
:runtime bugreport.vim
in the plain-vanilla distribution, which collects a huge lot of
information — in fact, usually too much for my health. ;-)
Oh, and BTW, how come your mailer always breaks threading when replying?
Best regards,
Tony.
--
There once was a fiesty young terrier
Who liked to bite girls on the derriere.
He'd yip and he'd yap,
Then leap up and snap;
And the fairer the derriere the merrier.
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
