To give an idea, here's a hypothetical test result file. Format is YAML
(http://en.wikipedia.org/wiki/YAML), which has the benefits of being formal
as XML, still human readable (and editable). There's a nice python lib to
handle this format, playing with YAML and python is amazingly easy. See
attachment. (there's one successful sample, one failed tests, one test to yet
run).
Using this sample, a test result page can be generated. The more there are
samples and tests, the more it's accurate, the more the device will have a
good note. (we'll need to find a formula, for instance to give more weight to
tests than to samples, since tests are supposed to heavy-test the lib at
limits, etc...).
Seb
Le Saturday 11 October 2008 20:45:05 Sebastien LELONG, vous avez écrit :
> > Just updated the testresult page.
>
> You may be talking about the validating page, where we vote, right ?
>
>
> About the testing matrix itself, the test result pages, and the jallib
> wrapper script, I see two main parts:
>
> 1. helping to deal with multiple files (board + tests)
> 2. generating wiki page for test results
>
> About 1., there won't be magic. I think using the "-i" compiler option is
> valuable. The wrapper may handle values taking from a environment variable
> to avoid too much command line, but that's all it could do.
>
> About 2., we need to define a formatted text file (let's avoid xml), which
> will store the test results. Each device will have its own formatted file.
> We still need to discuss the content, at a start, we need the following
> information:
>
> * test name + result (0 or 1)
> * sample name (yes, we could also mention this here, so we could have a
> complete overview for each device)
> * timestamp for each test/sample, to have an history
> * device file information (dev2jal version, or better svn revision)
>
> Starting from this file, in a first step, the jallib wrapper could generate
> the correct wiki page (eg. http://code.google.com/p/jallib/wiki/Test16F88).
> This mean we'd need to adjust manually the text file. If these manual
> operations are too heavy, or too error-prone, in a second step, the jallib
> wrapper could handle the text file update (less manual operation).
>
> Finally I can imagine a last case where we provide all the text files to
> the wrapper, which will generate THE testing matrix (eg.
> http://code.google.com/p/jallib/wiki/TestingMatrix, see the table).
>
> If this sounds absurd, if this is not valuable, please tell me. If this
> sounds like a nice idea, if this will save our time and help us, please
> tell me too ! In others words: I'm waiting for your feedback.
>
> Cheeeeers,
> Seb
>
> PS: I plan this for the final release (I hope), not for the beta.
--
Sébastien LELONG
http://www.sirloon.net
http://sirbot.org
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"jallib" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/jallib?hl=en
-~----------~----~----~----~------~----~------~--~---
device: {name: 16f877a, revision: 387}
samples:
- {date: 2008-10-11, file: print_serial_numbers.jal, result: OK, revision: 387}
tests:
- {boardfile: board_16f877a_dwarf.jal, date: 2008-10-11, result: KO, revision:
387,
testfile: test_i2c_hw_l0.jal}
- {boardfile: board_16f877a_dwarf.jal, date: 2008-10-8, result: null, revision:
387,
testfile: test_i2c_sw_l0.jal}