On Tue, 2009-05-19 at 15:35 -0700, David Brownell wrote:
> On Monday 18 May 2009, Zach Welch wrote:
> > The tables track reports from users detailing the results from running
> > the regression test suite. A "report" will specify the "platform",
> > "interface", and "target" or "board" that was tested by the "user",
> > along with the 'version' of OpenOCD, the test 'results', along with a
> > 'comment'. In this description, words in double-quotes ('"')
> > effectively describe various primary tables, while the words in
> > single-quotes ("'") describe column fields.
>
> Let me suggest a summary page with a matrix that mostly
> overlooks the specific boards involved, and focusses on
> more straightforward issues:
>
> CPU/Platform JTAG Adapter/Interface
> ft2232 parport jlink ...
> arm920t g y r ...
> arm926ejs g g y ...
> xscale g - - ...
> mips - - g
> avr8 y g -
> ....
>
> where a "g/green" indicates multiple positives, no negatives;
> a "y/yellow" indicates mixed results or only one positive;
> a "r/red" indicates more negatives than positives; and a
> blank indicates no results. (Or some other metric which
> is useful and morphs easily to such a status panel display.)
>
> That could be emailed to the list ... or turned into a web
> or wiki page. In HTML form, maybe the body of the cell would
> be a link pointing to more details ... semi-fancy versions of
> this could highlight results for the *last* release, so it'd
> be easy to see notice regressions.
>
> If such a page shows a lot of red, that would strongly suggest
> it's not ship-ready. Too many blanks would show "not enough
> testing yet". Rows or columns would show patterns indicating
> particular code needs work.
>
> Key point: without such a summary, it'll be hard to know
> where the problems (including big unknowns!) lurk.
I agree this will be the pinnacle of the display output produced from
the data; your description is pretty much spot on what I imagined when I
started coding. My scripts are not quite there, but they are a start.
Someone can probably use what I have done and do exactly what we want.
Since these results will only be useful once we have them, I have been
trying to focus more on the collection and storage side, such that we
can make it easy for users to produce the required smoke test reports.
If it's not fairly easy, then we may not collect as many results as we
need to make everything turn green.
Cheers,
Zach
_______________________________________________
Openocd-development mailing list
Openocd-development@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/openocd-development
