Phil - interesting your comment on path-level coverage being tricky.
I really like the ability to be able to see in app.c how the components are
linked together. I would think that as nesC is more widely used, a tool
would evolve that could show the call graph.
By path-level coverage I really meant "path-level coverage in each
component" - sorry for not being more precise with the context.
So I'm thinking of were individual components can be wired together, and
stimulated in a test harness to test a new component expected operation.
Of course in an ideal world, when I find an interesting component, it would
be so nice to know that it is tested and some has put their quality
assurance thumb print that this component is tested, where when and how.
(...please let me dream.... ).
When I'm generating large amounts of code from a specification, I typically
track the stages each procedure/function in a module "or component" is
going to go through. For me these are
Unit Design
Coding
Unit Test
Integration Test
The integration test is of course the most visible to others and what my
reputation as a software engineer stands on.
Maybe I'm slower than some on the production of code (or perhaps more
realistic- if there is such a thing in software) - but I want working
functionality. I've spent so many late nights getting my code working to
were I'm confident in it before releasing it. And I've spent a much larger
number of late nights working with other peoples poorly tested code and
finding out what doesn't work - but hey that's team work.
I agree with you on the benefits of probabilistic testing. Absolutely
essential for a large group of software entities to have their interactions
tested exhaustively and their loading characterized under varying conditions
including defining what overload is.
My difference would be to say, I would like a high level of confidence that
the components are working as expected before launching an exhaustive
probabilistic test.
On telephone systems, we would run 200,000 telephone calls through the
system to characterize the systems for its 99.999% reliability requirement.
For the motes - I wonder what are the parameters - 500,000 messages with
close motes and motes at a distance and simulated weather conditions to
cause fade and lightening!!! :)
Which brings me back to the visual path level testing of modules.
I really like what I saw with the usr\local\bin\avr-insight and avr-gdb. But
I've not been able to find out how to get the tool up. I'm missing
something. Any pointers greatly appreciated.
I'm also thinking of getting the AVR-ICE mkII - but it seems like its had
its interface revamped and is no longer gdb compatible. Any experience or
knowledge any one as to what is happening.
So excuse the dumb questions from a poor software engineer trying to get
a productive tool chain in order...
(::)
Neil Hancock
-----Original Message-----
From: Philip Levis [mailto:[EMAIL PROTECTED]
Sent: Tuesday, March 29, 2005 2:20 PM
To: Neil Hancock
Cc: Raghu Ganti; [email protected]
Subject: Re: [Tinyos-users] Debugging
On Mar 28, 2005, at 9:06 AM, Neil Hancock wrote:
> I'm also thinking about the same issue - how to do path level code
> coverage
> on the motes in a reasonably productive manner, and then integrate it
> into
> and debug on mote2dot.
>
Path-level code coverage is a bit tricky, due to the component
separations and implicit shared state across those boundaries.
My approach so far is to do so probabilistically. I try out lots of
scenarios with many nodes. As my code is almost always application
level (not, for example, a radio stack), I've found TOSSIM to generally
be sufficient. If there's a real crazy edge case that I think unlikely
to happen, then I trigger it explicitly, through Tython or a simple
nesC test case framework.
This hasn't caught every single edge case, but usually those that
remain are performance, rather than correctness problems, which I then
slowly discover and work through (e.g., send a redundant message rather
than crash).
Phil
-------
"We shall not cease from exploration
And the end of all our exploring
Will be to arrive where we started
And know the place for the first time."
- T. S. Eliot, 'Little Gidding'
_______________________________________________
Tinyos-users mailing list
[email protected]
http://mail.Millennium.Berkeley.EDU/mailman/listinfo/tinyos-users
