I think to get a very good coverage for the basic functionalities we need some sort of stress testing or fuzzing infrastructure because the most issues I hit come from compilers generating something a bit different then we are expecting. For stack unwinding I added a stress test (TestStandardUnwind.py, working on arm and aarch64) what catches a lot of bugs but running it takes so long that it can't be added to the everyday test suit. Should we try to design a fuzz testing infrastructure what runs on a build bot so we don't have to depend on the fuzz testing behavior of the normal test suit?
On Wed, Jan 20, 2016 at 7:22 PM Zachary Turner <ztur...@google.com> wrote: > zturner added a comment. > > In http://reviews.llvm.org/D16334#331338, @jingham wrote: > > > I sort of agree with this and sort of don't. Formally, I agree with the > notion of limited focused tests. But in practice it is often the noise in > tests that catches bugs that we don't yet have tests for. And especially > when the "noise" is doing things like step over that 100% should work in > any functional debugger... So I am also a little leery of cleaning up the > tests too much so that they only test the things we've thought to test and > miss other things. > > > > Jim > > > I think we should solve that by adding more tests though. I mean I don't > want to do a 1 step forward, 2 steps back kind of thing. If someone knows > of an area where we're missing coverage, then stop what you're doing and go > add the coverage. If it's the type of thing where we don't know we're > missing coverage and we're just hoping an unrelated test catches it > someday, I don't agree with that. Just because we know we have a problem > doesn't mean we should solve it the wrong way, because then we can never > possibly reach the desired end state since we're actively moving farther > away from it. > > > http://reviews.llvm.org/D16334 > > > >
_______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits