tberghammer added a comment. In http://reviews.llvm.org/D16334#331428, @zturner wrote:
> In http://reviews.llvm.org/D16334#331420, @tberghammer wrote: > > > In http://reviews.llvm.org/D16334#331368, @zturner wrote: > > > > > I don't know, I still disagree. If something in step-over breaks, I > > > dont' want to dig through a list of 30 other tests that have nothing to > > > do with the problem, only to find out 2 days later that the problem is > > > actually in step over. The only reason this helps is because the test > > > suite is insufficient as it is. But it doesn't need to be! > > > > > > I agree but first we should fix the test coverage and then fix the > > individual tests. Doing it in the opposite way will cause a significant > > drop in quality (we will fix individual tests but not increase the coverage > > enough). > > > > > The real solution is for people to start thinking about tests more. I've > > > hounded on this time and time again, but it seems most of the time tests > > > only get added when I catch a CL go by with no tests and request them. > > > Sometimes they don't even get added then. "Oh yea this is on my radar, > > > I'll loop back around to it." <Months go by, no tests>. Hundreds of CLs > > > have gone in over the past few months, and probably 10 tests have gone > > > in. *That's* the problem. People should be spending as much time > > > thinking about how to write tests as they are about how to write the > > > thing they're implementing. Almost every CL can be tested. Everything, > > > no matter how small, can be tested. If the SB tests are too heavyweight, > > > that's what the unit tests are for. IF there's no SB API that does what > > > you need to do to test it, add the SB API. "But I have to design the API > > > first" is not an excuse. Design it then. > > > > > > I think we need a different API for tests then the SB API which can be > > changed more freely without have to worry about backward compatibility. > > When adding a new feature I try to avoid adding an SB API until I know for > > sure what data I have to expose because a wrong decision early on will > > carry forward (how many deprecated SB API calls we have?). > > > Do you have a concrete example of where you don't want to add an SB API, but > a unit test isn't ideal? Recently we added several language specific commands for render script and for go. I don't think we want to add SB API support for these at the moment because nobody uses them so maintaining it could be problematic. In http://reviews.llvm.org/D16334#331429, @zturner wrote: > In http://reviews.llvm.org/D16334#331420, @tberghammer wrote: > > > It is true that every CL can be tested but a lot of change is going in to > > address a specific edge case generated by a specific compiler in a strange > > situation. To create a reliable test from it we have to commit in a > > compiled binary with the strange/incorrect debug info and then it will be a > > platform and architecture specific test what is also very hard to debug > > because you most likely can't recompile it with your own compiler. I am not > > sure we want to go down in this road. > > > You can test cases like this easily with unit tests and dependency injection. > Just make a mock interface that returns the exact values you need when > queried for certain symbols or whatever. To do this we have to mock a lot of thing including the full Process and Thread classes what I am not too fancy doing because I expect that a completely unrelated change will break the mock. Personally I am more happy to debug a complicated test then to maintain a huge number of tests but I know that lot of people have different preferences. Also I prefer using stress tests and fuzzing instead of trying to create a specific test for each edge case we can think about because it keeps the number of tests low and relatively easy to maintain with having a high probability of detecting issues. http://reviews.llvm.org/D16334 _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
