On Fri, Sep 16, 2011 at 11:39 PM, Akash Ashok <[email protected]> wrote: > Firstly I end up running the whole suite without a the patch and second > with patch and compare the results. But it turns out most of the times > there's always inconsistency. Meaning to say a few tests are erratic. The > fail while running the whole suite and pass when run alone. Moreover running > the whole test suite takes about 2 hours. > > This is taking a lot of time for me to work on the simplest of the patches. >
Yes, this is a problem. That the tests take so long to run, I'm sure, is a barrier to contribution. It also in part explains why our test builds are so often red -- because its onerous running full suite. We used to huff and puff about the fact that our test suite took an hour. I hadn't realized we were beyond the two hour mark now. Need to work on this. Will file some issues for the longer-running tests. Some we can aggregate. Whats really needed are better tools and more use of Interfaces so the parts not under test can be mocked rather than have to stand up real instances of an hdfs or mapreduce cluster. Need to work on this too. That our test suite is seen to be flakey doesn't inspire confidence and makes it so you do the crazy thing of having to run once to see what the current state is before you apply your patch (two hours later). This should start to improve at least on the branch when we start to work on stabilization. At a place I used to work, if you broke the build you were given a giant purple barney -- http://www.barney.com/usa/index.asp -- and you had to wear it on your desk until someone else broke the build and then you could pass it off. The contortions fellas went through to avoid being barney-ified were monstrous. We need something like a virtual barney for hbase. Suggestions? St.Ack
