On Mon, Sep 19, 2011 at 1:31 PM, N Keywal <nkey...@gmail.com> wrote: > Hi, > > Something that would be useful as well is being able to match a given > version of the trunk with the test results. For example: > - Dev pull the last version of the trunk at time t > - Dev do its modification locally to the code > - Dev runs the unit tests, and compare it to the results on the server at > time t. > - Then he can compare the results locally after its modifications vs. the > remote results on the trunk. > > This would save the time of running the tests twice. > > May be it's actually already possible? > > Ideally (and its generally the case) that the tests should all be passing on the server when developers pull down trunk. So when the developer runs the test on their changes, all the tests should still pass.
The main problem seems to come in with the quality/configuration of developer machines when running the tests. Maybe you don't have enough memory, or local config is a little weird, etc. so some of the tests don't end up passing locally - even given a 'working' version from the server. A reason for interest in this patch is that many times, some of the tests don't seem to pass deterministically on the local machine, for whatever reason. Therefore, the dev is left without certainty of whether or not the change they made _really_ works. By cutting down the test time we can make it easier to iteratively see which tests are failing. Also a lot of this cleanup should help reduce some of the non-determinism. Also, if you do break of other tests, it will be pretty obvious what was broken. So you still need to double runs tests to handle/catch local idiosyncrasies, but they will be less common AND easier to find/debug. Unless I'm missing something. -Jesse Yates
