Mikhail, Please see below...
On 11/30/06, Mikhail Loenko <[EMAIL PROTECTED]> wrote:
>4) We have cruise controls running classlibrary tests on DRLVM. We >need to decide what will we do when DRLVM+Classlib cruise control >reports failure. >IMHO It's pretty obvious that before we have found a commit causing that >we stop further commits.
We can try and see how this works. It is important to reduce regressions, but depending on the time it takes to find the truant commit, repeated failures could literally stop new commits for a very long timetime. I think that 0 regression periods can exist in a project, but it cannot be zero regression always. However, I am perfectly OK with declaring a zero regression period for a trial period of a few weeks.
5) If the cause of failure is DRLVM commit than it's pretty obvious >that we roll it back.
Yes.
6) If the cause is Classlib test commit than we check wether the test >is valid (e.g. run it on RI) and not VM-specific. If the test is valid >we exclude it in DRLVM exclude list. If the test is invalid or >VM-specific we roll it back or fix ASAP.
Put it in the DRLVM exclude list and file a JIRA with the test case. Not sure what VM-specific means. Ideally DRLVM and RI should be able to run the same tests, so not being able to run it is a DRLVM problem, isn't it?
7) If the cause is a Classlib commit then this commit either reveals >existing problem in DRLVM or introduces a regression. I don't have >strong preferences here whether we'd better exclude the test or roll >the commit back
If the test fails only on DRLVM why should the classlib commit roll back? I would think that you may want to do the same as in 6). I think the catch is figuring out quickly what broke things...a DRLVM change, classlib change or test change. I think that in this post mortem model we will have to hope that this can be detected quickly, specially if commits are going to stop.
8) We may also advise that committers don't commit changes just before >going sleep or vacation. They should wait reasonable time to receive >CC failure report if any. The problem is "reasonable time" is hardly >predictable
He should wait for results of the next CC run.
