[ http://issues.apache.org/jira/browse/GUMP-21?page=all ] Leo Simons closed GUMP-21: --------------------------
Resolution: Fixed > FOG factor is misleading and requires improvement > ------------------------------------------------- > > Key: GUMP-21 > URL: http://issues.apache.org/jira/browse/GUMP-21 > Project: Gump > Type: Improvement > Components: Python-based Gump > Versions: Gump2 > Reporter: Daniel Savarese > Assignee: Adam Jack > Fix For: Gump2 > > FOG is defined right now as: > def getFOGFactor(self): > return round((float(self.successes) / (float(self.failures) + > float(self.prereqs))), 2) > This is not helpful. Last month, jakart-oro > failed to build by coincidence around the time of a Gump > run and the problem was fixed before the next Gump run, > yet its FOG factor dropped in half. The responsiveness > to resolving build and integration problems is not taken > into account. It is likely that it is impossible for FOG > to mean anything because the factors that determine > a project's "health" (to borrow the term used by Adam Jack > in an email to the commons-dev and gump lists) are largely > orthogonal. FOG should probably be a vector and if you really > wanted a single number you could use the length of that vector. > One of the major problems with the current factor is that > a project's FOG factor continues to increase when no changes > have been made to the repository. It's not meaningful to > count only successful builds towards improving the factor > because no changes may have been made to the code base. > Ideally, you count only builds that cause no failures after > a change to the code base. FOG doesn't factor in how quickly > (or slowly) projects recover from build failures, and how often > changes to a code base cause failures (either in itself or in > other projects). Right now, a project with few code changes > (say once a month) that causes a failure 50% (1 every 2 months) > of the time the code is changed with a mean time of failure > correction of 1 week is deemed more reliable (higher FOG factor) > than a project that makes frequent changes (say every day) to > its code base and causes a failure 10% of the time (1 every 10 > days) with a mean time of failure correction of 1 day. That's > because FOG is based almost entirely on consecutive successful > Gump builds, which is not useful. > Rather than propose a new meaningless way of computing FOG, I > suggest FOG stop being used temporarily. It would be more > useful to break out and report all of the independently useful > metrics and through experience over time determine how each > impacts the overall notion of FOG and eventually derive a > way of condensing the information (perhaps a FOG factor, > perhaps a vector) in a useful manner. Right now the state > of affairs is that someone decided FOG was a cool thing, so > it's being reported and people see this number and mistakenly > think a project with half the FOG factor of another is > less dependable and therefore avoid using it. Alternatively > an explanation of what FOG is intended to compute and how > it is computed should be placed on Gump reports that list > FOG so that consumers of the information will not be misled. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]