On Sat, Aug 22, 2009 at 5:54 PM, Peter Kasting <[email protected]>wrote:
> On Sat, Aug 22, 2009 at 4:29 PM, Jeremy Orlow <[email protected]> wrote: > >> On Sat, Aug 22, 2009 at 4:00 PM, Peter Kasting <[email protected]>wrote: >> >>> If you somehow managed to not see any comments in this file, I think >>> you're an outlier. >>> >> >> I was talking about the rebaselining teams, not the act of actually >> rebaselining. If someone's rebaselining a test, then it means we now >> believe it's passing. At that point, the notes are not very interesting, >> right? Are you saying that you looked at all the tests' notes before you >> looked through the results to determine if they should be rebaselined? >> > > I certainly looked at them during the process of determining what was going > on, and left several notes of my own. > > I don't think I understand your objection. Are you saying notes are > useless or that they're harmful? I don't think either is true. If you're > trying to determine how to fix a layout test, the notes in the file are one > of the first things you see, because you're looking in the file to find the > bug #, what OSes are affected, etc. At that point notes that say what to > look for are useful. If you're trying to determine whether to rebaseline a > test, notes are at worst harmless and at best useful in pointing out some > subtlety that you overlooked if you'd already made your decision. You HAVE > to see the notes because you HAVE to edit the file. > > Notes in test_expectations.txt are like comments in source code: A great > boon. > I've herd differing opinions, but you're the definitely the most gung-ho I've talked to about notes in the test_expectations.txt file. Typically bugs are where most if not all of the information on failures should be kept. If there is information in the test_expectations.txt file, it should certainly be a subset of the information in the bugs, would you not agree? > > There are different reasons for failing. A layout test could be failing >>>> because of a known bug and then start failing in a different way (later) >>>> due >>>> to a regression. When a bug fails in a new way, it's worth taking a quick >>>> look, I think. >>>> >>> >>> Why? Unless the earlier failure has been fixed we can't rebaseline the >>> test. (I ran into a number of tests like this when doing my rebaselining >>> pass.) What is the point of looking again? >>> >> >> In case the new failure is more serious than the earlier one. >> > > The only possible reason I could think that would matter is if we're using > this as a source of triage input into which bugs we should fix first. But > we have so many thousands of bugs, nearly all likely to be higher priority > than a second failure in a test we already haven't prioritized fixing, that > I don't consider this a valuable signal. > I suppose that is true. --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
