Re: [chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 1:09 PM, Tei oscar.vi...@gmail.com wrote: Chromium is kind of awesome, but on my Ubuntu crash a bit too much. I am looking forward for a stable version. Dudes!, speed the thing. Will be this thing in the hoven for much longer? I grown impatient. This is the wrong place to bring this up. Note that non-Google builds of Chromium do not send crash reports and that we can't fix bugs we don't hear about. Please file bugs here: http://code.google.com/p/chromium/issues/entry?template=Defect%20on%20Linux -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
Our messages crossed in the ether ... On Thu, Dec 3, 2009 at 10:07 AM, Ojan Vafai o...@google.com wrote: +chromium-dev as others who look at the waterfall might also be confused. On Thu, Dec 3, 2009 at 8:50 AM, Dimitri Glazkov dglaz...@google.com wrote: Sending out random people, because it's early :) There's a couple of things I see on the bot this morning: 1) There's a crashing test on all bots -- and the tree is still green! http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/layout_tests/flakiness_dashboard.html#tests=LayoutTests/plugins/embed-attributes-setting.html The test is consistently crashing when run with all the other tests, but passing when we retry it in isolation. Note that the test is listed as an unexpected flaky test on the waterfall. This is one of the downsides of retrying failing tests. We can't distinguish flakiness from this case. We just need to careful to not ignore unexpected flakiness on the waterfall. Note that the dashboard only shows the result from the first run. Including the retry results from the bots seems like more trouble than it's worth. Agreed. However, why aren't the webkit bots orange in the main waterfall? 2) WebKit (dbg) shows insane fixable numbers -- 6561 fixable? This is a bug from Dirk's commit yesterday. Dirk, PrepareListsAndPrintOutput creates the ResultSummary object with the full test list, then shards the test list, then runs the tests. So the value used for number of tests run is the total number of tests, not the sharded total number of tests. Yup. I'll fix this when I get in (~11:30ish). -- Dirk -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 10:07 AM, Ojan Vafai o...@google.com wrote: +chromium-dev as others who look at the waterfall might also be confused. On Thu, Dec 3, 2009 at 8:50 AM, Dimitri Glazkov dglaz...@google.com wrote: Sending out random people, because it's early :) There's a couple of things I see on the bot this morning: 1) There's a crashing test on all bots -- and the tree is still green! http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/layout_tests/flakiness_dashboard.html#tests=LayoutTests/plugins/embed-attributes-setting.html The test is consistently crashing when run with all the other tests, but passing when we retry it in isolation. Note that the test is listed as an unexpected flaky test on the waterfall. This is one of the downsides of retrying failing tests. We can't distinguish flakiness from this case. We just need to careful to not ignore unexpected flakiness on the waterfall. Note that the dashboard only shows the result from the first run. Including the retry results from the bots seems like more trouble than it's worth. Should unexpected flakiness turn the bot red? :DG -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 10:09 AM, Dimitri Glazkov dglaz...@google.comwrote: On Thu, Dec 3, 2009 at 10:07 AM, Ojan Vafai o...@google.com wrote: +chromium-dev as others who look at the waterfall might also be confused. On Thu, Dec 3, 2009 at 8:50 AM, Dimitri Glazkov dglaz...@google.com wrote: Sending out random people, because it's early :) There's a couple of things I see on the bot this morning: 1) There's a crashing test on all bots -- and the tree is still green! http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/layout_tests/flakiness_dashboard.html#tests=LayoutTests/plugins/embed-attributes-setting.html The test is consistently crashing when run with all the other tests, but passing when we retry it in isolation. Note that the test is listed as an unexpected flaky test on the waterfall. This is one of the downsides of retrying failing tests. We can't distinguish flakiness from this case. We just need to careful to not ignore unexpected flakiness on the waterfall. Note that the dashboard only shows the result from the first run. Including the retry results from the bots seems like more trouble than it's worth. Should unexpected flakiness turn the bot red? If it turns the bot red, then it defeats the purpose of that code. Might as well not retry and mark it as FAIL. (which turns the tree red). Nicolas :DG -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 10:17 AM, Nicolas Sylvain nsylv...@chromium.org wrote: Should unexpected flakiness turn the bot red? If it turns the bot red, then it defeats the purpose of that code. Might as well not retry and mark it as FAIL. (which turns the tree red). Duh :) Makes sense. How about we turn red for unexpected crashiness? :DG -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 10:21 AM, Dimitri Glazkov dglaz...@google.com wrote: How about we turn red for unexpected crashiness? Makes sense to me. We can just not retry tests that unexpectedly crash. On Thu, Dec 3, 2009 at 10:09 AM, Dirk Pranke dpra...@google.com wrote: On Thu, Dec 3, 2009 at 10:07 AM, Ojan Vafai o...@google.com wrote: The test is consistently crashing when run with all the other tests, but passing when we retry it in isolation. Note that the test is listed as an unexpected flaky test on the waterfall. This is one of the downsides of retrying failing tests. We can't distinguish flakiness from this case. We just need to careful to not ignore unexpected flakiness on the waterfall. Note that the dashboard only shows the result from the first run. Including the retry results from the bots seems like more trouble than it's worth. Agreed. However, why aren't the webkit bots orange in the main waterfall? They are orange. We don't show orange in the console view or summary view at the top part of the waterfall though. I'm not really sure why. Nicolas, you know? I know in the console view we use orange to mean something else, but maybe we should not overload the meaning of orange so much. :) -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 10:25 AM, Ojan Vafai o...@chromium.org wrote: On Thu, Dec 3, 2009 at 10:21 AM, Dimitri Glazkov dglaz...@google.com wrote: How about we turn red for unexpected crashiness? Makes sense to me. We can just not retry tests that unexpectedly crash. On Thu, Dec 3, 2009 at 10:09 AM, Dirk Pranke dpra...@google.com wrote: On Thu, Dec 3, 2009 at 10:07 AM, Ojan Vafai o...@google.com wrote: The test is consistently crashing when run with all the other tests, but passing when we retry it in isolation. Note that the test is listed as an unexpected flaky test on the waterfall. This is one of the downsides of retrying failing tests. We can't distinguish flakiness from this case. We just need to careful to not ignore unexpected flakiness on the waterfall. Note that the dashboard only shows the result from the first run. Including the retry results from the bots seems like more trouble than it's worth. Agreed. However, why aren't the webkit bots orange in the main waterfall? They are orange. We don't show orange in the console view or summary view at the top part of the waterfall though. I'm not really sure why. Nicolas, you know? No idea, did you not write this? :) I know in the console view we use orange to mean something else, but maybe we should not overload the meaning of orange so much. :) I think it's the goal of the console view to try hard not to attribute flakiness to a change that most likely did not cause it. The flakiness dashboard does a good job tracking this already. That said, if someone can find a good color for fail again, then I'd use it and use orange like on the waterfall. Nicolas -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: Something smells weird on the buildbot
On Thu, Dec 3, 2009 at 10:25 AM, Ojan Vafai o...@chromium.org wrote: On Thu, Dec 3, 2009 at 10:21 AM, Dimitri Glazkov dglaz...@google.com wrote: How about we turn red for unexpected crashiness? Makes sense to me. We can just not retry tests that unexpectedly crash. I'll make this change if we have a consensus. Do we? -- Dirk -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
