[chromium-dev] Re: Perf tests to run?
On Tue, Nov 3, 2009 at 7:30 AM, Anton Muhin ant...@chromium.org wrote: So the question: how could I check that I won't (notably) regress Chromium performance with this change? We have perf bots; you can check in and look at the effects there, and there might even be a way to get perf results from the tryservers. PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Perf tests to run?
Thanks a lot, Peter. Yes, I am aware of those. Just wanted to run something before committing :) I actually ran some DOM related tests. Adam (on IM) suggested to run SunSpider, V8 and page load cyclers. I am going to do it and if numbers are fine, then would try to commit my change. yours, anton. On Tue, Nov 3, 2009 at 9:25 PM, Peter Kasting pkast...@google.com wrote: On Tue, Nov 3, 2009 at 7:30 AM, Anton Muhin ant...@chromium.org wrote: So the question: how could I check that I won't (notably) regress Chromium performance with this change? We have perf bots; you can check in and look at the effects there, and there might even be a way to get perf results from the tryservers. PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Perf tests to run?
One trick that I've regularly done when concerned about the perf impact of a patch is to: a) Wait till late at night pacific time, when very few checkins are landing (often there are multi-hour gaps). b) Land your change, and wait till the build bots all start building (about a minute later). c) Revert your change. You then get a PILE of perf stats with minimal effort. It is MUCH harder to try to emulate this effect on your own machine(s), as the perf bots are dedicated to the chore, and you have a control build to watch... and a before/after set of results to compare with. By doing this late at night you have the highest probability of getting the impact of ONLY your change, and not having a second contemporaneous landing add noise to your results. :-) If you don't use those computer cycles late at night... we were going to have to waste them! YMMV. Jim p.s., Be sure you are good at doing reverts before playing this game ;-). On Tue, Nov 3, 2009 at 2:04 PM, Anton Muhin ant...@chromium.org wrote: Thanks, James. So I ran SunSpider, V8 and page_cyclers. SunSpider and V8 are roughly the same. page_cyclers trickier. First of all, I failed to run some tests on my box (for both clean and patched builds)---all *Http tests just hang. I hope it's missetup on my part, but hopefully not crucial. Of tests ran for reference build second run (patched version) was slightly faster, so numbers of patched version should be higher just due to somewhat different state of my box. Still for most of the tests patched version performed slightly faster, the only problematic test was the first one (MozFile): 35326 ms total for clean vs. 38936 ms total for patched. Given that's the only one (e.g. Intl1File ran 115913 ms on clean vs. 94831 ms on patched), that might have been a spike. If that sounds reasonable to you all, I'd ask to commit-queue+ a patch and see how it'd perform on real buildbots. tia and yours, anton. On Tue, Nov 3, 2009 at 11:31 PM, James Robinson jam...@google.com wrote: Essentially this. Some of the page cyclers have fairly interesting data sets. I think we are fairly lacking in good real-world performance tests to do comparisons like this, which is kind of unfortunate. I'd love to be able to point a build with a speculative patch in it at the internet and then have it report back how well it did but there are a lot of variables to control. - James On Tue, Nov 3, 2009 at 10:38 AM, Anton Muhin ant...@chromium.org wrote: Thanks a lot, Peter. Yes, I am aware of those. Just wanted to run something before committing :) I actually ran some DOM related tests. Adam (on IM) suggested to run SunSpider, V8 and page load cyclers. I am going to do it and if numbers are fine, then would try to commit my change. yours, anton. On Tue, Nov 3, 2009 at 9:25 PM, Peter Kasting pkast...@google.com wrote: On Tue, Nov 3, 2009 at 7:30 AM, Anton Muhin ant...@chromium.org wrote: So the question: how could I check that I won't (notably) regress Chromium performance with this change? We have perf bots; you can check in and look at the effects there, and there might even be a way to get perf results from the tryservers. PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Perf tests to run?
Thanks, James. So I ran SunSpider, V8 and page_cyclers. SunSpider and V8 are roughly the same. page_cyclers trickier. First of all, I failed to run some tests on my box (for both clean and patched builds)---all *Http tests just hang. I hope it's missetup on my part, but hopefully not crucial. Of tests ran for reference build second run (patched version) was slightly faster, so numbers of patched version should be higher just due to somewhat different state of my box. Still for most of the tests patched version performed slightly faster, the only problematic test was the first one (MozFile): 35326 ms total for clean vs. 38936 ms total for patched. Given that's the only one (e.g. Intl1File ran 115913 ms on clean vs. 94831 ms on patched), that might have been a spike. If that sounds reasonable to you all, I'd ask to commit-queue+ a patch and see how it'd perform on real buildbots. tia and yours, anton. On Tue, Nov 3, 2009 at 11:31 PM, James Robinson jam...@google.com wrote: Essentially this. Some of the page cyclers have fairly interesting data sets. I think we are fairly lacking in good real-world performance tests to do comparisons like this, which is kind of unfortunate. I'd love to be able to point a build with a speculative patch in it at the internet and then have it report back how well it did but there are a lot of variables to control. - James On Tue, Nov 3, 2009 at 10:38 AM, Anton Muhin ant...@chromium.org wrote: Thanks a lot, Peter. Yes, I am aware of those. Just wanted to run something before committing :) I actually ran some DOM related tests. Adam (on IM) suggested to run SunSpider, V8 and page load cyclers. I am going to do it and if numbers are fine, then would try to commit my change. yours, anton. On Tue, Nov 3, 2009 at 9:25 PM, Peter Kasting pkast...@google.com wrote: On Tue, Nov 3, 2009 at 7:30 AM, Anton Muhin ant...@chromium.org wrote: So the question: how could I check that I won't (notably) regress Chromium performance with this change? We have perf bots; you can check in and look at the effects there, and there might even be a way to get perf results from the tryservers. PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
