Success! I ran the Octane benchmarks with a developer tools session attached, hooking some code to tab to the 'Start Octane 2.0' button and press it instead, and got the following much more sensible results:
Richards: 2189 DeltaBlue: 2936 Crypto: 1355 RayTrace: 2532 EarleyBoyer: 1801 RegExp: 209 Splay: 318 SplayLatency: 2243 NavierStokes: 1073 PdfJS: 592 Mandreel: 763 MandreelLatency: 442 Gameboy: 1610 CodeLoad: 886 Box2D: 739 zlib: 2863 Typescript: 1081 Total: 1087 I then tried settings --js-flags="--always-opt" and using our normal developer tools approach, but that didn't seem to work (so 'always' doesnt really mean 'always'?). '--nodebugger' also didnt seem to affect things. Any ideas on how one could attach remote developer tools but still have V8 do its optimizations? Thanks, Stephen On Tuesday, 4 November 2014 09:25:02 UTC-5, Stephen wrote: > > Hi Paul, > > Thanks for the quick and enthusiastic reply! Notes inline: > > I see _no_ perf degradation. This is on 'mipsel', a mips32r2 device with >> floating point. > > > Alas, I am not that surprised. I assumed that such a large performance hit > wouldn't have slipped past the v8 folks, so I was reckoning it would turn > out to be something platform/environment specific. > > Such a serious perf drop is very likely due to incorrect deoptimizations. >> Or in the browser you could try --js-flags="--trace-deopt" in the browser. >> If you get something, please send logs. > > > I ran with --trace-opt and --trace-deopt this morning, and got some very > interesting logs, attached (and truncated, because its just the same thing > over and over). It looks like v8 is not incorrectly deoptimizing, instead > it is actually failing to optimize because it thinks the debugger is on: > > "[failed to optimize am3: Debugger is active]" > > Do you know how v8 decides if its in debug mode? We are building Chromium > for Release, so I'm not sure why it would put v8 in debug mode (if that's > what is happening here). I do fire the Octane tests via an attached Chrome > Developer Tools session, so I wonder if that puts v8 into debug mode? > > As far as debugging on your side, I suggest you do try v8 standalone >> > > Still working on this, albeit it may be moot. Managed to get it build > correctly for our platform but running into SIGILL when running Octane, so > either something is horribly corrupted or I failed to describe our platform > correctly to the build process :). > > On Monday, 3 November 2014 19:59:16 UTC-5, paul lind wrote: >> >> Hi Stephen - >> >> I've built v8 standalone at today's top-of-tree, the top of 3.27 branch >> (3.27.34.21), and your specific version 3.27.34.0) and all are fine. >> >> I see _no_ perf degradation. This is on 'mipsel', a mips32r2 device with >> floating point. >> >> I've just checked a Chromium ToT Android build from mid-last week, and it >> looks fine also. I will pull a M37 release branch, and build that too. >> >> You are on 37.0.2062.0. I see Android stable is on 37.0.2062.117. Not >> that it should make any difference with this terrible perf drop you are >> seeing. >> >> I'll build both Android and Linux at 37.0.2062.0 and see what I find. >> I've not looked at your traces in detail yet, I've been just >> sanity-checking perf on the devices we usually test on. >> >> As far as debugging on your side, I suggest you do try v8 standalone. >> Also, I'm using gcc 4.8.1 for these tests, using the Mentor/Code-Sourcery >> lite toolchain available here: >> >> https://sourcery.mentor.com/GNUToolchain/package12215/public/mips-linux-gnu/mips-2013.11-36-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2 >> >> paul >> >> >> On Nov 3, 2014, at 3:43 PM, Paul Lind <[email protected]> wrote: >> >> Very sorry Stephen, that is crazy bad. We'll get on this immediately and >> get it understood and fixed asap. >> >> paul >> >> >> On Nov 3, 2014, at 2:59 PM, Stephen <[email protected]> wrote: >> >> Dear v8-users, >> >> We have a port of Chromium to a MIPS(el) embedded system, and are >> currently in the process of upgrading it from M31 (31.0.1650.61, v8 version >> 3.21.18.9[1]) to M37 (37.0.2062.0, v8 version 3.27.34.0[1]). In doing so, >> we have run into what seems to be a serious regression in V8 performance. >> Running the Octane benchmark ( >> http://octane-benchmark.googlecode.com/svn/latest/index.html), these are >> the numbers on our platform for M31 and M37: >> >> M31: >> >> Richards: 1578 >> DeltaBlue: 951 >> Crypto: 1133 >> RayTrace: 1830 >> EarleyBoyer: 2347 >> RegExp: 256 >> Splay: 341 >> SplayLatency: 1434 >> NavierStokes: 1223 >> PdfJS: 580 >> Mandreel: 654 >> MandreelLatency: 649 >> Gameboy: 1112 >> CodeLoad: 756 >> Box2D: 501 >> zlib: 1769 >> Typescript: 887 >> Overall: 905 >> >> M37: >> >> Richards: 5.88 >> DeltaBlue: 3.66 >> Crypto: 21.4 >> RayTrace: 15.3 >> EarleyBoyer: 67.4 >> RegExp: 57.9 >> Splay: 55.5 >> SplayLatency: 487 >> NavierStokes: 36.6 >> PdfJS: 50.2 >> Mandreel: 4.60 >> MandreelLatency: 66.2 >> Gameboy: 1589 >> CodeLoad: 801 >> Box2D: 689 >> zlib: 2650 >> Typescript: 968 >> Overall: 87.0 >> >> Obviously we are rather concerned by these numbers - a 10x degradation in >> overall score, and nearly 300x in some benchmarks! SunSpider shows similar >> levels of regression - a 3x overall increase in time taken, and multiple >> 100x's in some individual benchmarks. >> >> I have begun to dig into some of the benchmarks, and am attaching traces >> taken against the Octane 'crypto' benchmark on M31 and M37 using the >> --js-flags="--perf" switch. The M37 version seems to spend an enormous >> amount of time in the 'am3' function: 16913 ticks (as opposed to 39 ticks >> in the M31 case). Might this indicate some sort of regression in MIPS math >> code? >> >> Are there any known performance regressions in v8 on MIPS(el) in M37, or >> any suggestions on how we can debug this further? Feel free to ask for >> further information/testing/etc, and I will do my best to provide! I have >> tried to build standalone ToT V8 for our platform, but ran into issues with >> warnings being converted to errors that I haven't dealt with yet. >> >> Thanks, >> Stephen >> >> [1]: According to http://omahaproxy.appspot.com/ >> >> -- >> -- >> v8-users mailing list >> [email protected] >> http://groups.google.com/group/v8-users >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> <v8_m37_crypto_trace.log><v8_m31_crypto_trace.log> >> >> >> >> -- >> -- >> v8-users mailing list >> [email protected] >> http://groups.google.com/group/v8-users >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- -- v8-users mailing list [email protected] http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
