Hi, TL;DR: I wrote a crude yet useful JS memory profiler. I told jimb about it at the work week and am posting the details here in case it's of interest to anyone.
Earlier this year I made some large memory usage improvements to pdf.js (https://blog.mozilla.org/nnethercote/2014/06/16/an-even-slimmer-pdf-js/). I started by looking at about:memory, which told me that pdf.js was using lots of typed arrays. So I manually instrumented every place in pdf.js where a typed array was allocated -- a couple of hundred places, few enough to be doable -- with a dump() that printed the allocation size and identified the allocation site. I then used a post-processing script to collate and sort these. This was enough to identify and fix quite a few problems. But maintaining those dump() calls every time I updated the pdf.js code was tedious, so I switched to adding fprintf() calls in SpiderMonkey in all the places where we allocated significant things. Here's some example output from a pdf.js run: 241288386 counts: ( 1) 63183232 (26.2%, 26.2%): canvas 934 x 1208 [http://localhost:8888/src/display/canvas.js:719:8] (4513088) ( 2) 15795808 ( 6.5%, 32.7%): canvas 467 x 604 [http://localhost:8888/web/thumbnail_view.js:234:0] (1128272) ( 3) 3604480 ( 1.5%, 34.2%): buffer-new -> 32768 [http://localhost:8888/src/core/stream.js:160:20] ( 32768) ( 4) 3444672 ( 1.4%, 35.7%): gc-new tenured bg-object4 [<internal>] ( 64) ( 5) 3338000 ( 1.4%, 37.0%): gc-new tenured shape [<internal>] ( 40) ( 6) 2945664 ( 1.2%, 38.3%): gc-new tenured script [<internal>] ( 192) ( 7) 2800512 ( 1.2%, 39.4%): gc-new nursery bg-object8 [http://localhost:8888/src/core/evaluator.js:2212:14] ( 96) ( 8) 2654856 ( 1.1%, 40.5%): elems-grow 2048 -> 4097 [http://localhost:8888/src/core/function.js:53:8] ( 32776) ( 9) 2097152 ( 0.9%, 41.4%): buffer-new -> 1048576 [http://localhost:8888/src/core/stream.js:160:20] (1048576) ( 10) 1917048 ( 0.8%, 42.2%): gc-new tenured string [<internal>] ( 24) Things to note: - It only records allocations, not frees, so it shows cumulative allocations rather than live allocations. This is still very useful because (a) it captures short-lived allocations, (b) you don't need to worry about choosing the right time to measure. - All the recorded allocations combined totalled 241,288,386 bytes. - Each line indicates what kind of allocation it is, where it occurred, the cumulative size of all allocations matching that, and the percentage of all allocations that this represents. - It measures everything of interest, including objects, slots, elements, strings, string chars, and shapes. I also ended up including canvases, because they are significant for pdf.js. You can see them here. - The <internal> ones are mostly due to structured cloning, because pdf.js passes lots of data from a worker to the main thread. In practice that was typically 10--30% of the total. - It's effectively using a stack trace depth of 1. 90% of the time this is good enough. - Ion-compiled code can allocate GC things. I didn't have a good solution for handling this, so I just force-disabled Ion compilation. Here's output from the same run, but with the allocation kinds filtered out. So if you have multiple allocations from the same line they get combined. Also, all the <internal> ones get combined into a single line here. 241288386 counts: ( 1) 63183232 (26.2%, 26.2%): [http://localhost:8888/src/display/canvas.js:719:8] ( 2) 36305454 (15.0%, 41.2%): [<internal>] ( 3) 15795808 ( 6.5%, 47.8%): [http://localhost:8888/web/thumbnail_view.js:234:0] ( 4) 11519448 ( 4.8%, 52.6%): [http://localhost:8888/src/core/stream.js:160:20] ( 5) 7414440 ( 3.1%, 55.6%): [http://localhost:8888/src/core/fonts.js:6789:22] ( 6) 5303880 ( 2.2%, 57.8%): [http://localhost:8888/src/core/function.js:53:8] ( 7) 4774136 ( 2.0%, 59.8%): [http://localhost:8888/src/display/canvas.js:172:8] ( 8) 4415896 ( 1.8%, 61.6%): [http://localhost:8888/src/core/stream.js:471:16] ( 9) 3890869 ( 1.6%, 63.2%): [http://localhost:8888/src/shared/util.js:1570:6] ( 10) 3858452 ( 1.6%, 64.8%): [http://localhost:8888/src/core/image.js:590:23] Anyway, I'm writing about this because I thought some people might find it interesting. Although it's crude, it's effective -- almost all my pdf.js improvements were found with it. I've made two files available. First is the patch, which applies against revision 51419e405e61 from August: njn.valgrind.org/memprof.patch. Second is the post-processing script: njn.valgrind.org/memprof.py. The script should be used with the -w option and possibly the -f option, which filters out the allocation kinds. Nick _______________________________________________ dev-tech-js-engine-internals mailing list [email protected] https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
