Thanks a lot for doing this work Nicholas.
I would like to add that once this hits automation the perf and memory
usage improvements will have a measurable impact on the tests runtime.
That's because we feed the output of all our tests on automation to the
scripts, and if they hit even a single function in libxul they will load
the entire symbol file and keep it in memory in a very
memory-inefficient format. This can take minutes and consume gigabytes
of memory. So much that at some point we even ran into OOM scenarios
because of it (bug 1493365 [1] for example).
Gabriele
[1] Intermittent PLDHashTableTest.GrowToMaxCapacity | Value of: false
https://bugzilla.mozilla.org/show_bug.cgi?id=1493365
bug 1493365
On 05/03/2020 23:52, Nicholas Nethercote wrote:
> Hi all,
>
> I have written a new stack-fixing tool, called `fix-stacks`, which
> symbolizes stack traces produced by Firefox.
>
> `fix-stacks` is intended to replace our existing stack-fixing scripts,
> `fix_{linux,macosx}_stack.py` and `fix_stack_using_bpsyms.py`, which are
> used (a) on many test outputs, both locally and on automation, and (b) by
> DMD, Firefox's built-in heap profiler.
>
> `fix-stacks` is now installed by `mach bootstrap` on Windows, Mac, and
> Linux. It is usable from Python code via the `fix_stacks.py` wrapper
> script. Its code is at https://github.com/mozilla/fix-stacks/.
>
> In bug 1604095 I replaced the use of `fix_{linux,macosx}_stack.py` with
> `fix_stacks.py` for DMD, with the following benefits.
>
> * On Linux, stack-fixing of DMD output files (which occurs when you run
> `dmd.py`) is roughly 100x faster. On my Linux box I saw reductions from
> 20-something minutes to ~13 seconds.
>
> * On Mac, stack-fixing of DMD output files on Mac is roughly 10x faster. On
> my Mac laptop I saw reductions from ~7 minutes to ~45 seconds.
>
> * On Windows, stack-fixing of DMD output files now occurs. (It previously
> did not because there is no `fix_window_stacks.py` script.) This means that
> DMD is now realistically usable on Windows without jumping through hoops to
> use breakpad symbols.
>
> There is more work to be done. Soon, I plan to:
>
> * use `fix-stacks` on test outputs (in `utils.py` and `automation.py.in`);
>
> * re-enable stack fixing on Mac test runs on local builds, which is
> currently disabled because it is so slow;
>
> * add breakpad symbol support to `fix-stacks`;
>
> * remove the old scripts.
>
> The tree of relevant bugs can be seen at
> https://bugzilla.mozilla.org/showdependencytree.cgi?id=1596292&hide_resolved=1
> .
>
> The stack traces produced by `fix-stacks` are sometimes different to those
> produced by the old stack-fixers. In my experience these differences are
> minor and you won't notice them if you aren't looking for them. But let me
> know if you have any problems.
>
> Nick
> _______________________________________________
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
