I've posted updated code coverage information for mozilla-central to
<https://www.tjhsst.edu/~jcranmer/m-ccov/>. This data is accurate as of
yesterday. For those of you who are unaware, I periodically run these
code coverage statistics by use of the try server and instrumented runs.
This has been made easier over the years by standardization of some of
the hacks, such that you can now push to linux64-cc and get most of the
same information.
Notable changes I've made since the last upload:
1. I dropped debug builds, so all the information comes from Linux opt,
both 32 and 64-bit.
2. Test names now derive from builder names directly, removing the need
for a very long hardcoded list of "M-bc means mochitest-browser-chrome".
2a. This means that what was once mochitest-1, mochitest-2, etc. is now
condensed into "mochitest". Mochitest-e10s-browser-chrome, etc., remain
split out.
3. Minor changes in the UI frontend to help deal with the fact that my
hosting webserver changed to forcing https.
4. I can now generate the ultimate combined .info file without needing
manual post-processing, for the first time ever.
The marionette and web-platform tests remain unaccounted for in coverage
(Mn, Mn-e10s, Wr, W-* in treeherder lingo), and the new "Ld"
(luciddream?) appears to be broken as well.
On the possibility of expanding code coverage information to different
platforms, languages, and tests:
1. OS X still has a link error and/or fail-to-run issue. I suspect a
newer clang would help, but I lack a local OS X instance with which to
do any detailed tests. I've never tested the ability of my scripts to
adequately collect clang code coverage data, and I suspect they would
themselves need some modification to do so.
2. Android builds work and can report back code coverage data, but so
intermittently that I didn't bother to try including them. In my try run
that I used to generate these results, mochitest-2 reported back data
but mochitest-6 did not, yet both testsuites reported back success. The
reason for this is not clear, so any help people could give in debugging
issues would be most appreciated.
3. B2G desktop builds and Mulet builds on Linux appeared to work.
However, the builds didn't appear to upload the gcno package for unknown
reasons, and taskcluster uses such different mechanisms to upload the
files that my scripts are of no use in collecting the gcda packages.
4. Windows is... special when it comes to code coverage. This is the
last platform I would look at tackling.
5. JS code coverage is of course a hole I'd like to see rectified, but I
don't have the time to invest in solving it myself.
6. Are we actually using Rust yet on our tryserver builds?
7. Android Java coverage is deferred until after I can get reliable
Android code coverage in the first place.
8. I'd have to look into modifying mozharness to run code coverage on
marionette et al builds. It shouldn't be hard, but it is annoying to
have to hook into so many places to insert code coverage.
9. Ditto for Android xpcshell and cppunit tests.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
