Hi everybody,
During December 2019 we released the first version of PerfDocs
<https://firefox-source-docs.mozilla.org/tools/lint/linters/perfdocs.html>
which initially verified if all the Raptor performance tests have a test
description. Last week we (me [:alexandrui] and Gregory Mierzwinski
[:sparky]) made another release that added the document generation feature.
PerfDocs
<https://firefox-source-docs.mozilla.org/tools/lint/linters/perfdocs.html>
was born for the need of having a centralized way for the test metadata
(e.g. descriptions). It can be used with the command ./mach lint -l
perfdocs and it has 2 steps: verification and generation (--fix ing).
While the verification is performed by default, the generation requires
passing the --fix optional argument. What it does is to search in the
tree for any perfdocs directory that has certain content (2 files). The
first content file is config.yml which holds the location of the
framework's test manifests and the descriptions for the tests that are
found by PerfDocs using the manifests. The second file that is provided
in the folders, calledindex.rst, determines where/how the test
information will be displayed in the mozilla source tree docs
<https://firefox-source-docs.mozilla.org/> (under Testing/Performance
Testing) and lets users add extra framework information if needed. See
here
<https://dxr.mozilla.org/mozilla-central/source/testing/raptor/raptor/perfdocs>
for an example folder with Raptor. You can find the documentation
generated from that specification here
<https://firefox-source-docs.mozilla.org/testing/perfdocs/index.html>.
One thing to note is the ./mach lint -l perfdocs linter will FAIL if any
test/suite present in the framework manifest doesn't have an associated
description (individual or pattern-matched). If there are no tests
missing a description and all descriptions in perfdocs/config.yml are
matching at least one test then the linting will pass. It will also let
the user know if the documentation needs to be regenerated. When --fix
is provided as an argument, then verification will be performed along
with document (re-)generation. In the case of a linting failure, you
will need to update config.yml.
Example:
Given tests: raptor-tp6-amazon-firefox, raptor-tp6-amazon-chrome,
raptor-tp6-amazon-chromium and their -cold variants.
Individual matching:
raptor-tp6-amazon-firefox: "Warm desktop page-load performance test for
amazon on firefox using WebExtension."
Pattern matching:
raptor-tp6-amazon: "Cold and warm desktop page-load performance tests
for amazon on firefox, chrome, and chromium using WebExtension."
Many thanks to the perftest team who helped on reviewing this all along
the way.
For any questions do not hesitate to contact us.
Alex
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
