On 10/02/17 06:27, Brian Birtles wrote:
Hi,
It seems like the MANIFEST.json for web platform tests now includes a
checksum of test file contents. As a result, if you run './mach
web-platform-tests --manifest-update yer' on a clean checkout of m-c
you're likely to get a bunch of changes to MANIFEST.json showing up.
Should we be requiring people to update the MANIFEST.json whenever
they touch a file in testing/web-platform/tests (i.e. not just when
they add/remove files)?
This is too much to ask of people; it was effectively required when the
manifest format change first landed and the result was that the
associated lint got hidden on treeherder for being orange too often.
Therefore I changed the lint to only complain if the manifest changes
would result in the wrong tests being run, or tests being run incorrectly.
The current situation isn't ideal; this manifest is effectively a build
artifact, but the process of generating from scratch is rather slow and
would add around a minute to the build time, which isn't acceptable.
Making this faster is non-trivial because it's mostly parsing HTML
files; that's super slow in Python (Python bindings for html5ever would
be of interest here).
The advantage of the checksums in the file is faster incremental updates
with lower complexity. Given that, one possible short term win would be
to make --manifest-update the default for |mach wpt| so that people are
more likely to update it correctly when they are authoring tests.
(Currently when creating a patch queue that adds/removes files from
web-platform-tests my workflow involves first creating a base patch to
include all the checksum changes to m-c that haven't been added to
MANIFEST.json (so they don't show up in later patches in the series
when I need to run --manifest-update), regenerating that patch
whenever I rebase, and dropping it when I land the patch queue.
Probably I'm doing something wrong, however.)
Rebasing is a bit of a pain. I think it is possible to configure
mercurial (or git) to just run |mach wpt-manifest-update| whenever
there's a conflict in this file so that it's regenerated to be correct
rather than needing manual work (there are edge cases where this likely
won't work, but I think those are rare enough that it's worth trying).
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
