On 03/01/2019 18:16, Steve Fink wrote:
Good points, but given that most failures will show up debug builds, it
seems like a more relevant metric is the difference between time(Opt) vs
min(time(debug), time(PGO)). Though debug builds may run slow enough
that it boils down to what you said?
Looking at Windows 64-bit jobs from a random push (
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=63027ff03effb04ed4bf53bbb0c9aa1bad4b4c9b
), I see:
pgo: build=119min + Wd1=15min
opt: build=55min + Wd1=13min
debug: build=46min + Wd1=22min
So by that, you get opt and debug Wd1 results back at the same time
(67-68min) and pgo Wd1 results take twice as long (134min). I imagine
there are much slower test jobs that make this situation cloudier, but
assuming the general pictures holds then it seems like opt is mostly
redundant with debug.
I think a good rule of thumb is that debug tests are about twice as slow
as opt, with the same chunking. So for a test job taking closer to an
hour on opt (which some do), you can easily be at 45 minutes longer for
opt results than debug. We could of course chunk more, but there's
overhead there that would eat some of the regained capacity.
I wonder if an alternative would be running opt+debug on integration
branches and pgo+debug on central. That would have the obvious
disadvantage that pgo-only failures would be caught much later, but it
would keep current end-to-end times for integration and slightly better
capacity savings. I don't know how common pgo-only failures are compared
to other things that we are only catching on central.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
