Re: Quantum Flow Engineering Newsletter #14

2017-06-27 Thread Benjamin Bouvier 
2017-06-26 19:40 GMT+02:00 Ehsan Akhgari :
> On 06/26/2017 09:52 AM, Armen Zambrano Gasparnian wrote:
>>
>> On Friday, 23 June 2017 10:05:42 UTC-4, Ehsan Akhgari  wrote:
>>>
>>> On Fri, Jun 23, 2017 at 4:19 AM, Chris Peterson 
>>> wrote:
>>>

 On 6/23/17 12:17 AM, Ehsan Akhgari wrote:

 But to speak of a more direct measurement of performance, let's look at
 our progress on Speedometer V2

 .
>>
>> Ehsan, were you comparing against http://speedometer2.benj.me?
>
> I used this instance for these measurements.
>>
>> Or this http://browserbench.org/Speedometer/?
>
> This is still Speedometer V1, FWIW.

Both websites indicate Speedometer V1 but are currently running Speedometer V2:
- The hosted version (under benj.me) is a direct clone of the Webkit
repository PerformanceTests/Speedometer with changes for running under
AWFY. I've manually updated the title's version number.
- For browserbench.org, see
https://github.com/WebKit/webkit/commit/ae7c2b286458944f61b33eea85fb89f263245b52#diff-0858c771e4c688cca20fc04d800816aa
. I've opened a pull request for updating their version number as
well.

>
>> Or using the AWFY code? (It uses local proxy)
>>
 Today, I measured our progress so far on this benchmark by comparing
 Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest
 Nightly, all x64 builds, on the reference hardware
 .  This is the result (numbers are
 the reported benchmark score, higher is better):

 [image: Speedometer improvements]


 How do these Speedometer V2 scores map to the results on AWFY? AWFY
 shows
 many Speedometer sub-tests, but no score in the range of 70.21. AWFY
 machine #36 is the reference hardware.


 https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc

>>> Armen has been investigating the difference.  The Speedometer benchmark
>>> is
>>> extremely sensitive to anything else that is going on on the machine at
>>> the
>>> time you are running the tests, see for example
>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1373396#c3 where he
>>> discovered
>>> that turning off the "Shut off display after 15 minutes" setting improves
>>> our benchmark score by about 10 points or so!  I think there are other
>>> investigations ongoing to dig into the remaining difference as well.
>>>
>> Currently, machine #36 is testing against non-PGO builds from inbound (to
>> catch regressions).
>> I'm looking into adding mozilla-central PGO builds to the mix.
>> I assume PGO builds can run slightly faster than non-PGO builds.
>>
>> Once we have PGO builds we will be able to see if there are anymore
>> machine configration changes required (I doubt it).
>>
>> Direct link to the speedometer score on machine #36:
>>
>> https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score
>>
>>> (Also note that there is a speed difference on Speedometer between
>>> Nightly
>>> and Beta, where Nightly with the same code will be a bit slower than Beta
>>> since some Nightly specific features do show up in Speedometer profiles
>>> currently, for example things like bug 1375568 are currently Nightly
>>> only,
>>> and there are also debugging checks like <
>>>
>>> https://searchfox.org/mozilla-central/rev/3291398f10dcbe192fb52e74974b172616c018aa/ipc/chromium/src/base/pickle.h#26>
>>> that show up a bit in profiles as well.  I think that explains why 55.0b3
>>> is scoring so high comparing to 56.0a1 there.)
>>>
>>> Cheers,
>>> --
>>> Ehsan
>>
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #14

2017-06-26 Thread Ehsan Akhgari 

On 06/26/2017 09:52 AM, Armen Zambrano Gasparnian wrote:

On Friday, 23 June 2017 10:05:42 UTC-4, Ehsan Akhgari  wrote:

On Fri, Jun 23, 2017 at 4:19 AM, Chris Peterson 
wrote:



On 6/23/17 12:17 AM, Ehsan Akhgari wrote:

But to speak of a more direct measurement of performance, let's look at
our progress on Speedometer V2
.

Ehsan, were you comparing against http://speedometer2.benj.me?

I used this instance for these measurements.

Or this http://browserbench.org/Speedometer/?

This is still Speedometer V1, FWIW.

Or using the AWFY code? (It uses local proxy)


Today, I measured our progress so far on this benchmark by comparing
Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest
Nightly, all x64 builds, on the reference hardware
.  This is the result (numbers are
the reported benchmark score, higher is better):

[image: Speedometer improvements]


How do these Speedometer V2 scores map to the results on AWFY? AWFY shows
many Speedometer sub-tests, but no score in the range of 70.21. AWFY
machine #36 is the reference hardware.

https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc


Armen has been investigating the difference.  The Speedometer benchmark is
extremely sensitive to anything else that is going on on the machine at the
time you are running the tests, see for example
https://bugzilla.mozilla.org/show_bug.cgi?id=1373396#c3 where he discovered
that turning off the "Shut off display after 15 minutes" setting improves
our benchmark score by about 10 points or so!  I think there are other
investigations ongoing to dig into the remaining difference as well.


Currently, machine #36 is testing against non-PGO builds from inbound (to catch 
regressions).
I'm looking into adding mozilla-central PGO builds to the mix.
I assume PGO builds can run slightly faster than non-PGO builds.

Once we have PGO builds we will be able to see if there are anymore machine 
configration changes required (I doubt it).

Direct link to the speedometer score on machine #36:
https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score


(Also note that there is a speed difference on Speedometer between Nightly
and Beta, where Nightly with the same code will be a bit slower than Beta
since some Nightly specific features do show up in Speedometer profiles
currently, for example things like bug 1375568 are currently Nightly only,
and there are also debugging checks like <
https://searchfox.org/mozilla-central/rev/3291398f10dcbe192fb52e74974b172616c018aa/ipc/chromium/src/base/pickle.h#26>
that show up a bit in profiles as well.  I think that explains why 55.0b3
is scoring so high comparing to 56.0a1 there.)

Cheers,
--
Ehsan

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #14

2017-06-26 Thread Armen Zambrano Gasparnian 
On Friday, 23 June 2017 10:05:42 UTC-4, Ehsan Akhgari  wrote:
> On Fri, Jun 23, 2017 at 4:19 AM, Chris Peterson 
> wrote:
> 
> >
> >
> > On 6/23/17 12:17 AM, Ehsan Akhgari wrote:
> >
> > But to speak of a more direct measurement of performance, let's look at
> > our progress on Speedometer V2
> > .

Ehsan, were you comparing against http://speedometer2.benj.me?
Or this http://browserbench.org/Speedometer/?
Or using the AWFY code? (It uses local proxy)

> > Today, I measured our progress so far on this benchmark by comparing
> > Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest
> > Nightly, all x64 builds, on the reference hardware
> > .  This is the result (numbers are
> > the reported benchmark score, higher is better):
> >
> > [image: Speedometer improvements]
> >
> >
> > How do these Speedometer V2 scores map to the results on AWFY? AWFY shows
> > many Speedometer sub-tests, but no score in the range of 70.21. AWFY
> > machine #36 is the reference hardware.
> >
> > https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc
> >
> 
> Armen has been investigating the difference.  The Speedometer benchmark is
> extremely sensitive to anything else that is going on on the machine at the
> time you are running the tests, see for example
> https://bugzilla.mozilla.org/show_bug.cgi?id=1373396#c3 where he discovered
> that turning off the "Shut off display after 15 minutes" setting improves
> our benchmark score by about 10 points or so!  I think there are other
> investigations ongoing to dig into the remaining difference as well.
> 

Currently, machine #36 is testing against non-PGO builds from inbound (to catch 
regressions).
I'm looking into adding mozilla-central PGO builds to the mix.
I assume PGO builds can run slightly faster than non-PGO builds.

Once we have PGO builds we will be able to see if there are anymore machine 
configration changes required (I doubt it).

Direct link to the speedometer score on machine #36:
https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score

> (Also note that there is a speed difference on Speedometer between Nightly
> and Beta, where Nightly with the same code will be a bit slower than Beta
> since some Nightly specific features do show up in Speedometer profiles
> currently, for example things like bug 1375568 are currently Nightly only,
> and there are also debugging checks like <
> https://searchfox.org/mozilla-central/rev/3291398f10dcbe192fb52e74974b172616c018aa/ipc/chromium/src/base/pickle.h#26>
> that show up a bit in profiles as well.  I think that explains why 55.0b3
> is scoring so high comparing to 56.0a1 there.)
> 
> Cheers,
> -- 
> Ehsan

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #14

2017-06-23 Thread Ehsan Akhgari 
On Fri, Jun 23, 2017 at 4:19 AM, Chris Peterson 
wrote:

>
>
> On 6/23/17 12:17 AM, Ehsan Akhgari wrote:
>
> But to speak of a more direct measurement of performance, let's look at
> our progress on Speedometer V2
> .
> Today, I measured our progress so far on this benchmark by comparing
> Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest
> Nightly, all x64 builds, on the reference hardware
> .  This is the result (numbers are
> the reported benchmark score, higher is better):
>
> [image: Speedometer improvements]
>
>
> How do these Speedometer V2 scores map to the results on AWFY? AWFY shows
> many Speedometer sub-tests, but no score in the range of 70.21. AWFY
> machine #36 is the reference hardware.
>
> https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc
>

Armen has been investigating the difference.  The Speedometer benchmark is
extremely sensitive to anything else that is going on on the machine at the
time you are running the tests, see for example
https://bugzilla.mozilla.org/show_bug.cgi?id=1373396#c3 where he discovered
that turning off the "Shut off display after 15 minutes" setting improves
our benchmark score by about 10 points or so!  I think there are other
investigations ongoing to dig into the remaining difference as well.

(Also note that there is a speed difference on Speedometer between Nightly
and Beta, where Nightly with the same code will be a bit slower than Beta
since some Nightly specific features do show up in Speedometer profiles
currently, for example things like bug 1375568 are currently Nightly only,
and there are also debugging checks like <
https://searchfox.org/mozilla-central/rev/3291398f10dcbe192fb52e74974b172616c018aa/ipc/chromium/src/base/pickle.h#26>
that show up a bit in profiles as well.  I think that explains why 55.0b3
is scoring so high comparing to 56.0a1 there.)

Cheers,
-- 
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Quantum Flow Engineering Newsletter #14

2017-06-23 Thread Chris Peterson 



On 6/23/17 12:17 AM, Ehsan Akhgari wrote:


But to speak of a more direct measurement of performance, let's look 
at our progress on Speedometer V2 
.  
Today, I measured our progress so far on this benchmark by comparing 
Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest 
Nightly, all x64 builds, on the reference hardware 
.  This is the result (numbers 
are the reported benchmark score, higher is better):


Speedometer improvements



How do these Speedometer V2 scores map to the results on AWFY? AWFY 
shows many Speedometer sub-tests, but no score in the range of 70.21. 
AWFY machine #36 is the reference hardware.


https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Quantum Flow Engineering Newsletter #14

2017-06-23 Thread Ehsan Akhgari 
Hi everyone,

We have about 13 more weeks before the train of Firefox 57 leaves the
station.  Next week many of you will be at the upcoming work week, so I
thought it may be a good time to have some retrospection over our progress
so far, just so that you can get a good sense of how to extrapolate when
you are planning things next week.

One difficulty with the Quantum Flow project is that since it touches many
different areas of the browser, it doesn't lend itself very easily to
drawing nice charts for it.  :-)  It is hard to find one metric that all of
this work fits inside, and that's OK.  My goal this week is to highlight
what we can achieve with focus in a limited amount of time, so I'll bring a
couple of examples.

This is a snapshot of our burndown chart1.  We currently have 182 closed
bugs and 136 open bugs.  That's great progress, and I'd like to thank
everyone who helped with all aspects of this!



But to speak of a more direct measurement of performance, let's look at our
progress on Speedometer V2
.
Today, I measured our progress so far on this benchmark by comparing
Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest
Nightly, all x64 builds, on the reference hardware
.  This is the result (numbers are
the reported benchmark score, higher is better):

[image: Speedometer improvements]

There are also many other top level performance related projects that are
ongoing and approaching final stages.  I'm really excited to see what the
next few months are going to uncover for Firefox performance.

One bit of administrative note, as next week most people are able to get
updates from each other face to face, I won't send out the newsletter.  Now
let's finish with this week's list of acknowledgements to those who helped
make Firefox faster during the past week, hopefully I'm not forgetting any
names!

   - Jan de Mooij optimized object property enumeration
    with great results
   .  He finished
   his ongoing optimization effort on adding/defining properties
   .  He also reordered
   the checks in ValueToId() and ValueToIdPure() to cover the more common
   cases first .  He
   then moved the object/string pre-barrier null check to JIT code
   .  This speeds up
   initializing unboxed objects.
   - Makoto Kato improved the performance of HTMLInputElement.value setters
   by avoiding doing some unnecessary work
   .
   - Alessio Placitelli deferred querying whether we're the default browser
   to until after session restore completes
    in order to avoid
   potential startup slowdown issues.
   - Markus Stange made arrow
    panel animations
    much more
   efficient on macOS!
   - Doug Thayer made it so that we shrink the Places SQLite database off
   of the main thread
   
,
   avoiding some jank when hitting memory pressure.
   - Jonathan Kew switched to using a faster API for retrieving font glyph
   advances instead of all glyph metrics on DirectWrite
   .
   - Mark Banner converted the bookmarks backup code to use the new
   asynchronous bookmarks API
   .
   - Matthew Noorenberghe made the context menu code lazily import
   LoginManagerContextMenu.jsm and InlineSpellChecker.jsm
    at startup.
   - Florian Quèze avoided attaching an XBL binding to hidden 
   elements  to
   improve startup performance.
   - Boris Zbarsky avoided querying the preferences service to determine
   whether interface enablement conditions are met
   .
   - Jonathan Watt optimized calls to nsUXThemeData::InitTitlebarInfo()
   , improving the
   performance of opening a window on Windows.
   - Nicholas B. Pierron enabled backtracking to a normal call when
   inlining fails in IonMonkey
   .
   - Mats Palmgren continued his
    battle
    on unnecessary