On 2013-02-04 4:39 PM, Brian Smith wrote:> Ehsan Akhgari wrote:
>> On 2013-02-04 11:44 AM, Ehsan Akhgari wrote:
>>> 3. What is the performance difference between Visual Studio
>>> 2012 PGO builds and Visual Studio 2010 builds? IMO, before
>>> we decide whether to disable PGO on Windows, we need to get
>>> good benchmark results for Visual Studio **2012** PGO builds,
>>> to make sure we're not throwing away wins that could come
>>> "just" solving this problem in a different way + upgrading
>>> the compiler.
>>>
>>>
>>> That's something that we should probably measure as well. Filed
>>> bug 837724 for that.
>>
>> Note that I misread this and thought you're talking about VS2010 PGO
>> builds versus VS2012 non-PGO builds, and that's what bug 837724 is
>> about. As I've already said in this thread, VS2012 uses more memory
>> for PGO compilations than VS2010, so upgrading to that for PGO builds
>> is out of the question.
>
> That seems to be assuming that there is nothing reasonable we can do
to make VS2012 PGO builds work. However, in order to know what is a
reasonable amount of effort, you have to know what the benefits would
be. For example, let's say we lived in a magical alternate universe
where VS2012 PGO builds cut Firefox's memory usage by 50% and made
everything twice as fast compared to VS2010 PGO builds. Then, we would
consider even man-years of effort to be reasonable. On the other hand,
if Firefox were twice as slow when built with VS2012 PGO builds, then no
amount of effort would be reasonable. So, you have to know the
performance difference between VS2012 PGO builds and VS2010 PGO builds
before we can reject the possibility of VS2012 PGO.
I'm entirely puzzled by what you're suggesting here... VS2012 PGO
builds work fine. It's just that the linker max vmem usage is higher,
as I have mentioned in my first post to this thread. So for the current
discussion, we don't get any gains from switching to VS2012 (and in fact
we lost ~200MB of memory.) As long as that stands, any other changes in
the characteristics of Firefox compiled with VS2012 (such as Firefox's
memory usage or speed) are irrelevant for the current discussion.
If and when we decide to look into upgrading to VS2012 for other
reasons, those other factors can be investigated.
> Also, I want to echo khuey's comment: It seems like a lot of the
argument against PGO is that, while our benchmarks are faster, users
won't actually notice any difference. If that is true, then I agree with
khuey that that is a massive systemic failure; we shouldn't be guiding
development based on benchmarks that don't correlate positively with
user-visible improvement. If all of our benchmarks showing the benefits
of PGO are useless and there really isn't any difference between PGO and
non-PGO builds, then I'm not going to push for us to continue doing PGO
builds any more. But, in that case I hope we also come up with a plan
for making better benchmarks.
Nobody knows whether that's true or not for a fact. As mentioned
previously in this thread, the measurements that Vladan performed seem
to suggest that PGO builds can have a user visible effect in terms of
some performance characteristics we measure. Measurements performed by
dmandelin based on what appears on the screen shows that the difference
for a typical small work load is smaller than the resolution of the
measurement. These two can happen at the same time, there is no
contradiction between them.
> And, also, if PGO doesn't have a significant positive performance
difference, I would be very curious as to why not. Is PGO snake oil in
general? Is there something about our codebase that is
counter-productive to PGO? And, if the latter, then is there anything we
can do to undo that counter-productivity?
We don't know of anything going particularly bad in PGO builds of our
code base. If you look at the generated PGO code for most functions in
Gecko, you'll see that the compiler has heavily optimizied just about
everything that there is in the non-PGO optimized version of the same
function.
Cheers,
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
