Re: Proposal: Remove Linux PGO Testing
On 10/10/2012 11:57 PM, Justin Lebar wrote: > The main reason I'd want Linux PGO is for mobile. On desktop Linux, > most users (I expect) don't run our builds, so it's not a big deal if > they're some percent slower. (Unless distros commonly do PGO builds > of Firefox?) But we're not doing mobile Linux PGO builds (that I know > of), and I don't expect success with desktop PGO is much related to > success with mobile PGO. You may be right for release builds but that doesn't hold true for Nightly/Aurora/Beta users. I don't think it's a good idea to make those builds ~20% slower when of course we want and need more testers. Don't forget that testers on Linux do not only test Linux-only features but also features we have on every platform. Nobody likes running debug builds because they're slower so why would that be different for non-PGO builds? Also, I'm not sure how this affects Telemetry results. In terms of perf measurements we'd probably need to completely ignore everything from non-release builds as the results might differ heavily for some use cases. - Tim ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/12 3:05 AM, Tim Taubert wrote: Also, I'm not sure how this affects Telemetry results. In terms of perf measurements we'd probably need to completely ignore everything from non-release builds as the results might differ heavily for some use cases. I'm not following. The suggestion, as far as I can tell, is to drop Linux PGO completely. We woudln't have it in nightly, Aurora, Beta, or releases. Compiling with PGO on Linux would be an unsupported configuration that we'd probably advise distros against, because it wouldn't be particularly well-tested. So the real question is whether PGO on Linux is worth it in general to us, again as far as I can tell. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/2012 09:32 AM, Boris Zbarsky wrote: > The suggestion, as far as I can tell, is to drop Linux PGO completely. > We woudln't have it in nightly, Aurora, Beta, or releases. Compiling > with PGO on Linux would be an unsupported configuration that we'd > probably advise distros against, because it wouldn't be particularly > well-tested. Yes, if we don't distribute PGO builds at all we'd see a one-time bump and Telemetry is then a non-issue. I was misguided by the thread's title. If we turn it off for testing we can't ship it. - Tim ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
Right, exactly. I am arguing that testing PGO, which is a buggy optimization pass, incurs too much developer cost to justify a "5-20%" talos improvement on select benchmarks. On Linux, which is a very small percentage of our market share, and where distributions make their own builds anyway. Whether we'd tell distributions that PGO was unsupported: it actually seems difficult to say that it *is* supported, even now. PGO bugs will likely be highly dependent on the environment and compiler version, which are won't be exactly the same anywhere as they are on Windows. -David On Thursday, October 11, 2012 12:32:10 AM UTC-7, Boris Zbarsky wrote: > On 10/11/12 3:05 AM, Tim Taubert wrote: > > > Also, I'm not sure how this affects Telemetry results. In terms of perf > > > measurements we'd probably need to completely ignore everything from > > > non-release builds as the results might differ heavily for some use > > > cases. > > > > I'm not following. > > > > The suggestion, as far as I can tell, is to drop Linux PGO completely. > > We woudln't have it in nightly, Aurora, Beta, or releases. Compiling > > with PGO on Linux would be an unsupported configuration that we'd > > probably advise distros against, because it wouldn't be particularly > > well-tested. > > > > So the real question is whether PGO on Linux is worth it in general to > > us, again as far as I can tell. > > > > -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
Keep in mind that debug builds are probably at least an order of magnitude slower (or a large factor), whereas PGO is a very small factor. (After all, we do not PGO on Mac and it doesn't seem to be a problem.) -David On Thursday, October 11, 2012 12:05:35 AM UTC-7, Tim Taubert wrote: > On 10/10/2012 11:57 PM, Justin Lebar wrote: > > > The main reason I'd want Linux PGO is for mobile. On desktop Linux, > > > most users (I expect) don't run our builds, so it's not a big deal if > > > they're some percent slower. (Unless distros commonly do PGO builds > > > of Firefox?) But we're not doing mobile Linux PGO builds (that I know > > > of), and I don't expect success with desktop PGO is much related to > > > success with mobile PGO. > > > > You may be right for release builds but that doesn't hold true for > > Nightly/Aurora/Beta users. I don't think it's a good idea to make those > > builds ~20% slower when of course we want and need more testers. Don't > > forget that testers on Linux do not only test Linux-only features but > > also features we have on every platform. > > > > Nobody likes running debug builds because they're slower so why would > > that be different for non-PGO builds? > > > > Also, I'm not sure how this affects Telemetry results. In terms of perf > > measurements we'd probably need to completely ignore everything from > > non-release builds as the results might differ heavily for some use > > cases. > > > > - Tim ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
Tim Taubert wrote: Nobody likes running debug builds because they're slower I always run debug builds. What does that make me? ;-) -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
How to be notified when a node gets detached/reparented?
Context: in the firefox devtools, we need to track some nodes and update different "views" based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to "childList" mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a "childList" mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Thanks, -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Hi Paul, so this means we do not have anymore DOMnodeRemoved from the mutation events? I find your use case sort of important specially now that I believe pages will suffer more changes based in template operations in the client. So "detecting context" is key for client apps to know where they were and what to do next. A more specific case is a 4x4 grid abcd that loses quadrant a. It is pretty important to signal the new state, let's say "null,bcd" to consumers — for example a grid rearrangement may take place. m On Thu, Oct 11, 2012 at 8:40 AM, Paul Rouget wrote: > Context: in the firefox devtools, we need to track some nodes and update > different "views" based on what's happening to this node (show its parents, > show its child, show its attributes, …). > > The new Mutation observers are very helpful. But there's one thing I am not > really sure how to handle correctly . > > When a node gets detached (parent.removeChild(node)) or reparented, I need to > be notified. > > My current idea is to listen to "childList" mutations from the parent, > then, on this mutation, check if the node is still part of the children of > the parent, if not, check if it has a parent, if so, the node has been > *relocated*, then I need re-listen to a "childList" mutation from this > new parent, if no parent, the node has been *detached*. > > I was wondering if there was any better way to do that. > > Thanks, > > -- Paul > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform -- www.telasocial.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Marcio Galli wrote: > Hi Paul, so this means we do not have anymore DOMnodeRemoved from the > mutation events? There's no "DOMNodeRemoved" type: https://developer.mozilla.org/en-US/docs/DOM/MutationObserver#MutationObserverInit But there's a "removedNodes" from the mutation record. Maybe this array include the target if we listen to the subtree. I'll try. > > I find your use case sort of important specially now that I believe > pages will suffer more changes based in template operations in the > client. So "detecting context" is key for client apps to know where > they were and what to do next. A more specific case is a 4x4 grid abcd > that loses quadrant a. It is pretty important to signal the new state, > let's say "null,bcd" to consumers — for example a grid rearrangement > may take place. > > m > > On Thu, Oct 11, 2012 at 8:40 AM, Paul Rouget wrote: > > Context: in the firefox devtools, we need to track some nodes and update > > different "views" based on what's happening to this node (show its parents, > > show its child, show its attributes, …). > > > > The new Mutation observers are very helpful. But there's one thing I am not > > really sure how to handle correctly . > > > > When a node gets detached (parent.removeChild(node)) or reparented, I need > > to > > be notified. > > > > My current idea is to listen to "childList" mutations from the parent, > > then, on this mutation, check if the node is still part of the children of > > the parent, if not, check if it has a parent, if so, the node has been > > *relocated*, then I need re-listen to a "childList" mutation from this > > new parent, if no parent, the node has been *detached*. > > > > I was wondering if there was any better way to do that. > > > > Thanks, > > > > -- Paul > > ___ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > > > > -- > www.telasocial.com -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Paul Rouget wrote: > Marcio Galli wrote: > > Hi Paul, so this means we do not have anymore DOMnodeRemoved from the > > mutation events? > > There's no "DOMNodeRemoved" type: > https://developer.mozilla.org/en-US/docs/DOM/MutationObserver#MutationObserverInit > > But there's a "removedNodes" from the mutation record. Maybe this array > include > the target if we listen to the subtree. > > I'll try. Nope. > > I find your use case sort of important specially now that I believe > > pages will suffer more changes based in template operations in the > > client. So "detecting context" is key for client apps to know where > > they were and what to do next. A more specific case is a 4x4 grid abcd > > that loses quadrant a. It is pretty important to signal the new state, > > let's say "null,bcd" to consumers — for example a grid rearrangement > > may take place. > > > > m > > > > On Thu, Oct 11, 2012 at 8:40 AM, Paul Rouget wrote: > > > Context: in the firefox devtools, we need to track some nodes and update > > > different "views" based on what's happening to this node (show its > > > parents, > > > show its child, show its attributes, …). > > > > > > The new Mutation observers are very helpful. But there's one thing I am > > > not > > > really sure how to handle correctly . > > > > > > When a node gets detached (parent.removeChild(node)) or reparented, I > > > need to > > > be notified. > > > > > > My current idea is to listen to "childList" mutations from the parent, > > > then, on this mutation, check if the node is still part of the children of > > > the parent, if not, check if it has a parent, if so, the node has been > > > *relocated*, then I need re-listen to a "childList" mutation from this > > > new parent, if no parent, the node has been *detached*. > > > > > > I was wondering if there was any better way to do that. > > > > > > Thanks, > > > > > > -- Paul > > > ___ > > > dev-platform mailing list > > > dev-platform@lists.mozilla.org > > > https://lists.mozilla.org/listinfo/dev-platform > > > > > > > > -- > > www.telasocial.com > -- Paul > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Paul Rouget wrote: > Context: in the firefox devtools, we need to track some nodes and update > different "views" based on what's happening to this node (show its parents, > show its child, show its attributes, …). > > The new Mutation observers are very helpful. But there's one thing I am not > really sure how to handle correctly . > > When a node gets detached (parent.removeChild(node)) or reparented, I need to > be notified. > > My current idea is to listen to "childList" mutations from the parent, > then, on this mutation, check if the node is still part of the children of > the parent, if not, check if it has a parent, if so, the node has been > *relocated*, then I need re-listen to a "childList" mutation from this > new parent, if no parent, the node has been *detached*. > > I was wondering if there was any better way to do that. Another way to do it is to listen to subtree mutations from the documentElement, and then check if the targeted node is part of the removeNodes list. Would that deteriorate the performance? -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 11/10/12 08:54, David Anderson wrote: Keep in mind that debug builds are probably at least an order of magnitude slower (or a large factor), whereas PGO is a very small factor. (After all, we do not PGO on Mac and it doesn't seem to be a problem.) 5-20%, if it were a general slowdown, is _huge_. We have people who work for months to get speedups of 1 or 2%. We should find out whether the Linux distros are using PGO. If they are, it would be unwise for us to stop supporting their release configuration, and tell them that the new supported configuration is a lot slower... Gerv ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Xulrunner & UniversalXPConnect confusion
Hi Scott, Could you expand on your hack? I'm in a similar situation here :) Thanks! ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Xulrunner & UniversalXPConnect confusion
On Thu, Oct 11, 2012 at 11:53 AM, wrote:
> Hi Scott,
>
> Could you expand on your hack? I'm in a similar situation here :)
In the end we opted to get the filepath via a Java dialog but here's the gist:
In user.js (same folder as the prefs.js file) I added the following:lines:
user_pref("signed.applets.codebase_principal_support", true);
user_pref("capability.principal.codebase.p0.granted",
"UniversalXPConnect UniversalFileRead");
user_pref("capability.principal.codebase.p0.id", "");
user_pref("capability.principal.codebase.p0.subjectName", "");
Now when you read the value property of a file input it should return
the full path to the file selected by the user. Unfortunately this is
taken from some notes that I haven't looked at in a few months - if it
doesn't work, let me know and I'll try to whip up a quick sample.
Best,
--
Scott Elcomb
@psema4 on Twitter / Identi.ca / Github & more
Atomic OS: Self Contained Microsystems
http://code.google.com/p/atomos/
Member of the Pirate Party of Canada
http://www.pirateparty.ca/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Proposed W3C Charter: Web Fonts Working Group
W3C is proposing a revised charter for the Web Fonts Working Group. For more details, see: http://lists.w3.org/Archives/Public/public-new-work/2012Sep/0016.html http://www.w3.org/2012/06/WebFonts/draft-charter-ac.html Mozilla has the opportunity to send comments or objections through Monday, October 22. Please reply to this thread if you think there's something we should say. (I'd note that this charter doesn't have the typical bit about asynchronous decision making; however, our existing participants in the group may be ok with that.) -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla http://www.mozilla.org/ 𝄂 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Proposed W3C Charter: Pointer Events Working Group
W3C is proposing a charter for a new Pointer Events Working Group. For more details, see: http://lists.w3.org/Archives/Public/public-new-work/2012Sep/0017.html http://www.w3.org/2012/pointerevents/charter/charter-proposed.html Mozilla has the opportunity to send comments or objections through Thursday, October 25. Please reply to this thread if you think there's something we should say. -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla http://www.mozilla.org/ 𝄂 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
W3C Proposed Recommendation: Packaged Web Apps (Widgets) - Packaging and XML Configuration (Second Edition)
W3C recently published the following Proposed Edited Recommendation Packaged Web Apps (Widgets) - Packaging and XML Configuration (Second Edition) http://www.w3.org/TR/2012/PER-widgets-20120925/ If there are comments you think Mozilla should send as part of the review, or if you think Mozilla should voice support or opposition to the specification, please say so in this thread. (I'd note, however, that there have been many previous opportunities to make comments, so it's somewhat bad form to bring up fundamental issues for the first time at this stage.) The deadline for Mozilla to send comments is October 25. -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla http://www.mozilla.org/ 𝄂 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
> 5-20%, if it were a general slowdown, is _huge_. We have people who work > > for months to get speedups of 1 or 2%. Yes, I know, that is pretty much all I do at Mozilla ;) I don't think scattered Talos wins of 5-20% are so valuable and important that we should keep sacrificing developer time to debug problems from having the buggy optimization pass. > We should find out whether the Linux distros are using PGO. If they are, > > it would be unwise for us to stop supporting their release As I said, unless they're using the exact same compiler version and environment, we're already not supporting their configuration. C++ compilers are twitchy - much more so with PGO. -David On Thursday, October 11, 2012 7:46:00 AM UTC-7, Gervase Markham wrote: > On 11/10/12 08:54, David Anderson wrote: > > > Keep in mind that debug builds are probably at least an order of > > > magnitude slower (or a large factor), whereas PGO is a very small > > > factor. (After all, we do not PGO on Mac and it doesn't seem to be a > > > problem.) > > > > 5-20%, if it were a general slowdown, is _huge_. We have people who work > > for months to get speedups of 1 or 2%. > > > > We should find out whether the Linux distros are using PGO. If they are, > > it would be unwise for us to stop supporting their release > > configuration, and tell them that the new supported configuration is a > > lot slower... > > > > Gerv ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/2012 02:33 AM, Mike Hommey wrote: On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: By "turning off Linux PGO testing", you really mean "stop making and distributing Linux PGO builds," right? The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. Many people have made claims about that at several different occasions. Can we once and for all come up with actual data on that? That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Also note that disabling PGO currently also means disabling the optimizations we do on omni.ja (central directory optimizations and reordering). This is somehow covered by bug 773171. I wouldn't be surprised if most of the pgo benefit is because of bad inline decisions by gcc. If we can narrow the gap by adding MOZ_ALWAYS_INLINE, then maybe we can drop pgo. I did a talos run during the last clang update to compare it with the previous version on OS X, but I now added linux runs to compare gcc 4.5 and clang on linux. The results are interesting (see attached file). dromaeo_dom shows a 31% improvement on 64 bits for example. This also suggests another option: using clang on linux too. This would have the added benefit of using the same compiler for OS X and Linux, which would remove most of the argument of developers spending time on linux only issues. I can do a comparison of gcc+pgo and clang if others are interested. Mike Cheers, Rafael a11yr_paint Fedora 12 - Constantine | (527.681818182, 2.59651983057) -> (413.15, 1.41403960494) 1.2772x better Fedora 12 x64 - Constantine | (451.454545455, 1.24422992408) -> (349.45, 0.819984718342) 1.2919x better dromaeo_css Fedora 12 - Constantine | (2152.09181818, 12.0144869607) -> (2639.839, 8.95312637755) 1.2266x better Fedora 12 x64 - Constantine | (2485.29272727, 13.7659449067) -> (2921.445, 17.8096790459) 1.1755x better dromaeo_dom Fedora 12 - Constantine | (142.806181818, 0.744898237637) -> (187.9765, 0.650195498534) 1.3163x better Fedora 12 x64 - Constantine | (181.576818182, 0.806482009967) -> (232.0514, 1.34925457235) 1.2780x better kraken Fedora 12 - Constantine | (3939.40909091, 71.9270328223) -> (3767.41, 71.3917310166) 1.0457x better Fedora 12 x64 - Constantine | (3579.29090909, 15.596652341) -> (3446.56, 9.33275577468) 1.0385x better num_ctors CentOS (x86_64) release 5 (Final) | (232.0,None) -> (174.0, None) 1.x better CentOS release 5 (Final) | (232.0,None) -> (174.0, None) 1.x better sunspider Fedora 12 - Constantine | (364.172727273, 1.5583412188) -> (345.08, 1.26271089553) 1.0553x better Fedora 12 x64 - Constantine | (334.363636364, 2.07058055282) -> (321.97, 2.64172734312) 1.0385x better tdhtmlr_nochrome_paint Fedora 12 - Constantine | (417.558727273, 0.728348424302) -> (392.9735, 0.424360042529) 1.0626x better Fedora 12 x64 - Constantine | (390.3705, 0.633414181048) -> (359.1264, 6.1131889195) 1.0870x better tdhtmlr_paint Fedora 12 - Constantine | (433.884909091, 4.67646883558) -> (402.9236, 4.62643754814) 1.0768x better Fedora 12 x64 - Constantine | (398.0735, 3.51583823606) -> (368.2883, 1.94210482286) 1.0809x better tp5n_main_rss_paint Fedora 12 - Constantine | (114134400.0, 271577.523722) -> (115129700.0, 280881.107982) 1.0087x worse Fedora 12 x64 - Constantine | (149395900.0, 563431.271779) -> (148169900.0, 430915.422528) 1.0083x better tp5n_paint Fedora 12 - Constantine | (377.5355, 2.28065749844) -> (316.3663, 2.72038848381) 1.1933x better Fedora 12 x64 - Constantine | (325.3768, 3.31696113133) -> (278.6602, 3.33314780988) 1.1676x better tp5n_xres_paint Fedora 12 - Constantine | (7249195.0,114562.779547) -> (7640897.0,218688.473551) 1.0540x worse tpaint Fedora 12 - Constantine | (214.727272727, 2.47305127989) -> (187.4, 7.74235973322) 1.1458x better Fedora 12 x64 - Constantine | (203.545454545, 3.0539930444) -> (173.2, 4.81738641223) 1.1752x better tresize Fedora 12 - Constantine | (13.0813363636, 0.276579872514) -> (11.8753, 0.318387813206) 1.1016x better Fedora 12 x64 - Constantine | (12.0, 0.0)-> (11.0, 0.0)1.0909x better ts_paint Fedora 12 - Constantin
Re: Proposal: Remove Linux PGO Testing
I filed bug 800471 for considering using Clang on Linux. -Gary This also suggests another option: using clang on linux too. This would have the added benefit of using the same compiler for OS X and Linux, which would remove most of the argument of developers spending time on linux only issues. I can do a comparison of gcc+pgo and clang if others are interested. Cheers, Rafael ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
W3C Proposed Recommendation: WOFF File Format 1.0
W3C recently published the following Proposed Recommendation (the final stage in the W3C process before Recommendation) WOFF File Format 1.0 http://www.w3.org/TR/2012/PR-WOFF-20121011/ Mozilla's Jonathan Kew is one of the authors of this specification. If there are comments you think Mozilla should send as part of the review, or if you think Mozilla should voice support or opposition to the specification, please say so in this thread. Given Mozilla's involvement, I'd expect to express unqualified support for the specification's advancement. (I'd also note that there have been many previous opportunities to make comments, so it's somewhat bad form to bring up fundamental issues for the first time at this stage.) The deadline for Mozilla to send comments is November 8. -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla http://www.mozilla.org/ 𝄂 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 11/10/12 19:33, Mike Hommey wrote: > That being said, PGO on Linux is between 5 and 20% improvement on our > various talos tests. That's with the version of gcc we currently use, > which is 4.5. I'd expect 4.7 to do a better job even, especially if we > added lto to the equation (and since we are now building on x86-64 > machines, we wouldn't have to worry about memory usage ; link time could > be a problem, though). Perhaps the problems can be resolved or ameliorated by bumping the minimum version of GCC that we support for PGO. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Imported code
In Bug 794510, ehsan said in response to me:
>> Isaac makes a good point; we should clearly mark imported code, both for our
>> own purposes and for scripts. Biesi and I were commiserating about the lack
>> of
>> a standard for this ("third_party/blah" such as netwerk/third_party/sctp
>> instead of netwerk/sctp/src might be one way to do it, or put them ALL in a
>> top-level third-party directory, or add a file in the root of a third-party
>> directory (IMPORTED, with info on where/how/etc), ...)
>
>Well, marking imported code is definitely helpful, but really we should
>have a clear process on how to modify the imported code. It is entirely
>unreasonable to render ourselves unable to modify our imported code
>(just look at the current situation with NSPR which causes developers to
>go through huge pain in order to work around things which would be very
>simply dealt with if only we had the option of fixing NSPR...). We
>currently do a very good job in some of the projects (see angle for
>example: http://mxr.mozilla.org/mozilla-central/source/gfx/angle/) and
>in some other cases we do an extremely poor job (case in point:
>nsprpub/!). Really, whoever maintains a given imported code should come
>up with a clear process on how to take patches to that code and either
>try to upstream it if it makes sense or maintain a local patch to assist
>updates to the imported code in the future.
Right. I've tried to create automatic import scripts for all the
libraries I've imported as part of webrtc; some more complex
(webrtc/trunk, where we expect to be upstreaming a lot of stuff and
editing out a lot from the import) and others that basically import
updates from the source repo and require reapplying local updates (which
I maintained as independent patches when we need them); this is mostly
for libraries that we don't expect/want to modify, and if there are
changes needed we ask for updates upstream (netwerk/sctp/src for
example, and netwerk/srtp/src; see netwerk/srtp/update_srtp.sh).
Standardizing how patches are applied on top of imports would be good,
and also how to handle patches slated for upstreaming (that will
hopefully be backed out when upstream updates).
A tarball of patches in the directory is one way I believe is used
(though very un-source-control-management style). Most do updates
entirely by hand, which makes updates extrodinarily painful and
infrequent (or never).
We could also keep a list of bugs to apply patches from (somewhat
better), but still a very manual process.
I built a more complex process for media/webrtc/trunk, which I've had to
only partly use on alder and not m-c, because of the size of the
resultant imported changeset history. I plan to revise this process to
resolve that issue, but the fundamental way it works (in
media/webrtc/webrtc_update.sh) is to:
1) import a complete update (using hg addremove --similarity XX to catch
renames)
2) merge it to another head which has pending upstream patches
3) merge *that* to another head which has deletions not intended for
upstream to winnow it down to what we want in our tree. In webrtc,
this involves removing large number of third-party modules, remove
40MB video YUV test files, etc.
4) take that, and merge it to a mozilla repo (alder) which merges any local
(non-upstream) mods to our tree with the imported update.
I've used this to produce what I want on alder, and have released stuff
to m-c by copying the code over and running hg addremove, instead of
using hg pull from alder to m-c, because the imported patch history
would enormously expand the repo size. For smaller projects this would
not be an issue. This may also be overkill for smaller projects, however.
This itself may be overly complex, especially in the separation of
patches-for-upstream from our normal dev tree. When I designed it, I
didn't know how many upstream patches I'd be producing.
--
Randell Jesup, Mozilla Corp
remove ".news" for personal email
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
On Thu, Oct 11, 2012 at 6:04 AM, Paul Rouget wrote: > Paul Rouget wrote: >> Context: in the firefox devtools, we need to track some nodes and update >> different "views" based on what's happening to this node (show its parents, >> show its child, show its attributes, …). >> >> The new Mutation observers are very helpful. But there's one thing I am not >> really sure how to handle correctly . >> >> When a node gets detached (parent.removeChild(node)) or reparented, I need to >> be notified. >> >> My current idea is to listen to "childList" mutations from the parent, >> then, on this mutation, check if the node is still part of the children of >> the parent, if not, check if it has a parent, if so, the node has been >> *relocated*, then I need re-listen to a "childList" mutation from this >> new parent, if no parent, the node has been *detached*. >> >> I was wondering if there was any better way to do that. > > Another way to do it is to listen to subtree mutations from the > documentElement, > and then check if the targeted node is part of the removeNodes list. You also would have to check if any of the the node's ancestors is part of the removeNodes list. > Would that deteriorate the performance? That is obviously more work that has to be done both by the platform and by the webpage, so yes, it's worse performance. How much work I couldn't say offhand. It would be worth bringing this use-case to the webapps WG where MutationObservers are defined. Especially getting notifications when a node is moved in and out of a document seems like it would be worth having explicit notifications about. / Jonas ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
@Paul, What is your use case BTW? when you say "update views" based on mutations, is the goal is to let the user know what is going on? Or you actually performing other mutations back to the DOM or logging things or creating reports? m On Thu, Oct 11, 2012 at 4:27 PM, Jonas Sicking wrote: > On Thu, Oct 11, 2012 at 6:04 AM, Paul Rouget wrote: > > Paul Rouget wrote: > >> Context: in the firefox devtools, we need to track some nodes and update > >> different "views" based on what's happening to this node (show its > parents, > >> show its child, show its attributes, …). > >> > >> The new Mutation observers are very helpful. But there's one thing I am > not > >> really sure how to handle correctly . > >> > >> When a node gets detached (parent.removeChild(node)) or reparented, I > need to > >> be notified. > >> > >> My current idea is to listen to "childList" mutations from the parent, > >> then, on this mutation, check if the node is still part of the children > of > >> the parent, if not, check if it has a parent, if so, the node has been > >> *relocated*, then I need re-listen to a "childList" mutation from this > >> new parent, if no parent, the node has been *detached*. > >> > >> I was wondering if there was any better way to do that. > > > > Another way to do it is to listen to subtree mutations from the > documentElement, > > and then check if the targeted node is part of the removeNodes list. > > You also would have to check if any of the the node's ancestors is > part of the removeNodes list. > > > Would that deteriorate the performance? > > That is obviously more work that has to be done both by the platform > and by the webpage, so yes, it's worse performance. How much work I > couldn't say offhand. > > It would be worth bringing this use-case to the webapps WG where > MutationObservers are defined. Especially getting notifications when a > node is moved in and out of a document seems like it would be worth > having explicit notifications about. > > / Jonas > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > -- www.telasocial.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 2012-10-11 3:12 PM, Anthony Jones wrote: On 11/10/12 19:33, Mike Hommey wrote: That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Perhaps the problems can be resolved or ameliorated by bumping the minimum version of GCC that we support for PGO. Link-time optimization is described as an experimental new feature in the GCC 4.5.0 release notes[1]. The 4.6.0 release notes[2] say that it has now "stabilized to the point of being usable", and the 4.7.0 release notes[3] describe it as still further improved both in reliability and code quality. If we're trying to use the 4.5 LTO, I'm not at all surprised to hear it's causing more trouble than it's worth. PGO is not the same thing as LTO, of course, but GCC's PGO was kind of an unloved stepchild until they got serious about LTO, so that, too, is likely to be much improved in 4.7. zw [1] http://gcc.gnu.org/gcc-4.5/changes.html [2] http://gcc.gnu.org/gcc-4.6/changes.html [3] http://gcc.gnu.org/gcc-4.7/changes.html ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On 2012-10-11 3:16 PM, Randell Jesup wrote:
In Bug 794510, ehsan said in response to me:
Isaac makes a good point; we should clearly mark imported code, both for our
own purposes and for scripts. Biesi and I were commiserating about the lack of
a standard for this ("third_party/blah" such as netwerk/third_party/sctp
instead of netwerk/sctp/src might be one way to do it, or put them ALL in a
top-level third-party directory, or add a file in the root of a third-party
directory (IMPORTED, with info on where/how/etc), ...)
Well, marking imported code is definitely helpful, but really we should
have a clear process on how to modify the imported code. It is entirely
unreasonable to render ourselves unable to modify our imported code
(just look at the current situation with NSPR which causes developers to
go through huge pain in order to work around things which would be very
simply dealt with if only we had the option of fixing NSPR...). We
currently do a very good job in some of the projects (see angle for
example: http://mxr.mozilla.org/mozilla-central/source/gfx/angle/) and
in some other cases we do an extremely poor job (case in point:
nsprpub/!). Really, whoever maintains a given imported code should come
up with a clear process on how to take patches to that code and either
try to upstream it if it makes sense or maintain a local patch to assist
updates to the imported code in the future.
Right. I've tried to create automatic import scripts for all the
libraries I've imported as part of webrtc; some more complex
(webrtc/trunk, where we expect to be upstreaming a lot of stuff and
editing out a lot from the import) and others that basically import
updates from the source repo and require reapplying local updates (which
I maintained as independent patches when we need them); this is mostly
for libraries that we don't expect/want to modify, and if there are
changes needed we ask for updates upstream (netwerk/sctp/src for
example, and netwerk/srtp/src; see netwerk/srtp/update_srtp.sh).
Standardizing how patches are applied on top of imports would be good,
and also how to handle patches slated for upstreaming (that will
hopefully be backed out when upstream updates).
A tarball of patches in the directory is one way I believe is used
(though very un-source-control-management style). Most do updates
entirely by hand, which makes updates extrodinarily painful and
infrequent (or never).
I think owners of imported code should be responsible for updating it,
and it is nice of them if they include update scripts (it makes it
easier for them to do future updates, and it decreases the bus factor),
but I don't know how much we can enforce that. Other projects such as
WebRTC which have more complicated workflows can pick whatever works
best for them.
However, keeping the history of the changes over upstream code is
another issue. Really, we should not need to do anything special there
since the history of modifications is already recorded in the revision
control systems, but some people like to store them as patches etc,
probably because of CVS-nostalgia (you didn't use to have the entire
history of the repo on your hard disk!).
What I really don't want us to do is to prohibit people from fixing
things in the imported code. That is the absolute worst situation we
can face with a given piece of code, as we already have learned painfully.
Cheers,
Ehsan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On Thu, Oct 11, 2012 at 02:26:33PM -0400, Rafael Ávila de Espíndola wrote: > On 10/11/2012 02:33 AM, Mike Hommey wrote: > >On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: > >>By "turning off Linux PGO testing", you really mean "stop making and > >>distributing Linux PGO builds," right? > >> > >>The main reason I'd want Linux PGO is for mobile. On desktop Linux, > >>most users (I expect) don't run our builds, so it's not a big deal if > >>they're some percent slower. > > > >Many people have made claims about that at several different occasions. > >Can we once and for all come up with actual data on that? > > > >That being said, PGO on Linux is between 5 and 20% improvement on our > >various talos tests. That's with the version of gcc we currently use, > >which is 4.5. I'd expect 4.7 to do a better job even, especially if we > >added lto to the equation (and since we are now building on x86-64 > >machines, we wouldn't have to worry about memory usage ; link time could > >be a problem, though). > > > >Also note that disabling PGO currently also means disabling the > >optimizations we do on omni.ja (central directory optimizations and > >reordering). This is somehow covered by bug 773171. > > I wouldn't be surprised if most of the pgo benefit is because of bad > inline decisions by gcc. If we can narrow the gap by adding > MOZ_ALWAYS_INLINE, then maybe we can drop pgo. A non-unsignificant part of the performance improvements PGO gives come from code reordering to improve branch prediction. Presumably, we can use NS_LIKELY/NS_UNLIKELY to improve some branches manually. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
These Dromaeo improvements will in part be because IonMonkey is not fully JIT'ing these paths yet (a regression we're tracking from Firefox 17). -David On Thursday, October 11, 2012 11:26:49 AM UTC-7, Rafael Ávila de Espíndola wrote: > On 10/11/2012 02:33 AM, Mike Hommey wrote: > > > On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: > > >> By "turning off Linux PGO testing", you really mean "stop making and > > >> distributing Linux PGO builds," right? > > >> > > >> The main reason I'd want Linux PGO is for mobile. On desktop Linux, > > >> most users (I expect) don't run our builds, so it's not a big deal if > > >> they're some percent slower. > > > > > > Many people have made claims about that at several different occasions. > > > Can we once and for all come up with actual data on that? > > > > > > That being said, PGO on Linux is between 5 and 20% improvement on our > > > various talos tests. That's with the version of gcc we currently use, > > > which is 4.5. I'd expect 4.7 to do a better job even, especially if we > > > added lto to the equation (and since we are now building on x86-64 > > > machines, we wouldn't have to worry about memory usage ; link time could > > > be a problem, though). > > > > > > Also note that disabling PGO currently also means disabling the > > > optimizations we do on omni.ja (central directory optimizations and > > > reordering). This is somehow covered by bug 773171. > > > > I wouldn't be surprised if most of the pgo benefit is because of bad > > inline decisions by gcc. If we can narrow the gap by adding > > MOZ_ALWAYS_INLINE, then maybe we can drop pgo. > > > > I did a talos run during the last clang update to compare it with the > > previous version on OS X, but I now added linux runs to compare gcc 4.5 > > and clang on linux. The results are interesting (see attached file). > > dromaeo_dom shows a 31% improvement on 64 bits for example. > > > > This also suggests another option: using clang on linux too. This would > > have the added benefit of using the same compiler for OS X and Linux, > > which would remove most of the argument of developers spending time on > > linux only issues. I can do a comparison of gcc+pgo and clang if others > > are interested. > > > > > Mike > > > > Cheers, > > Rafael > > > > > > a11yr_paint > > Fedora 12 - Constantine | (527.681818182, 2.59651983057) -> > (413.15, 1.41403960494) 1.2772x better > > Fedora 12 x64 - Constantine | (451.454545455, 1.24422992408) -> > (349.45, 0.819984718342) 1.2919x better > > > > dromaeo_css > > Fedora 12 - Constantine | (2152.09181818, 12.0144869607) -> > (2639.839, 8.95312637755) 1.2266x better > > Fedora 12 x64 - Constantine | (2485.29272727, 13.7659449067) -> > (2921.445, 17.8096790459) 1.1755x better > > > > dromaeo_dom > > Fedora 12 - Constantine | (142.806181818, 0.744898237637) -> > (187.9765, 0.650195498534) 1.3163x better > > Fedora 12 x64 - Constantine | (181.576818182, 0.806482009967) -> > (232.0514, 1.34925457235) 1.2780x better > > > > kraken > > Fedora 12 - Constantine | (3939.40909091, 71.9270328223) -> > (3767.41, 71.3917310166) 1.0457x better > > Fedora 12 x64 - Constantine | (3579.29090909, 15.596652341) -> > (3446.56, 9.33275577468) 1.0385x better > > > > num_ctors > > CentOS (x86_64) release 5 (Final) | (232.0,None) -> (174.0, >None) 1.x better > > CentOS release 5 (Final) | (232.0,None) -> (174.0, >None) 1.x better > > > > sunspider > > Fedora 12 - Constantine | (364.172727273, 1.5583412188) -> > (345.08, 1.26271089553) 1.0553x better > > Fedora 12 x64 - Constantine | (334.363636364, 2.07058055282) -> > (321.97, 2.64172734312) 1.0385x better > > > > tdhtmlr_nochrome_paint > > Fedora 12 - Constantine | (417.558727273, 0.728348424302) -> > (392.9735, 0.424360042529) 1.0626x better > > Fedora 12 x64 - Constantine | (390.3705, 0.633414181048) -> > (359.1264, 6.1131889195) 1.0870x better > > > > tdhtmlr_paint > > Fedora 12 - Constantine | (433.884909091, 4.67646883558) -> > (402.9236, 4.62643754814) 1.0768x better > > Fedora 12 x64 - Constantine | (398.0735, 3.51583823606) -> > (368.2883, 1.94210482286) 1.0809x better > > > > tp5n_main_rss_paint > > Fedora 12 - Constantine | (114134400.0, 271577.523722) -> > (115129700.0, 280881.107982) 1.0087x worse > > Fedora 12 x64 - Constantine | (149395900.0, 563431.271779) -> > (148169900.0, 430915.422528) 1.0083x better > > > > tp5n_paint > > Fedora 12 - Constantine | (377.5355, 2.28065749844) -> > (316.3663, 2.72038848381) 1.1933x better > > Fedora 12 x64 - Constantine | (325.37
Re: Proposal: Remove Linux PGO Testing
On Wednesday, October 10, 2012 11:33:31 PM UTC-7, Mike Hommey wrote: > That being said, PGO on Linux is between 5 and 20% improvement on our > various talos tests. That's with the version of gcc we currently use, > which is 4.5. I'd expect 4.7 to do a better job even, especially if we > added lto to the equation (and since we are now building on x86-64 > machines, we wouldn't have to worry about memory usage ; link time could > be a problem, though). Do we have detailed data? I don't know how to interpret '5-20% on various Talos tests' without context. If it's a 10% improvement on something that takes 10ms per pageload, then I don't think it matters. If it cuts 100ms from a whole pageload, then it starts to sound like it matters. I'd really like to know not only so we can make a good decision now about whether to use PGO, but also so that we can understand why we made the decision later on. PGO bugs cause crashes for our users and are miserable to debug and can feel like a waste of time to developers. I'd like to have a clear, documented understanding of either why we don't need PGO, or why PGO is worth suffering for. Dave ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari wrote: > What I really don't want us to do is to prohibit people from fixing things > in the imported code. That is the absolute worst situation we can face with > a given piece of code, as we already have learned painfully. This should absolutely be at the discretion of the maintainer of the imported code in our tree. From personal experience, allowing local changes to land in Breakpad before they are landed upstream has caused huge headaches. Our in-tree copy of Breakpad was nearly 2 years out-of-date because of a few large patches that landed in support of OOP work and were difficult to upstream. -Ted ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On Thu, Oct 11, 2012 at 06:14:51PM -0400, Ted Mielczarek wrote: > On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari > wrote: > > What I really don't want us to do is to prohibit people from fixing things > > in the imported code. That is the absolute worst situation we can face with > > a given piece of code, as we already have learned painfully. > > This should absolutely be at the discretion of the maintainer of the > imported code in our tree. From personal experience, allowing local > changes to land in Breakpad before they are landed upstream has caused > huge headaches. Our in-tree copy of Breakpad was nearly 2 years > out-of-date because of a few large patches that landed in support of > OOP work and were difficult to upstream. Same experience with jemalloc, which is why the rule is to have things applied upstream first. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On 2012-10-11 6:36 PM, Mike Hommey wrote: On Thu, Oct 11, 2012 at 06:14:51PM -0400, Ted Mielczarek wrote: On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari wrote: What I really don't want us to do is to prohibit people from fixing things in the imported code. That is the absolute worst situation we can face with a given piece of code, as we already have learned painfully. This should absolutely be at the discretion of the maintainer of the imported code in our tree. From personal experience, allowing local changes to land in Breakpad before they are landed upstream has caused huge headaches. Our in-tree copy of Breakpad was nearly 2 years out-of-date because of a few large patches that landed in support of OOP work and were difficult to upstream. Same experience with jemalloc, which is why the rule is to have things applied upstream first. What is the nature of this problem? Is it the difficulty to reapply the patches when pulling new code from upstream? Note that I'm mostly talking about fixing small problems in our copy, not doing major architectural changes. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On Thu, Oct 11, 2012 at 06:38:57PM -0400, Ehsan Akhgari wrote: > On 2012-10-11 6:36 PM, Mike Hommey wrote: > >On Thu, Oct 11, 2012 at 06:14:51PM -0400, Ted Mielczarek wrote: > >>On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari > >>wrote: > >>>What I really don't want us to do is to prohibit people from fixing things > >>>in the imported code. That is the absolute worst situation we can face > >>>with > >>>a given piece of code, as we already have learned painfully. > >> > >>This should absolutely be at the discretion of the maintainer of the > >>imported code in our tree. From personal experience, allowing local > >>changes to land in Breakpad before they are landed upstream has caused > >>huge headaches. Our in-tree copy of Breakpad was nearly 2 years > >>out-of-date because of a few large patches that landed in support of > >>OOP work and were difficult to upstream. > > > >Same experience with jemalloc, which is why the rule is to have things > >applied upstream first. > > What is the nature of this problem? Is it the difficulty to reapply > the patches when pulling new code from upstream? Note that I'm > mostly talking about fixing small problems in our copy, not doing > major architectural changes. Experience shows that adding patches to our copies of third party libraries lead to, at least: - long-time forks - undocumented patches (no corresponding patch file in the tree to reapply = fun to handle upgrades, or broken upgrades if unnoticed) - outdated patches (a .patch exists but doesn't correspond to what is in the tree = most likely broken upgrades) - unapplied patches after an upgrade (= broken upgrade) I have seen multiple iterations of each of the above. This is not a theoretical problem. And some even happened with things that are much less third-party than most third-party code we have in the tree: nspr and nss. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 2012-10-11 4:34 PM, Mike Hommey wrote: On Thu, Oct 11, 2012 at 02:26:33PM -0400, Rafael Ávila de Espíndola wrote: On 10/11/2012 02:33 AM, Mike Hommey wrote: On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: By "turning off Linux PGO testing", you really mean "stop making and distributing Linux PGO builds," right? The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. Many people have made claims about that at several different occasions. Can we once and for all come up with actual data on that? That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Also note that disabling PGO currently also means disabling the optimizations we do on omni.ja (central directory optimizations and reordering). This is somehow covered by bug 773171. I wouldn't be surprised if most of the pgo benefit is because of bad inline decisions by gcc. If we can narrow the gap by adding MOZ_ALWAYS_INLINE, then maybe we can drop pgo. A non-unsignificant part of the performance improvements PGO gives come from code reordering to improve branch prediction. Presumably, we can use NS_LIKELY/NS_UNLIKELY to improve some branches manually. Don't both of these proposals map to tons of manual work? I'm not convinced that doing either of those would necessarily be easier than finding and fixing the PGO bug at hand. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
On 10/11/2012 02:40 PM, Paul Rouget wrote: Context: in the firefox devtools, we need to track some nodes and update different "views" based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to "childList" mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a "childList" mutation from this new parent, if no parent, the node has been *detached*. Why do you need to re-listen anywhere? You get the node in a MutationRecord and when the callback is called you check where it is. ( node.contains can be useful and certainly faster than anything in JS. ) If the node doesn't have parent, it is detached. I was wondering if there was any better way to do that. Thanks, -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Minimum Required Python Version
The general consensus seems to be "2.7 is good," so I filed https://bugzilla.mozilla.org/show_bug.cgi?id=800614 to have configure enforce Python 2.6 as the minimum required to *build* the tree. Note that building is different from running tests (some test runners still run on Python 2.5 and Talos is on Python 2.4). We'll figure out the bump to 2.7 once we see how well 2.6 goes. If anyone has new objections, please state them on the bug. On 9/9/12 12:54 PM, Gregory Szorc wrote: The subject of which version of Python to require to build the tree came up in bug 784841. We currently require Python >= 2.5 but <3 to build the tree. The main reason for the 2.5 requirement is the Linux build slaves still run Python 2.5. Those of us who code Python for the tree have long wanted to require at least 2.6 because 2.5 is missing many desired features. And, since 2.6+ is ubiquitous these days, people (sometimes unknowingly) target it (because it's what's installed locally) and then have to go through trouble (or tree breakage) to backport compatibility to 2.5. Personally, I feel that targeting 2.5 is extremely painful (especially once you've used 2.6+) that I sometimes get discouraged from landing new features to the build system or test suites because I don't want to deal with 2.5 compatibility. I'm pretty sure that no reasonably sized faction will have complaints about bumping up the minimum version to 2.6. So, the question becomes whether we should jump all the way to 2.7. I believe we should. Taking the long view, we will eventually need to switch to Python 3. Our migration to Python 3 will likely involve porting all the code to simultaneously run on both 2.x and 3.x. Python 2.7 has more backported features from Python 3 than Python 2.6, so ensuring dual compatibility while employing useful and convenient newer features [1] should be easier with 2.7. Shorter term, 2.7 is the superior Python release. There are literally dozens of bug fixes and minor newer features. Individually, these don't seem like much. Cumulatively, they represent a lot of saved time and pain. Objections to requiring 2.7 will likely be about it not being installed everywhere out of the box. Let's examine that. MozillaBuild has shipped with Python 2.7 since November 2011. So, Windows is taken care of. OS X 10.7+ ship with Python 2.7. No action necessary. OS X 10.6 ships with 2.6. However, 2.7 is easy to install through Homebrew, MacPorts, or an official installer available through python.org. I believe the same is true for 10.5. I don't consider this to be a hurdle on OS X, especially since we already require similar steps for other required packages there. Linux distros are all over the map. Many include 2.7 as part of the standard distribution. If they don't, they often include a "python27" package. Or, at least it is a popular enough package that someone on the internets provides an RPM, .deb, etc. We would just need to point people at those in the build instructions on MDN. In the worst case, you will need to compile Python from source. This is literally |./configure && make && make install|. Not difficult if you ask me. Now, for those who need them (and that number goes down with time as 2.7 becomes more prevalent than 2.6), these will be extra steps. And, every extra step makes getting started for first-time developers a little harder. In the grand scheme of all the steps required to build the tree today, I don't think it's such a big deal. Besides, work is currently being done to enable one-line system bootstrap to help people initially configure their systems [2]. Once landed, concerns about setting up systems to build the tree should be rendered irrelevant for supported platforms. Some may say "why not go all the way and require Python 3?" Well, "require" is a strong word. In my opinion we need to "support" it first. This is because we almost certainly want to avoid a flag day conversion because it would be a huge headache for releng and everyone else. This means a period where we simultaneously support 2.x and 3.x. Once we have dual support, then we can talk about requiring 3.x. So, 2.6 or 2.7? [1] http://docs.python.org/release/2.7.3/whatsnew/2.7.html [2] https://bugzilla.mozilla.org/show_bug.cgi?id=774112 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/2012 03:49 PM, Ehsan Akhgari wrote: Don't both of these proposals map to tons of manual work? I'm not convinced that doing either of those would necessarily be easier than finding and fixing the PGO bug at hand. The problem is that fixing this one bug might take only a few days, but that underestimates the true cost of PGO. The fact is that, as long as we have it turned on, PGO will keep introducing new bugs. Each new line of code that we write gives PGO an opportunity to miscompile it. This represents an unbounded amount of work for us. This can be compared to the benefits PGO provides, which are growing very slowly, if at all (about 2% per year [1]). Additionally, I don't think we have a good idea of how many bugs are caused by GCC PGO. Windows PGO issues often turn into topcrashes. On Linux, we may not have enough users for these bugs to bubble up far enough for us to investigate them. -Bill [1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.434 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Why we avoid making private modifications to NSPR and NSS (was Re: Imported code)
Randell quoted: > Ehsan wrote: > >It is entirely unreasonable to render ourselves unable to modify > >our imported code (just look at the current situation with NSPR > >which causes developers to go through huge pain in order to work > > around things which would be very simply dealt with if only we > > had the option of fixing NSPR...). First, I think that a lot of Mozillian's concerns about how we can(not) change NSPR and NSS are based on old events and circumstances that do not apply now. For example, the time where NSS 3.whatever was being FIPS validated seemed to cause a lot of unfortunate misunderstandings all around. But, that was literally years ago. I encourage people to be more optimistic about things NSPR and NSS related. And, please keep in mind that there is already progress being made (if not a definitive agreement) on the Mercurial vs. CVS issue. There is a very practical reason for avoiding making private changes to NSPR and NSS. The most obvious reason is that it makes it esay to merge changes between upstream and our codebase. However, the more critical reason is that we support --use-system-nss and --use-system-nspr, and some Linux developers build their Firefox packages with those options. As long as we support --use-system-nss and --use-system-nspr, we need to make sure the upstream NSPR and NSS contain every bugfix and every feature that we require. If we were to give up on the idea of supporting --use-system-nspr and --use-system-nss, then we could gain more flexibility in how we change NSPR and/or NSS in mozilla-central. However, at least in theory, --use-system-nss and --use-system-nspr should be superior performance-wise, on Linux, because usually the system NSPR and NSS libraries are already loaded before Firefox starts. Thus, our startup time should be lower. Also, according to some conversation on dev-tech-crypto, the system NSS may eventually provide better platform integration regarding centralized certificate and smartcard handling, allowing us to share various security-related features with other applications (between Firefox and Thunderbird, or Firefox and whatever-the-Gnome-email-app-is). So, I think there are still advantages to supporting system NSS. In the event that we really do need to make private changes to NSPR and NSS, we should be able to do so. I think there's been a lot of unnecessary misunderstanding about this. If you *need* a change made to NSPR or NSS before we're ready to make a NSPR or NSS release, please make sure that the NSPR and NSS teams are aware of that, so we can help. And, whenever possible, try to avoid creating emergency situations. I have found everybody to be quite reasonable about it, especially when my request didn't involve me needing them to drop their work to do a code review for me. Usually we upstream those changes into NSS and NSPR first, because we need NSPR and NSS peers to do the review anyway. We try to avoid making "temporary" fixes only in mozilla-central, because, well, what happens if they don't get accepted upstream? Then we've broken --use-system-nspr and --use-system-nss. Still, I think there is more flexibility here than people realize. In general, it is harder to get changes made to NSS and NSPR than it is to get changes made to the rest of Gecko. One reason is that the reviewing standards are different/stricter in these modules than they are in some/many (but not all) Gecko modules. I actually prefer the stricter NSPR/NSS reviews and I hope that doesn't change. Another reason is that, generally mozilla-central is primarily geared towards Mozilla products (especially Firefox), whereas NSPR and NSS are shared between us, Chrome, and all Linux distributions. NSPR and NSS are part of the Linux Standard Base, which means that it is difficult to make compatibility-breaking changes to them. Sometimes when people suggest changes to NSPR and/or NSS, it isn't clear how urgent that change is. Definitely, I have sat on a review for too long because I didn't realize it was actually as urgent as other work I am doing. Because many of the NSPR and NSS peers do not work for Mozilla, and do not work on Mozilla stuff full-time, it definitely isn't as obvious to them what is a high-priority request and what isn't. And, also, because they have their own jobs and their own schedules, sometimes schedules are not aligned as well as we would like/need them to be. IMO, the solution to that is to have more MoCo employees and other Mozillians become peers on the NSPR and NSS projects, so that we can help with the code reviews in these projects. I know on the NSS part, we're definitely trying to make progress in getting more Mozilla people involved. For NSPR, my understanding is that we're generally migrating away from using NSPR in Gecko, except for networking. Lots of stuff that's in NSPR already has replacements in mfbt and/or in ISO C/C++. One reason for doing this is that we can
Re: Proposal: Remove Linux PGO Testing
Zack Weinberg wrote: > Link-time optimization is described as an experimental new feature in > the GCC 4.5.0 release notes[1]. The 4.6.0 release notes[2] say that > it has now "stabilized to the point of being usable", and the 4.7.0 > release notes[3] describe it as still further improved both in > reliability and code quality. If we're trying to use the 4.5 LTO, > I'm not at all surprised to hear it's causing more trouble than it's > worth. > > PGO is not the same thing as LTO, of course, but GCC's PGO was kind > of an unloved stepchild until they got serious about LTO, so that, > too, is likely to be much improved in 4.7. I think it is important to give Linux users the fastest browser we can give them, because: 1. Linux users tend to be disproportionately influential in the markets we care the most about (web developers, techies) 2. Linux is the foundation of B2G and Firefox for Android, where we *definitely* must deliver the fastest product we can Now, if it were up to me, I'd try to reproduce this on a build built with the latest stable GCC or latest stable clang, and if that fixes the issue, I'd consider this a big motivation for upgrading to GCC 4.5 to a better compiler, which we need to do anyway for language feature support. Definitely, I don't think we should be adding hacks to our code to work around GCC problems that are already fixed in later releases of GCC. It's better to just make the build fail when the user attempts to use one of those older GCC releases. Now, if PGO doesn't result in the fastest browser, then of course we should disable PGO. Or, if there is no better compiler possible, then yes, I think it makes sense to disable PGO temporarily until there is a better compiler available. (And/or, help fix the compiler, either by contributing a patch, or by commissioning somebody else to contribute a patch.) Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Why we avoid making private modifications to NSPR and NSS (was Re: Imported code)
Ehsan wrote: > It is entirely unreasonable to render ourselves unable to modify > our imported code (just look at the current situation with NSPR > which causes developers to go through huge pain in order to work > around things which would be very simply dealt with if only we > had the option of fixing NSPR...). I don't know what the current situation with NSPR is. I guess I am not reviewing patches fast enough. That's certainly my fault. As for the option of fixing NSPR: you have that option. NSPR public functions need to stay backward compatible. This means the prototypes of public functions and the definitions of public types cannot change. (There are exceptions, if done carefully.) Bugs in function behavior can certainly be fixed. Wan-Teh ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposed W3C Charter: Pointer Events Working Group
On 10/11/2012 07:55 PM, L. David Baron wrote: W3C is proposing a charter for a new Pointer Events Working Group. For more details, see: http://lists.w3.org/Archives/Public/public-new-work/2012Sep/0017.html http://www.w3.org/2012/pointerevents/charter/charter-proposed.html Mozilla has the opportunity to send comments or objections through Thursday, October 25. Please reply to this thread if you think there's something we should say. -David We should join PEWG. Nicer API than touch API. The spec needs some work but is a good approach. -Olli ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
PRtypes [was Re: Why we avoid making private modifications to NSPR and NSS]
On 10/11/2012 7:52 PM, Wan-Teh Chang wrote: NSPR public functions need to stay backward compatible. This means the prototypes of public functions and the definitions of public types cannot change. (There are exceptions, if done carefully.) Bugs in function behavior can certainly be fixed. Wan-Teh I think the discussion that's currently going around the block with respect to NSPR is the entire PR types issue, particularly the fact that PRUint64 != uint64_t on some platforms (notably BSD). I do realize that changing this in NSPR carries backwards-compatibility issues that are important to manage, but I wonder if there is a way to let users of NSPR choose whether or not to have stdint-compliant definitions of PR types instead of the backwards-compatible variants. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
> 2. Linux is the foundation of B2G and Firefox for Android, where we > *definitely* must deliver > the fastest product we can I totally agree, but it's not clear to me whether continuing to do PGO on desktop Linux has any effect on our ability to potentially do PGO on Android/B2G. If we were to stop doing PGO on desktop Linux tomorrow, would that make it much harder to do PGO on mobile in the future? Not to beg the question as to whether we want to do PGO on desktop Linux. We may want to for the other reasons you suggest... On Thu, Oct 11, 2012 at 8:49 PM, Brian Smith wrote: > Zack Weinberg wrote: >> Link-time optimization is described as an experimental new feature in >> the GCC 4.5.0 release notes[1]. The 4.6.0 release notes[2] say that >> it has now "stabilized to the point of being usable", and the 4.7.0 >> release notes[3] describe it as still further improved both in >> reliability and code quality. If we're trying to use the 4.5 LTO, >> I'm not at all surprised to hear it's causing more trouble than it's >> worth. >> >> PGO is not the same thing as LTO, of course, but GCC's PGO was kind >> of an unloved stepchild until they got serious about LTO, so that, >> too, is likely to be much improved in 4.7. > > I think it is important to give Linux users the fastest browser we can give > them, because: > > 1. Linux users tend to be disproportionately influential in the markets we > care the most about (web developers, techies) > 2. Linux is the foundation of B2G and Firefox for Android, where we > *definitely* must deliver the fastest product we can > > Now, if it were up to me, I'd try to reproduce this on a build built with the > latest stable GCC or latest stable clang, and if that fixes the issue, I'd > consider this a big motivation for upgrading to GCC 4.5 to a better compiler, > which we need to do anyway for language feature support. Definitely, I don't > think we should be adding hacks to our code to work around GCC problems that > are already fixed in later releases of GCC. It's better to just make the > build fail when the user attempts to use one of those older GCC releases. > > Now, if PGO doesn't result in the fastest browser, then of course we should > disable PGO. > > Or, if there is no better compiler possible, then yes, I think it makes sense > to disable PGO temporarily until there is a better compiler available. > (And/or, help fix the compiler, either by contributing a patch, or by > commissioning somebody else to contribute a patch.) > > Cheers, > Brian > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
