Re: Landing c-c-breaking changes that remove c-c-only code from m-c
On Thu, Apr 17, 2014 at 5:00 PM, Neil wrote: > Henri Sivonen wrote: > >> I am not done preparing the removal patches yet, but with my current patch >> queue I can already get 149 KB off of Android ARMv7 optimized apk size and >> 138 KB off of Android ARMv7 optimized libxul size. >> > What would be your opinion on the interim use of ifdef (or moz.build > equivalent) to prevent some or all of those files from being built on > browser builds? I actually asked about this twice. (Message-IDs: and ; Google Groups seems broken, so I can't link there.) The first time around, I got no reply, so I started a new thread with the #ifdef subject. Nobody told me to go do it or what symbol to #ifdef on. Instead, bsmedberg said we shouldn't do #ifdefs if Linux distros still build Thunderbird on top of XULRunner. Gabriele Svelto took a look at the distros, and there was actually no evidence of the distros building Thunderbird that way. Mike Hommey said: "Thunderbird is already linking stuff in libxul from c-c. Why wouldn't it be possible to move those bits to c-c and do the same?" So I think proponents of #ifdefing had their opportunity to be heard. Now that I've actually prepared some patches and discovered how easy it is to move C++ code from m-c to c-c (at least when c-c is built the normal way), I'm strongly opposed to #ifdefing. Let's be realistic about "interim". I've been unsuccessfully looking for a volunteer to do the easy UTF-7 move since November. Now I've lowered the bar to looking for a volunteer who is familiar with c-c jar manifests to fix a couple of those--still unsuccessfully. If I can't find a c-c developer who could be bothered to fix a couple of jar manifests, realistically, no c-c developer will be bothered to get rid of the #ifdefs later if I instead put some #ifdefs in m-c jar manifests. On Thu, Apr 17, 2014 at 6:52 PM, Kent James wrote: > Henri, is this work affected by the deadline for the upcoming Gecko 31 to > aurora lift, which effectively determines what is in Thunderbird code for > the next year? > > If not, and if there is any way you could delay your landing until mid-May, > then there would be less pressure on Thunderbird developers. At the moment, > we are trying to land last-minute changes needed for Thunderbird 31 prior to > Gecko 31 hitting comm-aurora. There is no strong need for the m-c removals to be tied to the ESR schedule. However, I think it would be preferable to make the changes before the ESR deadline so that future c-c patches backport easily to the ESR (if there are security holes in the code being moved; and there may well be, since decoders for multibyte encodings are prone to buffer overflows / pointer arithmetic errors) and future m-c patches would backport easily to the m-c ESR, too. Also, at this point, I see pretty much any concern of the form "wait for x" as Stop Energy. On Thu, Apr 17, 2014 at 7:38 PM, Chris Peterson wrote: > Note that the current mozilla-central (Gecko 31) will become the next ESR. > If you remove the UTF-7 code now, then TB developers will need to fix their > UTF-7 issues for the next GA release of TB (because it is based on ESRs). It's a code move. Either the moved classes instantiate or they don't. If they don't, IMAP won't work, so it'll be fairly obvious if they fail to instantiate. As far as the UTF-7 move goes, I'm not worried about creating new subtle bugs. The nsCharsetMenu and nsCharsetConverterManager moves will similarly either work or fail very, very obviously. Only the later encoding moves are subtle in the sense that they could be broken and no one might notice. - - Considering that off-list, the sentiment I've heard implies that moving the C++ code is seen as somehow difficult, I think I should clarify what the issues are. Moving the C++ code is not difficult. That is, assuming that Thunderbird is built the way Mozilla builds it with its own libxul. If there's some Linux distro out there that wants to split Thunderbird into two or more packages somehow, that should be entirely their problem to deal with and not something that we should block m-c cleanup on. Fixing jar manifests should be no big deal, either. However, since I happen to be a Gecko developer and not a Firefox UI developer, I'm not familiar with the finer points and conventions of jar manifests on the Firefox side, either, and it seems that with the Thunderbird/SeaMonkey duality, c-c jar manifests are a bit more complicated. Obviously, with enough trial and error I could learn. But spending the time to put together something that would run only to be told by a reviewer to do it differently seems terribly inefficient and a bad use of time when someone who already knows their way around c-c jar manifests could probably finish my patches as a matter of minutes. And let's not forget that I'm not supposed to be working on the c-c side at all--I'm trying to be nice not burning it outright, which, AFAICT, I'd be allowed to do per m-c tree rules. The unit
Intent to remove: EUC-TW as nsIPlatformCharset
EUC-TW is a rare Traditional Chinese encoding. (Big5 is the more common legacy encoding.) The Web is not affected: We already are not exposing EUC-TW to the Web. Mac, Android and B2G are not affected (except in terms of libxul becoming smaller): nsIPlatformCharset is always UTF-8 there. Windows is not affected (except in terms of libxul becoming smaller): nsIPlatformCharset for Traditional Chinese is either Big5 or Big5-HKSCS there. Gtk *nix systems whose locale is not Traditional Chinese are not affected (except in terms of libxul becoming smaller). Gtk *nix systems whose locale is Traditional Chinese but whose locale encoding is UTF-8 or Big5 are not affected. File system access is not affected. For historical reasons unclear to me, Unix file system code doesn't actually use nsIPlatformCharset. However, I think file: URL directory listings will be affected. On Gtk *nix systems with an EUC-TW locale, localized date (month name, weekday name) formatting will be affected. It seems to me that X clipboard handling for non-UTF-8 text is already broken, so we don't *really* support non-UTF-8 *nix locales anyway. Indications in our code base suggest that EUC-TW support has mainly been a Solaris and AIX thing. Also, it appears that Red Hat 7 had it as an option. Red Hat 8, released in 2002, defaulted to UTF-8. On the Debian side, UTF-8 has been the default since 2007. As far as I was able to find instructions for using Traditional Chinese on Debian before that, it appears that Debian used Big5 before moving to UTF-8. It's imaginable that some organization might have a Solaris/AIX-influenced NFS system that's been up and running using EUC-TW for more than 15 years and, therefore, still have Linux or Solaris systems configured with an EUC-TW locale. Other than that, it seems very improbable that anyone would use an EUC-TW system locale these days. Unfortunately, I missed EUC-TW when I added telemetry for encodings that we'd want to remove. EUC-TW is the only (non-OS/2) nsIPlatformCharset that's not in the Encoding Standard and not exposed to the Web. Removing all support for EUC-TW allows us to use the Web-oriented encoding label handling internally for all things and allows us to get rid of the lookup tables that and to the size of libxul. This does not affect Thunderbird's ability to continue to support EUC-TW if the Thunderbird developers so choose. (But EUC-TW email seems improbable, since no one has bothered to register an IANA label for it.) P.S. If your module uses nsIPlatformCharset, please stop using it! All our platform have UTF-16 (Windows) or UTF-8 (other) APIs these days. -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Using promises from C++
On 22/04/2014 05:24, Joshua Cranmer 🐧 wrote: For more specific explanation then: I have a (non-negotiably) JS-implemented XPCOM service called the Folder Lookup Service. I want to add an API to it called getOrCreateFolder that returns a Promise. I have non-negotiable C++ code that wants to call this method, and do something when the promise is resolved or rejected. Dumb question time: why not provide an XPIDL'd alternative JS wrapper method (getOrCreateFolderForNativeConsumers or whatever) that takes two nsIRunnables or equivalent XPIDL'd C++ callback wrappers, and wrapping the getOrCreateFolder method with .then((x) => successRunnable.run(x), (x) => failRunnable.run(x)); ? That requires 0 API changes on the JSAPI/xpconnect side... (note that I'm approximately the worst person to ask about how to efficiently pass a C++ callback to a JS XPIDL implementation, but an nsIRunnable would at least work, AIUI - there may be better things that you can use) ~ Gijs ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
It's all over the tree, inconsistently applied. Is it relevant anymore? Can we remove it entirely, or there some places where it's still relevant, and if so, where ... XPCOM? Or should we be using it everywhere? Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Intent to remove: MS-DOS encodings other than Cyrillic as encodings exposed to extensions
We already do not expose MS-DOS encodings other than Cyrillic to the Web. We still expose them to Firefox extensions in some APIs. Telemetry shows that usage varies from non-existent to extremely rare (28 sessions out of 180.82 million sessions for the Western European DOS code page). In order to be able to use the same alias code for the Web and for extensions and in order to make libxul smaller for memory-constrained B2G phones, I intend to remove support for MS-DOS encodings other than Cyrillic as a feature exposed to extensions. Thunderbird developers will be able to import the code into comm-central and continued to support these encodings for email and NNTP if they so choose. (I'm not counting Thai and CJK encodings that are shared by DOS and Windows as MS-DOS encodings here. Those will stay.) P.S. Fun fact: We've never supported the full set of DOS encodings. In particular, we've never supported the U.S. English DOS encoding that remains the default encoding of en-US cmd.exe to this day! -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to remove: MS-DOS encodings other than Cyrillic as encodings exposed to extensions
On 2014-04-22, 8:09 AM, Henri Sivonen wrote: We already do not expose MS-DOS encodings other than Cyrillic to the Web. We still expose them to Firefox extensions in some APIs. Telemetry shows that usage varies from non-existent to extremely rare (28 sessions out of 180.82 million sessions for the Western European DOS code page). How reliable is our telemetry data here? I'm not really sure how the measurement works. Is it based on the number of times that we encounter this encoding per the number of pages loaded in a session? Or is it a boolean flag indicating whether we've hit the encoding in a given session? Do we have any reason to believe that the portion of our users opting in to Telemetry on the release channel correctly represents the potential demographics who might be viewing pages written in these kinds of encodings? Also, what is the status of the support for these encodings in other UAs? I'm somewhat worried that we might break some Web pages for users who are not fairly represented in our Telemetry data, and that we may not hear about this before this change hits the release channel. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Landing c-c-breaking changes that remove c-c-only code from m-c
(2014/04/22 2:09), Joshua Cranmer 🐧 wrote: On 4/21/2014 11:50 AM, Ehsan Akhgari wrote: You can check CONFIG['MOZ_BUILD_APP'] against what you pass to --enable-application in mozconfig files. So that would be 'mailnews' (or 'mail'?) and 'suite' for Thunderbird and Seamonkey respectively. Actually, CONFIG['MOZ_THUNDERBIRD'] and CONFIG['MOZ_SUITE'] are what you want. Trying to check MOZ_BUILD_APP is wrong, even if you had the name of Thunderbird right (it's mail), do the multiple topsrcdir issue. Maybe I am missing something, but I thought some of the changes mentioned involve c++ files and so I wanted to learn a simple identifier to use as in #ifdef XYZ,#endif CONFIG['MOZ_THUNDERBIRD'] and CONFIG['MOZ_SUITE'] are likely to be used in shell/python/ruby or whatever script. Am I correct? Henry Svenson now seems inclined against the "#ifdef"ing, and so the issue is moot. TIA ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to remove: MS-DOS encodings other than Cyrillic as encodings exposed to extensions
On Tue, Apr 22, 2014 at 2:58 PM, Ehsan Akhgari wrote: > How reliable is our telemetry data here? I'm not really sure how the > measurement works. Is it based on the number of times that we encounter > this encoding per the number of pages loaded in a session? Or is it a > boolean flag indicating whether we've hit the encoding in a given session? > Do we have any reason to believe that the portion of our users opting in to > Telemetry on the release channel correctly represents the potential > demographics who might be viewing pages written in these kinds of encodings? > > Also, what is the status of the support for these encodings in other UAs? > > I'm somewhat worried that we might break some Web pages for users who are > not fairly represented in our Telemetry data, and that we may not hear about > this before this change hits the release channel. As Henri wrote, we are already not exposing these encodings to the web. This data is about non-web usage only. -- http://annevankesteren.nl/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to remove: MS-DOS encodings other than Cyrillic as encodings exposed to extensions
On Tue, Apr 22, 2014 at 3:58 PM, Ehsan Akhgari wrote: > How reliable is our telemetry data here? I suppose as reliably as telemetry data in general. I suppose its reliability is hard to measure. > I'm not really sure how the > measurement works. Is it based on the number of times that we encounter > this encoding per the number of pages loaded in a session? No. > Or is it a > boolean flag indicating whether we've hit the encoding in a given session? Yes. > Do we have any reason to believe that the portion of our users opting in to > Telemetry on the release channel correctly represents the potential > demographics who might be viewing pages written in these kinds of encodings? I have no reason to believe that telemetry wouldn't be representative of actual demand. > Also, what is the status of the support for these encodings in other UAs? Chrome doesn't support the encoding being removed here and Presto-Opera didn't support them, either. Non-support in Chrome and Presto-Opera is what made us dare to stop exposing these encodings to the Web earlier. IE and Safari support all kinds of cruft, including DOS encodings, as a matter of exposing underlying system libraries without regard to the actual needs of the Web. > I'm somewhat worried that we might break some Web pages for users who are > not fairly represented in our Telemetry data, and that we may not hear about > this before this change hits the release channel. We have already stopped exposing these encodings to the Web. We stopped recognizing the labels for these encodings in Firefox 19 for the purpose of Web content being able to request the use these encodings. However, we still kept around the possibility to manually choose these encodings from the menu as an override. We removed them from the menus in Firefox 28. At the time of Firefox 19, one person in Spain noticed that IBM850 (DOS Western European) went away. The affected page was a page maintained by this person himself. At the time of Firefox 28, one person noticed that IBM862 (DOS Hebrew) went away. These have been isolated cases of people who knowingly use legacy encodings noticing their own stuff breaking. At the time of Firefox 28, another person in Spain noticed that IBM850 went away. Unclear if that was a self-maintained case or not. DOS Hebrew has been even less interoperable than the other DOS encodings, because browsers didn't agree on whether it was a visual or logical encoding. -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to remove: MS-DOS encodings other than Cyrillic as encodings exposed to extensions
On 4/22/2014 7:09 AM, Henri Sivonen wrote: Thunderbird developers will be able to import the code into comm-central and continued to support these encodings for email and NNTP if they so choose. Based on my charset explorations in NNTP, the only common IBM* encoding not present on the encoding spec is one we never supported in the first place, so I don't think comm-central has a desire for any of these. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On 4/22/2014 7:31 AM, Robert O'Callahan wrote: It's all over the tree, inconsistently applied. Is it relevant anymore? Can we remove it entirely, or there some places where it's still relevant, and if so, where ... XPCOM? Or should we be using it everywhere? Short answer: I don't think it's relevant any more. Long answer: Each compiler and platform is a little different, so here's my take on where we are: * Windows/MSVC. NS_HIDDEN never had any effect; symbols are hidden by default and need dllexport to be exported * Mac/GCC or clang. We compile with -fvisibility=hidden. This means that all symbols are hidden by default and you need NS_EXPORT to mark them as exported/dynamic. Declarations are treated as dynamic. However because Mach-O executables use direct relocations, there is no actual difference between the generated code referencing a hidden or dynamic symbol, so we're fine. * Linux/GCC or clang. We compile with #pragma visibility. This means that all symbols and declarations are treated as hidden by default and require NS_EXPORT if they are exported/dynamic. I believe that we're also using #pragma visibility on Android and B2G, but somebody should probably check. We should be! --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to remove: EUC-TW as nsIPlatformCharset
On 4/22/2014 4:16 AM, Henri Sivonen wrote: This does not affect Thunderbird's ability to continue to support EUC-TW if the Thunderbird developers so choose. (But EUC-TW email seems improbable, since no one has bothered to register an IANA label for it.) I estimate EUC-TW to be unnecessary to gain 99.99% charset coverage of email with very high confidence (no messages in either my recent 1M message dataset or my older 10M message dataset). -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Using promises from C++
On 4/22/2014 5:55 AM, Gijs Kruitbosch wrote: On 22/04/2014 05:24, Joshua Cranmer 🐧 wrote: For more specific explanation then: I have a (non-negotiably) JS-implemented XPCOM service called the Folder Lookup Service. I want to add an API to it called getOrCreateFolder that returns a Promise. I have non-negotiable C++ code that wants to call this method, and do something when the promise is resolved or rejected. Dumb question time: why not provide an XPIDL'd alternative JS wrapper method (getOrCreateFolderForNativeConsumers or whatever) that takes two nsIRunnables or equivalent XPIDL'd C++ callback wrappers, and wrapping the getOrCreateFolder method with .then((x) => successRunnable.run(x), (x) => failRunnable.run(x)); That's certainly possible, but, as my reviewer commented, if we want to start using promises a lot more in the API, it makes more sense to try to find a better solution now rather than proliferate ad-hoc solutions all over the place (the current state of the art is to stuff the thing being processed into a URL and use an nsIUrlListener, which doesn't work quite so well if the object isn't really a URL in the first place). -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Using promises from C++
On 4/22/14, 2:51 AM, Joshua Cranmer 🐧 wrote: I'm giving to thinking in terms of all of the magic hiding that XPIDL/XPCOM/xpconnect does, so things like "what is my global object" being intrinsically important are annoying to me. I understand that. but something like Promises or TextEncoder which is more or less like "syntactic sugar for code I could write but don't want to" The problem is that the global they're attached to is observable. For example, TextEncoder returns returns a typed array, and the returned object is obviously tied to some global... When I create an nsIJSImplementedInterfaceMagicObject from C++, I don't have to worry about global objects. Perhaps xpconnect should just expose the "here's a dummy global if you need one" more easily? xpc::GetJunkScopeGlobal exists (though perhaps not outside libxul?). You also need explicit approval from the xpconnect module owner to use it, because if you create objects in there and hand them to someone else they can tell you used the junk scope. There's a tension here between hiding the implementation details and making sure people get them right if the code you hand your objects to decides to poke at them. :( I've heard that Promises.jsm are still better to use than DOM Promises... Clearly they're not if you need to deal with C++ code! ToJSValue I think makes some of the pain of converting from T (for almost any xpidl-usable type, at least) much less painful. A corresponding FromJSValue could probably work well too. We could certainly add some helpers for common operations like that, yes. When I have some more time, I can try playing with some magic prototype implementations and suggest which methods would be sufficient to be useful for my needs. That would be awesome! The IDL annotation I think I'd want first is not defining these in the mozilla::dom namespace. We could do that, with enough work... but I suspect it would take a surprising amount of work. I'd like to see good reason for this one. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Landing c-c-breaking changes that remove c-c-only code from m-c
On 4/22/2014 2:12 AM, Henri Sivonen wrote: There is no strong need for the m-c removals to be tied to the ESR schedule. However, I think it would be preferable to make the changes before the ESR deadline so that future c-c patches backport easily to the ESR (if there are security holes in the code being moved; and there may well be, since decoders for multibyte encodings are prone to buffer overflows / pointer arithmetic errors) and future m-c patches would backport easily to the m-c ESR, too. If these encoders/decoders have been around for 10 years without security holes being found, I'll doubt there'll be any reported in the next year. Given that there is less than a week to the next branch and we're already fighting multiple failures due to m-c related bustage, if this isn't any real need to force the removal to occur before the ESR branch, -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to remove: MS-DOS encodings other than Cyrillic as encodings exposed to extensions
On 2014-04-22, 9:24 AM, Henri Sivonen wrote: I'm somewhat worried that we might break some Web pages for users who are not fairly represented in our Telemetry data, and that we may not hear about this before this change hits the release channel. We have already stopped exposing these encodings to the Web. We stopped recognizing the labels for these encodings in Firefox 19 for the purpose of Web content being able to request the use these encodings. However, we still kept around the possibility to manually choose these encodings from the menu as an override. We removed them from the menus in Firefox 28. Ah, sorry, I was misled by the title of your post and admittedly lack of complete attention to the first sentence and though that this is a Web platform removal intent, but it seems like it's not, therefore I withdraw my concerns. At the time of Firefox 19, one person in Spain noticed that IBM850 (DOS Western European) went away. The affected page was a page maintained by this person himself. At the time of Firefox 28, one person noticed that IBM862 (DOS Hebrew) went away. These have been isolated cases of people who knowingly use legacy encodings noticing their own stuff breaking. At the time of Firefox 28, another person in Spain noticed that IBM850 went away. Unclear if that was a self-maintained case or not. That's reassuring! DOS Hebrew has been even less interoperable than the other DOS encodings, because browsers didn't agree on whether it was a visual or logical encoding. Hmm, I thought that everyone agrees that it's visual (as DOS lacked any support for bidirectional text), but that's off topic for this discussion. :-) Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
NS_ERROR_NET_PARTIAL_TRANSFER
Hey all, In the networking team we're about to introduce an error code for "partial transfers" (see https://bugzilla.mozilla.org/show_bug.cgi?id=237623). It will be returned when the underlying protocol's framing is cut off and we know that the transfer was indeed partial - not all data arrived. Right now the patch in progress returns this value for HTTP 1.1 Content-Length: underrun, cut off chunked-encoding chunks and broken SPDY or http2 streams. While the primary bug report for this identifies a problem in the download manager that makes it not discover cut off transfers and instead signal them as complete, this change will and can have some further impact on other areas of Firefox and I want to highlight this, give you all a heads up and listen in if there are any additional thoughts or feedback on this pending change. The question I think all consumers of protocol data need to ask ourselves is: how should we deal with partial contents? Up until today, we silently just ignored the potential problems and considered whatever we got fine enough to use. Images? Javascript? CSS? Anything else? This change will also make Firefox align with the other browsers that already are stricter in this area. -- / daniel.haxx.se ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Wed, Apr 23, 2014 at 1:25 AM, Benjamin Smedberg wrote: > On 4/22/2014 7:31 AM, Robert O'Callahan wrote: > >> It's all over the tree, inconsistently applied. Is it relevant anymore? >> Can >> we remove it entirely, or there some places where it's still relevant, and >> if so, where ... XPCOM? Or should we be using it everywhere? >> > > Short answer: I don't think it's relevant any more. > > Long answer: Each compiler and platform is a little different, so here's > my take on where we are: > > * Windows/MSVC. NS_HIDDEN never had any effect; symbols are hidden by > default and need dllexport to be exported > * Mac/GCC or clang. We compile with -fvisibility=hidden. This means that > all symbols are hidden by default and you need NS_EXPORT to mark them as > exported/dynamic. Declarations are treated as dynamic. However because > Mach-O executables use direct relocations, there is no actual difference > between the generated code referencing a hidden or dynamic symbol, so we're > fine. > * Linux/GCC or clang. We compile with #pragma visibility. This means that > all symbols and declarations are treated as hidden by default and require > NS_EXPORT if they are exported/dynamic. > On B2G and Android we're compiling with -fvisibility=hidden. configure output says: checking For gcc visibility bug with class-level attributes (GCC bug 26905)... yes checking For x86_64 gcc visibility bug with builtins (GCC bug 20297)... yes Is that a problem? Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On 4/22/2014 6:41 PM, Robert O'Callahan wrote:
On B2G and Android we're compiling with -fvisibility=hidden. configure
output says:
checking For gcc visibility bug with class-level attributes (GCC bug 26905)...
yes
checking For x86_64 gcc visibility bug with builtins (GCC bug 20297)... yes
Is that a problem?
Yes! We can probably get codesize, startup perf, and runtime perf gains
of 3-8% by fixing this. There was a recent thread in dev.builds about
this, actually, which I didn't realize was related but probably is:
I found a bug in configure.in around line 2640:
ac_cv_have_visibility_builtin_bug=no
if ! ${CC-cc} ${CFLAGS} ${DSO_PIC_CFLAGS}
${DSO_LDOPTS} -O2 -S -o conftest.S conftest.c > /dev/null 2>&1 ; then
ac_cv_have_visibility_builtin_bug=yes
else
if test `grep -c "@PLT" conftest.S` = 0; then
ac_cv_visibility_builtin_bug=yes
fi
fi
The second 'if' uses a wrong variable name, so the result of 'grep' is
effectively ignored. The problem is that if I fix this bug, clang
(3.4.1) fails this test (it optimizes all code out with -O2), the
configuration variables (related to -fvisibility) are changed
accordingly, and then a simple firefox build fails. The same bug is
also in js/src/configure.in.
We should either fix or remove this configure test. I think that
removing it and always using pragmas is probably correct, but we might
need to opt mac out of that because our list of system headers on mac
isn't up to date.
--BDS
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Wed, Apr 23, 2014 at 10:41:06AM +1200, Robert O'Callahan wrote: > On Wed, Apr 23, 2014 at 1:25 AM, Benjamin Smedberg > wrote: > > > On 4/22/2014 7:31 AM, Robert O'Callahan wrote: > > > >> It's all over the tree, inconsistently applied. Is it relevant anymore? > >> Can > >> we remove it entirely, or there some places where it's still relevant, and > >> if so, where ... XPCOM? Or should we be using it everywhere? > >> > > > > Short answer: I don't think it's relevant any more. > > > > Long answer: Each compiler and platform is a little different, so here's > > my take on where we are: > > > > * Windows/MSVC. NS_HIDDEN never had any effect; symbols are hidden by > > default and need dllexport to be exported > > * Mac/GCC or clang. We compile with -fvisibility=hidden. This means that > > all symbols are hidden by default and you need NS_EXPORT to mark them as > > exported/dynamic. Declarations are treated as dynamic. However because > > Mach-O executables use direct relocations, there is no actual difference > > between the generated code referencing a hidden or dynamic symbol, so we're > > fine. > > * Linux/GCC or clang. We compile with #pragma visibility. This means that > > all symbols and declarations are treated as hidden by default and require > > NS_EXPORT if they are exported/dynamic. > > > > On B2G and Android we're compiling with -fvisibility=hidden. configure > output says: > > checking For gcc visibility bug with class-level attributes (GCC bug > 26905)... yes > checking For x86_64 gcc visibility bug with builtins (GCC bug 20297)... yes > > Is that a problem? No, that's just how we choose between #pragma visibility and -fvisibility=hidden Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Wednesday 2014-04-23 07:51 +0900, Mike Hommey wrote: > On Wed, Apr 23, 2014 at 10:41:06AM +1200, Robert O'Callahan wrote: > > checking For gcc visibility bug with class-level attributes (GCC bug > > 26905)... yes > > checking For x86_64 gcc visibility bug with builtins (GCC bug 20297)... yes > > > > Is that a problem? > > No, that's just how we choose between #pragma visibility and > -fvisibility=hidden https://bugzilla.mozilla.org/show_bug.cgi?id=273336#c0 describes the advantages of #pragma visibility over -fvisibility=hidden (at least on Linux; from bsmedberg's post it sounds like it doesn't make a difference on Mac). (That patch was later conditioned on configure tests in bugs 293438 and 307168 to work around compiler bugs; this is the configure test you're observing.) -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914) signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Wed, Apr 23, 2014 at 07:51:54AM +0900, Mike Hommey wrote: > On Wed, Apr 23, 2014 at 10:41:06AM +1200, Robert O'Callahan wrote: > > On Wed, Apr 23, 2014 at 1:25 AM, Benjamin Smedberg > > wrote: > > > > > On 4/22/2014 7:31 AM, Robert O'Callahan wrote: > > > > > >> It's all over the tree, inconsistently applied. Is it relevant anymore? > > >> Can > > >> we remove it entirely, or there some places where it's still relevant, > > >> and > > >> if so, where ... XPCOM? Or should we be using it everywhere? > > >> > > > > > > Short answer: I don't think it's relevant any more. > > > > > > Long answer: Each compiler and platform is a little different, so here's > > > my take on where we are: > > > > > > * Windows/MSVC. NS_HIDDEN never had any effect; symbols are hidden by > > > default and need dllexport to be exported > > > * Mac/GCC or clang. We compile with -fvisibility=hidden. This means that > > > all symbols are hidden by default and you need NS_EXPORT to mark them as > > > exported/dynamic. Declarations are treated as dynamic. However because > > > Mach-O executables use direct relocations, there is no actual difference > > > between the generated code referencing a hidden or dynamic symbol, so > > > we're > > > fine. > > > * Linux/GCC or clang. We compile with #pragma visibility. This means that > > > all symbols and declarations are treated as hidden by default and require > > > NS_EXPORT if they are exported/dynamic. > > > > > > > On B2G and Android we're compiling with -fvisibility=hidden. configure > > output says: > > > > checking For gcc visibility bug with class-level attributes (GCC bug > > 26905)... yes > > checking For x86_64 gcc visibility bug with builtins (GCC bug 20297)... yes > > > > Is that a problem? > > No, that's just how we choose between #pragma visibility and > -fvisibility=hidden Well, its at least strange given the bugs mentioned in configure were supposedly fixed in gcc 4.1, but I wouldn't be at all suprised if investigating it finds the tests are wrong. Trev > > Mike > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Tue, Apr 22, 2014 at 07:25:46PM -0400, Trevor Saunders wrote: > > No, that's just how we choose between #pragma visibility and > > -fvisibility=hidden > > Well, its at least strange given the bugs mentioned in configure were > supposedly fixed in gcc 4.1, but I wouldn't be at all suprised if > investigating it finds the tests are wrong. They most certainly are. As Benjamin mentioned, we should just remove them and force pragmas with gcc/clang, except on mac, where we'd use -fvisibility=hidden. Want to finish bug 961264? ;) Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Wed, Apr 23, 2014 at 11:48 AM, Mike Hommey wrote: > On Tue, Apr 22, 2014 at 07:25:46PM -0400, Trevor Saunders wrote: > > > No, that's just how we choose between #pragma visibility and > > > -fvisibility=hidden > > > > Well, its at least strange given the bugs mentioned in configure were > > supposedly fixed in gcc 4.1, but I wouldn't be at all suprised if > > investigating it finds the tests are wrong. > > They most certainly are. As Benjamin mentioned, we should just remove > them and force pragmas with gcc/clang, except on mac, where we'd use > -fvisibility=hidden. Want to finish bug 961264? ;) > But the #pragma test is only failing on Mac because the test is wrong, right? Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Wed, Apr 23, 2014 at 11:52:12AM +1200, Robert O'Callahan wrote: > On Wed, Apr 23, 2014 at 11:48 AM, Mike Hommey wrote: > > On Tue, Apr 22, 2014 at 07:25:46PM -0400, Trevor Saunders wrote: > > > No, that's just how we choose between #pragma visibility and > > > -fvisibility=hidden > > > > Well, its at least strange given the bugs mentioned in configure were > > supposedly fixed in gcc 4.1, but I wouldn't be at all suprised if > > investigating it finds the tests are wrong. > > They most certainly are. As Benjamin mentioned, we should just remove > them and force pragmas with gcc/clang, except on mac, where we'd use > -fvisibility=hidden. Want to finish bug 961264? ;) yeah, should get to it soon. > But the #pragma test is only failing on Mac because the test is wrong, right? very likely yes, though making pragmas work on macmight take more effort and its not really clear it would be any better there. Trev > > Rob > -- > Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le > atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa > stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm > aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp > waanndt wyeonut thoo mken.o w signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we remove NS_HIDDEN, NS_HIDDEN_(...)?
On Tue, Apr 22, 2014 at 09:25:01AM -0400, Benjamin Smedberg wrote: > On 4/22/2014 7:31 AM, Robert O'Callahan wrote: > >It's all over the tree, inconsistently applied. Is it relevant anymore? Can > >we remove it entirely, or there some places where it's still relevant, and > >if so, where ... XPCOM? Or should we be using it everywhere? > > Short answer: I don't think it's relevant any more. > > Long answer: Each compiler and platform is a little different, so here's my > take on where we are: > > * Windows/MSVC. NS_HIDDEN never had any effect; symbols are hidden by > default and need dllexport to be exported > * Mac/GCC or clang. We compile with -fvisibility=hidden. This means that all > symbols are hidden by default and you need NS_EXPORT to mark them as > exported/dynamic. Declarations are treated as dynamic. However because > Mach-O executables use direct relocations, there is no actual difference > between the generated code referencing a hidden or dynamic symbol, so we're > fine. FWIW, this is not entirely true: https://bugzilla.mozilla.org/show_bug.cgi?id=961264#c11 Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: NS_ERROR_NET_PARTIAL_TRANSFER
Do we currently return NS_OK from Necko in such circumstances or another error code? On 2014-04-22, 4:21 PM, Daniel Stenberg wrote: Hey all, In the networking team we're about to introduce an error code for "partial transfers" (see https://bugzilla.mozilla.org/show_bug.cgi?id=237623). It will be returned when the underlying protocol's framing is cut off and we know that the transfer was indeed partial - not all data arrived. Right now the patch in progress returns this value for HTTP 1.1 Content-Length: underrun, cut off chunked-encoding chunks and broken SPDY or http2 streams. While the primary bug report for this identifies a problem in the download manager that makes it not discover cut off transfers and instead signal them as complete, this change will and can have some further impact on other areas of Firefox and I want to highlight this, give you all a heads up and listen in if there are any additional thoughts or feedback on this pending change. The question I think all consumers of protocol data need to ask ourselves is: how should we deal with partial contents? Up until today, we silently just ignored the potential problems and considered whatever we got fine enough to use. Images? Javascript? CSS? Anything else? This change will also make Firefox align with the other browsers that already are stricter in this area. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: NS_ERROR_NET_PARTIAL_TRANSFER
On 4/22/14, 9:30 PM, Ehsan Akhgari wrote: Do we currently return NS_OK from Necko in such circumstances or another error code? Currently we return NS_OK, so the necko client thinks the transfer completed successfully. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: NS_ERROR_NET_PARTIAL_TRANSFER
On 2014-04-22, 9:59 PM, Boris Zbarsky wrote: On 4/22/14, 9:30 PM, Ehsan Akhgari wrote: Do we currently return NS_OK from Necko in such circumstances or another error code? Currently we return NS_OK, so the necko client thinks the transfer completed successfully. That seems like a huge behavior change. :( ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: NS_ERROR_NET_PARTIAL_TRANSFER
On 4/22/14, 10:06 PM, Ehsan Akhgari wrote: That seems like a huge behavior change. :( Yes, it is. There's some at-length discussion in the bug about what behavior various consumers want here. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Removing "[PID]" prefix from |make mozmill| warning/error/assertion lines?
On (2014年04月22日 08:11), Zack Weinberg wrote:
> On 2014-04-21 1:07 PM, Steve Fink wrote:
>> On Sat 19 Apr 2014 08:36:22 AM PDT, ISHIKAWA,chiaki wrote:
>>> egrep "^(\\[[0-9]*\\] |)WARNING" $1 | egrep NS_ENSURE | grep -v "sort
>>> operation has occurred for the SQL statement" | sort | uniq -f1 -c |
>>> sort -n -r
>>
>> It'd be easier if you threw in a *little* bit of perl:
>>
>>perl -lne 'print $1 if /WARNING: (NS_ENSURE.*)/' | sort | uniq -c |
>> sort -nr
>
> If you're going to use perl, you might as well use perl:
>
> perl -e 'my %hits;
> while (<>) { $hits{$1}++ if /WARNING: (NS_ENSURE.*)/ }
> printf("%d\t%s\n", $hits{$_}, $_)
> for sort { $hits{$b} <=> $hits{$a} } keys %hits;'
>
> Untested but, I believe, equivalent. Could perhaps be further optimized by
> use of each() instead of keys() but I have forgotten too much to do it myself.
>
> zw
Thank you for the suggestions.
My original intent was to avoid script programming as much as possible
and yet gain reasonably good summary.
I guess the time is over for a simple shell scripting to
produce a meaningful summary from ever evolving log record.
(Yes, my script goes over the log many times and in that sense it is not
very efficient, but
it can handle 100+MiB |mach mochitest-plain| of debug build of FF in several
seconds on my local PC and not too bad for my purpose of obtaining a
meaningful summary without investing too much in programming.)
When I have a time to re-engineer my script, I will post a prototype for
review by people who may be interested in such a script.
TIA
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Landing c-c-breaking changes that remove c-c-only code from m-c
On Tue, Apr 22, 2014 at 5:48 PM, Joshua Cranmer 🐧 wrote: > If these encoders/decoders have been around for 10 years without security > holes being found, I'll doubt there'll be any reported in the next year. The encoding converters have been around for more than 10 years and there have been sec-* bugs in them. In fact, sec-* bugs have been found recently after the code had aged for 10 years. One reason I want to get rid of multi-byte encodings that we don't absolutely need is that multi-byte decoders written with C-style pointer arithmetic and gotos are prone to bugs. The idea of Thunderbird adopting the least scrutinized multi-byte decoders creeps me out as a Thunderbird user, but at least the attack surface in Firefox will get smaller. if (CHECK_OVERRUN(dest, destEnd, 2)) goto error1; At least there are overrun checks. As for the landing schedule relative to ESR, it looks like natural inertia might push the landings past the ESR branch point. (Speaking of ESR, I sure hope Thunderbird gets rid of VISCII and ARMSCII in the ESR message compose menu even if it means landing something on Aurora/Beta.) -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
