Re: Upcoming changes to the Linux 32-bits and Android builders
On Wed, May 15, 2013 at 11:42:33AM +0200, Mike Hommey wrote: Hi, With the increasing number of reports of local builds for Linux and Android hitting the 32-bits address space limits when linking, and with the fact that gcc 4.7 on try does hit that limit on PGO builds, it has become necessary to switch the 32-bits Linux and Android builds to 64-bits environments and toolchains. This switch is going to happen some time today (May 15). So, interestingly, while buildbot reconfig was triggered yesterday, somehow it didn't make the builds actually switch to 32-bits environment. After Justin Wood did a re-reconfig this morning (european time), things got straight up... but non-clobber builds got red because of the python virtualenv still containing a 32-bits binary when the system python library is now 64-bits. While I think I triggered all the necessary clobbers on all branches where it's needed, but I didn't inspect all of them to retrigger failed builds. In the event there are some project branches with the following error: error while loading shared libraries: libpython2.7.so.1.0: wrong ELF class: ELFCLASS64 Please retrigger these builds after checking the clobberer says the last clobber is newer than 2013-05-16 01:10:00 PDT. Following this, we're going to switch all Linux builds to gcc 4.7, later this week. Chances are this is not going to happen this week. Something in the last 569 changesets before 15ba59a74221 broke some webgl mochitest when built with gcc 4.7. Filed bug 872960. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Storage in Gecko
On 5/16/13 2:26 AM, Robert Kaiser wrote: David Rajchenbach-Teller schrieb: I'd even go as far as limiting it to 16kb. (possibly with a transition phase during which going above 16kb only prints warnings) I think most of us agree, but the problem is that apparently a number of add-ons rely on large prefs atm, so right now we did set to 1MB. Adding a warning for everything over 10KB or 16KB or something and targeting to move the limit down to that at some point would surely be a good idea, and I'd be happy about someone filing a bug and patch about this. Filed: https://bugzilla.mozilla.org/show_bug.cgi?id=872980 https://bugzilla.mozilla.org/show_bug.cgi?id=872981 Cheers, David -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Ordering shutdown observers?
Do you have use cases besides these two? In bug 845190 [1] we're seeing an issue which might be caused by an unexpected shutdown sequence. What seems to be happening is that a DOM worker thread is created very late during the test; once ShutdownXPCOM() is invoked the worker runtime service is supposed to tear down all remaining workers but in order to do so it has to wait for pending ones to finish loading and then stop. Unfortunately the said worker starts loading when many other subsystems have already been shut down (either by receiving a shutdown-xpcom or shutdown-xpcom-threads event) and trips over errors/assertions because of that. Gabriele [1] https://bugzilla.mozilla.org/show_bug.cgi?id=845190 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Ordering shutdown observers?
On 2013-05-16 7:30 AM, Gabriele Svelto wrote: Do you have use cases besides these two? In bug 845190 [1] we're seeing an issue which might be caused by an unexpected shutdown sequence. What seems to be happening is that a DOM worker thread is created very late during the test; once ShutdownXPCOM() is invoked the worker runtime service is supposed to tear down all remaining workers but in order to do so it has to wait for pending ones to finish loading and then stop. Unfortunately the said worker starts loading when many other subsystems have already been shut down (either by receiving a shutdown-xpcom or shutdown-xpcom-threads event) and trips over errors/assertions because of that. Is this not the OS.File issue that Vladan mentioned? My point is that there doesn't seem to be enough use cases to warrant a new infrastructure to handle shutdown dependencies. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Ordering shutdown observers?
On 5/15/13 11:32 PM, Gregory Szorc wrote: On 5/15/13 2:18 PM, Vladan Djeric wrote: I think the more compelling use case is service startup. Proper dependencies should allow us to more intelligently start services on demand. This should lead to lower resource utilization and faster startup times. Shutdown times should also speed up if there are fewer services to shut down. I agree, proper dependencies here would be a win. For Calendar/Lightning we have implemented our own startup service that starts up the calendar components in order. Having a startup/shutdown service would be very nice since we can wait for startup of important mail components, then load/unload our calendar components in order. Philipp ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Ordering shutdown observers?
On 5/16/13 3:03 PM, Ehsan Akhgari wrote: Is this not the OS.File issue that Vladan mentioned? My point is that there doesn't seem to be enough use cases to warrant a new infrastructure to handle shutdown dependencies. Well, as we expand our use of OS.File, we start observing a number of issues, most of which do not seem to be due to OS.File itself, but more generally to (chrome) workers. Here are a few: - clients of OS.File need to write their data before OS.File shuts down – that's Vladan's remark; - JS Workers (including OS.File's I/O worker) need to be properly initialized before shut down or to cancel themselves nicely upon shutdown – that's Gabriele's remark; - OS.File itself needs to be informed of shut down to (asynchronously) collect information and print warnings about leaking file descriptors, and also to start rejecting additional requests. That's from the top of my head, I am sure that I am missing a few. As we move as much code as possible to workers/threads, I believe that we are going to suffer from a growing number of such issues. So, yes, I am convinced that we need a way to handle dependencies. Moreover, I believe that we need to make dependencies somewhat explicit, otherwise we will at some point end up with unsatisfiable implicit dependencies and we will need large refactorings to get around these. Cheers, David -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Deferred display of XUL panel containing embedded iframe
Can't you just avoid calling openPopup until the page is loaded, and avoid messing with the panel's hidden state completely? Gavin On Thu, May 16, 2013 at 8:27 AM, Matthew Gertner matt...@salsitasoft.com wrote: I have a toolbar button that displays a XUL panel when pressed. The panel contains an iframe into which an HTML page is loaded. To avoid flicker, I'd like to defer display of the panel until the HTML page has finished loading (DOMContentLoaded). I tried setting the panel's hidden property to true in the popupshowing handler and then setting it back to true when the iframe has finished loading. The DOMContentLoaded handler for the iframe is called, but the panel remains invisible even after setting hidden back to false. The state of the panel is 'closed' at this point. It seems like the panel is automatically being closed when its hidden property is set to true. (The same happens when I set style.display to 'none' or collapsed to true). I nosed around the source code a bit but ended it quickly in the bowels of nsBoxObject.cpp. Before I start debugging the box object code, does anyone have any insights into why setting the hidden property of the panel's XUL element to true causes it to be closed (and thus remain invisible even when hidden is set back to false)? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Firefox Beta unable to load add-on with shared library built against XULRunner for Bets (xulrunner-22.0b1)
I have now tested with with Firefox-22 as beta and ot still does not work. Please see the attached message for the description of the problem. I think that the problem is related to mismatch between the shared libraries in XULRunner and the shared libraries in the Firefox-22 distribution. Firefox-Beta distribution does not have following libs: libnssutil3.dylib, libplc4.dylib libplds4.dylib libsmime3.dylib libnspr4.dylib libssl3.dylib When I build with source tree without XULRunner, the shared libraries matches with Firefox-22 distribution. It seems that all those missing libraries are merged with libnss3.dylib in Firefox-22 distribution. It looks like I cannot use the XULRunner to build my binary componenets. If I build with FF-22 source tree without xul-sdk would the binary components be compatible with FF-22? Thanks for your help. -- Subrata Original Message Subject: Firefox Aurora unable to load add-on with shared library built against XULRunner for Aurora (xulrunner-22.0a2) Date: Sat, 11 May 2013 11:40:40 -0400 From: Subrata Mazumdar subrata.mazum...@ieee.org Newsgroups: mozilla.dev.apps.firefox I am trying test my add-on (that has binary components) with Aurora (FF 22) built with xulrunner-22.0a2. I am testing on Mac OSX 10.7.5 (Darwin 11.4.2 x86_64). If I build the binary components with libraries in mozilla-source tree (i.e., without the --with-libxul-sdk option) option, Aurora app load the binary components of add-on properly and everything works fine. But, if I build the with libraries in xulrunner-22.0a2 (i.e., with the --with-libxul-sdk option), Aurora app fails to load the binary components of the add-on. I have no problem with loading the same binary components in FF 21 built with libraries for xulrunner-21. I have noticed that there are differences in contents of Application directory for FF-21 (Beta) and FF-22 (Aurora). Firefox-Beta (FF-21): (cd ~/Applications/FirefoxBeta.app/Contents/MacOS/; ls *.dylib) : Firefox-Beta libfreebl3.dylib*libnssckbi.dylib* libsmime3.dylib* libmozalloc.dylib* libnssdbm3.dylib* libsoftokn3.dylib* libmozglue.dylib*libnssutil3.dylib* libssl3.dylib* libmozsqlite3.dylib* libplc4.dylib* libxpcom.dylib libnspr4.dylib* libplds4.dylib* libnss3.dylib* libplugin_child_interpose.dylib* Firefox-Aurora (FF-22): (cd ~/Applications/FirefoxAurora.app/Contents/MacOS/; ls *.dylib) libfreebl3.dylib*libnss3.dylib* libplugin_child_interpose.dylib* libmozalloc.dylib* libnssckbi.dylib* libsoftokn3.dylib* libmozglue.dylib*libnssdbm3.dylib* Any suggestions how to fix this problem. Would this problem go away when FF22 moves to Firefox beta channel? Thanks. -- Subrata ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Firefox Beta unable to load add-on with shared library built against XULRunner for Bets (xulrunner-22.0b1)
On Thu, May 16, 2013 at 01:30:14PM -0400, Subrata Mazumdar wrote: I have now tested with with Firefox-22 as beta and ot still does not work. Please see the attached message for the description of the problem. I think that the problem is related to mismatch between the shared libraries in XULRunner and the shared libraries in the Firefox-22 distribution. Firefox-Beta distribution does not have following libs: libnssutil3.dylib, libplc4.dylib libplds4.dylib libsmime3.dylib libnspr4.dylib libssl3.dylib When I build with source tree without XULRunner, the shared libraries matches with Firefox-22 distribution. It seems that all those missing libraries are merged with libnss3.dylib in Firefox-22 distribution. It looks like I cannot use the XULRunner to build my binary componenets. If I build with FF-22 source tree without xul-sdk would the binary components be compatible with FF-22? That's bug 865655, and it likely will be fixed for beta2. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Ordering shutdown observers?
On 5/16/2013 8:12 PM, Ehsan Akhgari wrote: A dependency system for startup/shutdown isn't free. It's going to have runtime cost, and it's going to make the code more complicated. Given the fact that we've managed to get by just fine without one for years, I think we should continue to do that, and in the cases where you want specific ordering, just use hard-coded notifications (such as profile-before-change2 -- BTW, that's a terrible name!). As I understand, the original problem is that we instantiate a service for the first time too late. It then is not able to shut it self down using xpcom-shutodown notification, since it has been created from xpcom-shutodown call. IMO, enough would be to have XPCOM bool isAfterShutdown() flag somewhere. Services that depends on xpcom-shutdown notification would check for this flag and refuse to instantiate or init. Could also be integrated in the generic constructor somehow - constructors would express that instantiation may not happen after certain events of xpcom. -hb- ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Persistently storing build system state
As I wrote at [1] there are some scenarios where the build system and related tools would like to store persistent state. Some uses for this include: * Automatically recording previous build logs, compiler warnings, and test results. * Holding build environments (downloaded Mercurial extensions, downloaded or built toolchains, etc). * Global mach/mozconfig config files * Caching of state such as last bootstrap time, last Mercurial extension update check, etc. Some of these we can't do today because state in the object directory frequently gets lost (from clobbers) or is not shared among source directories. As a result, I'd argue that build system UX and productivity suffers because we can't tap this potential. I'd like to tap this potential. Since we can't store all state in the object directory or in the source directory, I'm proposing that at some time in the future the build system will automatically use ~/.mozbuild for persistent state storage. The location will be configurable via an environment variable or something, of course. And, when the directory is initially created, we can add a prompt or a long delay with a prominent notification message or something. Of course, things would be implemented such that multiple source directories and multiple object directories continue to work. Initially, I intend to utilize the state directory for holding a global mach config file and bootstrap state (so mach and/or the build system can prompt/notify you to rebootstrap every N days or something). Are there any objections or concerns? [1] http://gregoryszorc.com/blog/2013/05/13/mozilla-build-system-brain-dump/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Deferred display of XUL panel containing embedded iframe
On Thursday, May 16, 2013 7:20:33 PM UTC+2, Gavin Sharp wrote: Can't you just avoid calling openPopup until the page is loaded, and avoid messing with the panel's hidden state completely? I tried that but it seemed like the iframe didn't have a docShell until after the popup is shown so I couldn't load the document into it. If that seems strange then I'm probably doing something wrong (I only did a quick test earlier today). I'll have to investigate further. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Persistently storing build system state
On 13-05-16 12:53 PM, Gregory Szorc wrote: As I wrote at [1] there are some scenarios where the build system and related tools would like to store persistent state. Some uses for this include: * Automatically recording previous build logs, compiler warnings, and test results. * Holding build environments (downloaded Mercurial extensions, downloaded or built toolchains, etc). * Global mach/mozconfig config files * Caching of state such as last bootstrap time, last Mercurial extension update check, etc. Some of these we can't do today because state in the object directory frequently gets lost (from clobbers) or is not shared among source directories. As a result, I'd argue that build system UX and productivity suffers because we can't tap this potential. I'd like to tap this potential. Since we can't store all state in the object directory or in the source directory, I'm proposing that at some time in the future the build system will automatically use ~/.mozbuild for persistent state storage. The location will be configurable via an environment variable or something, of course. And, when the directory is initially created, we can add a prompt or a long delay with a prominent notification message or something. Of course, things would be implemented such that multiple source directories and multiple object directories continue to work. Initially, I intend to utilize the state directory for holding a global mach config file and bootstrap state (so mach and/or the build system can prompt/notify you to rebootstrap every N days or something). Are there any objections or concerns? I also think we should tap this potential. But: Some of the things you mention are truly global: mach configuration, hg extensions. But some things are more source-tree specific: build logs, compiler warnings, test results. I would not want to share too much state between my mozilla-inbound, mozilla-central, and services-central trees. I think making this distinction explicit will help: ~/.mozbuild and $SRCDIR/.mozbuild. Best, Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Visual C++ PGO linker memory usage
On 5/16/13 3:15 PM, Ted Mielczarek wrote: TL;DR: PGO on MSVC is now opt-in per-source-directory. This is awesome. Thank you, Ehsan and Ted, for getting this sorted out! -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Ordering shutdown observers?
Ehsan Akhgari wrote: On 2013-05-15 5:18 PM, Vladan Djeric wrote: I'd like to know if these use-cases are sufficiently rare that we should just add new shutdown events when needed (e.g. we added profile-before-change2 for Telemetry in bug 844331), or if we should come up with a general way to impose order of shutdown based on dependencies? Do you have use cases besides these two? Many things (and an increasing number) depend on PSM/NSS and the PSM team (a long time ago) implemented its own shutdown event registration scheme (nsNSSShutDownObject in nsNSSShutDown.h). There seems like there is at least one race due to NSS being shut down while things are still using NSS which is causing a crash or worse (presumably because there is not enough awareness of the need to implement nsNSSShutDownObject and/or it is too error-prone to do so). Also, NSS must be shut down in profile-before-change because it may write to the profile directory. So, basically the nsNSSShutDownObject scheme is a variant of the explicit dependencies scheme and, not a very successful one. Perhaps there are other variants of explicit dependency schemes that would be less error prone, but I am skeptical. In general, generic dependency schemes of the upstart variety seem like very complicated solutions, considering we have global knowledge of all the components of Firefox that we could just hard-code in, if we can assume that addons do not affect the ordering. Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Deferred display of XUL panel containing embedded iframe
Matthew Gertner wrote: It seems like the panel is automatically being closed when its hidden property is set to true. display: none; destroys the nsIFrame object (in this case an nsMenuPopupFrame), thus closing the panel. -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Forcing alphabetical order in moz.build files
On 4/16/13 10:15 PM, Robert O'Callahan wrote: I have a request ... can we require lists in moz.build files to be in alphabetical order, and actually enforce with some build-system check? I'm always annoyed by Makefiles where lists are sometimes unordered and it's hard to find items and know where to add items. This just landed in inbound. For variables where order isn't important (pretty much everything except *DIRS), every incoming list append or replacement must be sorted by byte order of the UTF-8 encoding of the string (typically strings are ASCII). If it isn't, you'll see an error message like: ERROR PROCESSING MOZBUILD FILE == The error occurred while processing the following file: /Users/gps/src/firefox/dom/devicestorage/moz.build The error was triggered on line 15 of this file: 'DeviceStorage.h', An error was encountered as part of executing the file itself. The error appears to be the fault of the script. The error as reported by Python is: ['UnsortedError: An attempt was made to add an unsorted sequence to a list. The incoming list is unsorted starting at element 0. We expected DeviceStorage.h but got nsDeviceStorage.h\n'] As you can see, it tells you exactly what it expects to aid you in fixing the problem. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
