Jason Evans wrote: > Christopher Blizzard wrote: >> I had some questions about embedding + jemalloc. We made a change >> recently to link jemalloc directly into libxul instead of the app >> (where app == xulrunner, firefox-bin, TestGtkEmbed, etc.) See this bug: >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=418016 > > In retrospect, I think that change was a mistake. Stuart and Benjamin > came up with a better solution, which one of them hinted at somewhere in > bugzilla (can't find it at the moment): link a libjemalloc[1] with each > and every application that links with libxul.
I think that's the way that it used to work. :) I had a patch to the GTK embedding widget test harness to link it with jemalloc for testing and I think it was just a question of going through and adding flags where required. Also a perfectly valid question: how many apps will want to worry about the savings that jemalloc provides? Many long-running apps (like a browser?) Memory-sensitive apps (like mobile apps?) We'll need to have a guide here. Some apps probably won't care that much, but many will. Do we have decent measurements of the benefits of jemalloc as part of of the larger picture of our memory improvements leading up to FF3? Some of it was fixing allocations but a lot of it came from a well-tuned allocator. Just wondering what we should tell people here. > >> So I have a concern about this from an embedding standpoint. If we're >> using jemalloc's allocator instead of the normal system one, how will >> this affect apps that want to link against libxul or the library that >> we provide for embedding? Specifically: >> >> 1. If the jemalloc allocator gets loaded, does it change the allocator >> used by the app? If so, does that happen after the app has done some >> allocations and might cause mis-matched malloc/free pairs? > > As things currently work, if libxul is a load-time library dependency > for the application, then having jemalloc embedded is fine, because > jemalloc will override the system malloc before any allocations happen. > > However, my understanding is that that some applications use dlopen(3) > to load libxul on the fly, which causes horrible problems if jemalloc is > part of libxul. This is why I think we need to make jemalloc a separate > library, and also require any application that dlopen()s libxul to be > linked against libjemalloc. This is a rather onerous requirement, but I > can't think of any reasonable alternative, if jemalloc is to be used at > all. (Incidentally, it shouldn't be hard to programmatically enforce > this requirement.) There are some applications that will do that, sure. And that sure would be some bad mojo. I suspect that given this and the fact that we're going down the route of having good support for embedding that we should revisit how we're linking in the jamalloc code so that we get the cases right. Ben, do you have any issue with that? It's also important to note that the original bug in question was a performance bug. Can anyone think of a way we can avoid the perf problem that the change was originally designed to avoid? > >> 2. If the jemalloc allocator is loaded and is private to libxul, does >> this mean that embedding applications that are passed data that's >> allocated inside of libxul will have to use a xul-private free() >> function? > > If jemalloc were used privately by libxul (jemalloc not exposed), then > embedding applications would have to use some special deallocation API > in order to work with objects that originate from within libxul. > Additionally, libxul would either have to copy externally allocated > objects to internally allocated memory, or jump through some other > hoops, like checking where objects came from in key places. This mixed > allocator model would be pretty bad from a fragmentation (and probably > speed) perspective. Yeah, uhh, ick. > > I was planning to hack on the build system tomorrow to make this happen, > so if there's something unworkable about the solution, this would be a > great time for someone to say so. =) > > Jason > > [1] There must be a better name that libjemalloc. libxulmm, "xul memory > management library", maybe? What, you don't want the credit you so richly deserve? :D --Chris _______________________________________________ dev-embedding mailing list [email protected] https://lists.mozilla.org/listinfo/dev-embedding
