Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
Hi On Tuesday March 18 2014 08:54:04 James Linder wrote: > Somewhat OT so would you mind mailing me directly (unless of interest here) I think it is (for any developer ...) > So … what did you do and how? I don't have the exact URLs at hand, but there is a description out there on how to get VMWare or Parallels to accept SL non-server. Basically, there has to be a file identifying the system as SL Server. The file can be removed after booting, and a pre-shutdown script puts it back. Or something like that. Removing it is less important now that there are hardly ASUs left, otherwise you'd get the SL Server updates. So what I did was - clone my boot disk to a dmg with Carbon Copy Cloner (CCC), skipping the largest things I wanted to skip to keep the image "small". - convert that dmg to R/W so that I could apply the hacks to turn SL into SL Server at boot - convert that new dmg to a VM image using VirtualBox's VBoxManage tool. I think I went with a .vdi - It is likely that I first created a guest in VirtualBox. But since it has no OS X guest additions, I got a copy of Parallels (VMWare requires 10.7+) and imported the VM in there. That was about all. Takes some time and you need quite a bit of disk space to jungle with, but it worked out fine. The only quirk is that my guest sees 4GB of RAM instead of the 2.5GB I gave it, but maybe that's because the host runs 10.6.8 ... Getting 10.9 into that VM was easy. I cloned my boot disk to an external, and updated that disk to 10.9. Once that was more or less settled (10.9.1 started making a local TimeMachine copy without asking or giving feedback, which of course took ages), I started the 10.6 VM. Parallels has this nice feature that it can mount external devices to the guest when you connect them, so I got my 10.9 boot disk on the VM's desktop. I'd given it a new harddisk before, so all I had to do was start CCC and clone the external 10.9 disk onto the new VM image, and then set to boot the guest off that. A bit surprisingly that just worked -- and I can even clone the VM back onto the external. And that gives us a very easy way to prepare OS X VM images ... I get that "Don't Steal OS X" kext or whatever it is when I boot the VM, but I consider that I own enough SL licenses to run 2 copies on Apple hardware to be fine legally ... and 10.9 can be run in a VM without the Server version hack. Before I forget: Paragon Software have a tool for making VM images off an existing set-up. I haven't tried it yet, though. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On 18 Mar 2014, at 3:00 am, macports-users-requ...@lists.macosforge.org wrote: >> Well yes, of course. I was thinking as much as a test, to see if its the >> optimisations that are the issue, as it is for me, than as a solution. In my >> case i simply could not compile at all without -O0... without it, the memory >> usage hit around 9GB before being killed by our nightly regression testing >> framework. If your case is not so bad and you can get by, then fine >> >>> >>> Anyway, on #calligra someone just claimed he got about 1200MB peak memory >>> usage building gmic.cpp with Apple's clang-3.4 . If someone else can >>> confirm that the issue is rather moot (meaning I can continue to use g++ >>> for just this file on 10.6 ... ) >>> ___ > > I've just been testing this a bit on my 10.9.2 VM with the latest Xcode (5.1, > clang 3.4svn) installed. That VM has only 2.5GB of RAM (which it sees as 4GB, > curiously). Still, gmic.cpp compiles much better with clang++ on that set-up > (haven't tested clang-mp-3.3) though g++ still uses less memory (1.2 vs 1.4GB > real peak) and completes about twice as quick. However, things go downhill > again with clang 3.5 (from MacPorts). I've seen a peak real mem. usage of > 1.75GB, and it is now sitting at <10%CPU with around 200MB real memory, but > with "red" memory pressure and 1.72GB total compressed memory. > > If you feel like filing reports against a current clang version, you might > want to check the 3.5 version on your code... Somewhat OT so would you mind mailing me directly (unless of interest here) I’ve tried a few times to install a VM on my iMac 27. I dont recall detail but every install failed with VirtualBox *except* the hacked snow leopard install that I can install on any hardware … Snow_Leopard_Client_Server_10.6.2_SSE2_SSE3_Intel_AMD_by_Hazard.iso So … what did you do and how? Thanks James ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
Hi, On Mar 16, 2014, at 18:05, Chris Jones wrote: > Well yes, of course. I was thinking as much as a test, to see if its the > optimisations that are the issue, as it is for me, than as a solution. In my > case i simply could not compile at all without -O0... without it, the memory > usage hit around 9GB before being killed by our nightly regression testing > framework. If your case is not so bad and you can get by, then fine > >> >> Anyway, on #calligra someone just claimed he got about 1200MB peak memory >> usage building gmic.cpp with Apple's clang-3.4 . If someone else can confirm >> that the issue is rather moot (meaning I can continue to use g++ for just >> this file on 10.6 ... ) >> ___ I've just been testing this a bit on my 10.9.2 VM with the latest Xcode (5.1, clang 3.4svn) installed. That VM has only 2.5GB of RAM (which it sees as 4GB, curiously). Still, gmic.cpp compiles much better with clang++ on that set-up (haven't tested clang-mp-3.3) though g++ still uses less memory (1.2 vs 1.4GB real peak) and completes about twice as quick. However, things go downhill again with clang 3.5 (from MacPorts). I've seen a peak real mem. usage of 1.75GB, and it is now sitting at <10%CPU with around 200MB real memory, but with "red" memory pressure and 1.72GB total compressed memory. If you feel like filing reports against a current clang version, you might want to check the 3.5 version on your code... R. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
Hi, > On 16 Mar 2014, at 10:34 pm, René J.V. Bertin wrote: > >> On Sunday March 16 2014 19:44:14 Christopher Jones wrote: >> >>> On 16 Mar 2014, at 7:14pm, René J.V. Bertin wrote: >> >> >> Its different because Apple, because of the above GPL3+ issue, will not >> provide either gcc release that uses libc++, or a clang release that uses >> libstdc++. Linux, being GPL3+ friendly is happy to do the work, so on recent >> Linux systems you get up to date clang and gcc releases that a) both support >> c++11 and b) can be intermixed, as both use the same c++ runtime. > > But both will use libstdc++ - because a Linux version of GCC that supports > clang's c++11 could surely be built on OS X too? In theory yes. In practise, it will will require someone to step up to the mark to provide it. And that someone will not be anyone associated to Apple in any way, as they cannot touch anything GPL3. Honestly, i don't really see the point. Better to just support clang (and provide bug reports to fix issues such as the one to hand.) > >> The standard (correctly) has absolutely nothing to do with specific >> implementations of that standard. The ‘issue’ is Apple will not touch >> anything released under GPL3+, and the most recent c++11 supporting version >> of libstdc++ is released under this. As such you will never get it on OSX, >> and thus the fact that in order to support c++11 on OSX you must use libc++. > > Not just C++11 if I understood correctly, but every dialect and flavour of > C++. I knew there was a reason I hung on to pure C for so long ^^ > > R. > > ___ > macports-users mailing list > macports-users@lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/macports-users ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Sunday March 16 2014 19:44:14 Christopher Jones wrote: > > On 16 Mar 2014, at 7:14pm, René J.V. Bertin wrote: > > Its different because Apple, because of the above GPL3+ issue, will not > provide either gcc release that uses libc++, or a clang release that uses > libstdc++. Linux, being GPL3+ friendly is happy to do the work, so on recent > Linux systems you get up to date clang and gcc releases that a) both support > c++11 and b) can be intermixed, as both use the same c++ runtime. But both will use libstdc++ - because a Linux version of GCC that supports clang's c++11 could surely be built on OS X too? > The standard (correctly) has absolutely nothing to do with specific > implementations of that standard. The ‘issue’ is Apple will not touch > anything released under GPL3+, and the most recent c++11 supporting version > of libstdc++ is released under this. As such you will never get it on OSX, > and thus the fact that in order to support c++11 on OSX you must use libc++. Not just C++11 if I understood correctly, but every dialect and flavour of C++. I knew there was a reason I hung on to pure C for so long ^^ R. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On 16 Mar 2014, at 7:14pm, René J.V. Bertin wrote: > > On Mar 16, 2014, at 19:49, Brandon Allbery wrote: >> http://trac.macports.org/wiki/FAQ#libcpp >> >> Linux gets around this by forcing everything to the new runtime; Apple will >> not ship GPL3 stuff so stuck to older gcc with the older runtime, until they >> moved to clang completely in 10.9 and switched to clang's C++11 runtime. >> (gcc/g++'s C++11 runtime is GPL3+ only.) > > But how can the situation be any different if system libraries use either the > GNU C++11 runtime or the clang one? Its different because Apple, because of the above GPL3+ issue, will not provide either gcc release that uses libc++, or a clang release that uses libstdc++. Linux, being GPL3+ friendly is happy to do the work, so on recent Linux systems you get up to date clang and gcc releases that a) both support c++11 and b) can be intermixed, as both use the same c++ runtime. > >> Hi, "C++11" refers to a standard defined by a standards committee. The >> problem is a *licensing* issue, and Apple made its choice in that matter and >> nobody can do anything about it. > > Sigh ... something like "or how lawyers can mess up perfectly good > standards"? Or does the standard not cover the ABI? The standard (correctly) has absolutely nothing to do with specific implementations of that standard. The ‘issue’ is Apple will not touch anything released under GPL3+, and the most recent c++11 supporting version of libstdc++ is released under this. As such you will never get it on OSX, and thus the fact that in order to support c++11 on OSX you must use libc++. Chris smime.p7s Description: S/MIME cryptographic signature ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Mar 16, 2014, at 19:49, Brandon Allbery wrote: > http://trac.macports.org/wiki/FAQ#libcpp > > Linux gets around this by forcing everything to the new runtime; Apple will > not ship GPL3 stuff so stuck to older gcc with the older runtime, until they > moved to clang completely in 10.9 and switched to clang's C++11 runtime. > (gcc/g++'s C++11 runtime is GPL3+ only.) But how can the situation be any different if system libraries use either the GNU C++11 runtime or the clang one? > > Hi, "C++11" refers to a standard defined by a standards committee. The > problem is a *licensing* issue, and Apple made its choice in that matter and > nobody can do anything about it. Sigh ... something like "or how lawyers can mess up perfectly good standards"? Or does the standard not cover the ABI? R. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Sun, Mar 16, 2014 at 11:33 AM, René J.V. wrote: > On Sunday March 16 2014 09:56:47 Ryan Schmidt wrote:> clang 3.5 and later > require C++11, and will say so if you try to install them on a system > without C++11. Effectively, this means clang 3.5 and later require OS X > 10.9 Mavericks or later. > > Hmmm, I thought clang supported C++11 ; I presume you're referring to a > system-dependent C++11 runtime that is not (or cannot) be provided by clang > itself? Out of curiosity, how does this work on Linux? > Apple provides, and ships libraries that require, specific runtimes. You could build the C++11 runtime yourself, but you will get link errors with Apple libraries that use C++ because they only work with the old runtime. (And conversely on 10.9, Apple libraries require the C++11 runtime and you can't use them with stuff requiring the older one.) http://trac.macports.org/wiki/FAQ#libcpp Linux gets around this by forcing everything to the new runtime; Apple will not ship GPL3 stuff so stuck to older gcc with the older runtime, until they moved to clang completely in 10.9 and switched to clang's C++11 runtime. (gcc/g++'s C++11 runtime is GPL3+ only.) I hate it when generic programming languages or their features become > dependent on an OS. I can more or less accept that for ObjC because like > .Net it is so closely related to an OS-specific GUI framework. But much > less so for C++ ... isn't there some kind of standards committee that could > avoid this from happening? Hi, "C++11" refers to a standard defined by a standards committee. The problem is a *licensing* issue, and Apple made its choice in that matter and nobody can do anything about it. -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
Hi, > On 16 Mar 2014, at 04:55 pm, "René J.V. Bertin" wrote: > >> On Mar 16, 2014, at 17:23, Christopher Jones wrote: >> >> Try -O0 instead of just -O. Not quite the same and the former worked for me… > > But that can also make a huge difference on performance ... rather than > providing a gmic extension that's unbearably slow, just as well not provide > it at all ;) Well yes, of course. I was thinking as much as a test, to see if its the optimisations that are the issue, as it is for me, than as a solution. In my case i simply could not compile at all without -O0... without it, the memory usage hit around 9GB before being killed by our nightly regression testing framework. If your case is not so bad and you can get by, then fine > > Anyway, on #calligra someone just claimed he got about 1200MB peak memory > usage building gmic.cpp with Apple's clang-3.4 . If someone else can confirm > that the issue is rather moot (meaning I can continue to use g++ for just > this file on 10.6 ... ) > ___ > macports-users mailing list > macports-users@lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/macports-users ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Mar 16, 2014, at 17:23, Christopher Jones wrote: > Try -O0 instead of just -O. Not quite the same and the former worked for me… But that can also make a huge difference on performance ... rather than providing a gmic extension that's unbearably slow, just as well not provide it at all ;) Anyway, on #calligra someone just claimed he got about 1200MB peak memory usage building gmic.cpp with Apple's clang-3.4 . If someone else can confirm that the issue is rather moot (meaning I can continue to use g++ for just this file on 10.6 ... ) ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
> I tried -O instead of -O2, didn't help. And not very surprising if indeed > it's the use of templates that's the cause of it all… Try -O0 instead of just -O. Not quite the same and the former worked for me… Chris smime.p7s Description: S/MIME cryptographic signature ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Sunday March 16 2014 09:56:47 Ryan Schmidt wrote: >This was an older version of clang, on older hardware with limited memory; I >haven’t noticed any such problems on my new machine which has gobs of memory >and the current versions of things. Not even when compiling the gmic port? I don't know how OS X (10.6) limits VM usage, and from what I hear 10.9's virtual memory subsystem has gotten substantially better, but comparing the total scores on my OS X and Linux systems I have the impression that clang could have taken even more memory if it had been able to (with 20GB swap taken I have only 20GB left on my OS X root partition). > Whether or not MacPorts uses the -pipe flag is controlled by the > configurepipe setting in macports.conf. If changing it to no makes the build > work better, please let us know. We changed the default of that setting from > no to yes several years ago, presumably because we thought it would perform > better, but I think that was before we were using clang. This is becoming difficult to assess, because recent documentation suggests -fno-pipe is required to deactivate the behaviour ... and I don't seem to have that flag. In any case, I *think* that on a multi-core SMP system where you use -jN to adapt the number of builder threads/processes to the number of cores, -fno-pipe makes more sense ... esp. if you're working on an SSD where the difference between file IO and pipe IO must be minimal. I'll check my setting, though. It's been awhile already that I've seen messages about emergency paging appear in the kernel.log when I'm updating ports from source. > clang 3.5 and later require C++11, and will say so if you try to install them > on a system without C++11. Effectively, this means clang 3.5 and later > require OS X 10.9 Mavericks or later. Hmmm, I thought clang supported C++11 ; I presume you're referring to a system-dependent C++11 runtime that is not (or cannot) be provided by clang itself? Out of curiosity, how does this work on Linux? I hate it when generic programming languages or their features become dependent on an OS. I can more or less accept that for ObjC because like .Net it is so closely related to an OS-specific GUI framework. But much less so for C++ ... isn't there some kind of standards committee that could avoid this from happening? ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Sunday March 16 2014 14:34:00 Christopher Jones wrote: > What OSX version are you running ? 3.4 and 3.5 install just fine for me on > OSX 10.9… No idea if clang 3.4 or 3.5 are supposed to work on older OSX > releases (I know the converse has problems, clang versions older than 3.3 do > not install on OSX 10.9). 10.6.8 . > There will be little point filing a bug report against clang 3.3. The first > thing they will ask is if the issue is still there with a newer release. > Hence my point above…. As I said, the issue exists with clang 3.4 on Linux too. I've had a chance to time compilation of the same file with gcc 4.8: 18.5 minutes with near 100% CPU. In other words, 3-4 times faster (clang didn't even reach 50%CPU on the same rig). The file is huge as I said, nearly 13000 lines with heavy use of templates. It's an adapted version of the file at http://gmic.sourceforge.net/ from what I understand. I haven't tried, but that project is small enough to make a good testcase. > The solution I have used so for is for the file in question, when compiling > with clang to disable compiler optimisations, which seem to be the issue. > i.e. use -O0 instead of -O2… I tried -O instead of -O2, didn't help. And not very surprising if indeed it's the use of templates that's the cause of it all... R. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Mar 16, 2014, at 05:14, René J.V. Bertin wrote: > I use clang to avoid C++ lib clashes on recent OS X versions, MacPort's own > 3.3 (because 3.0 has known issues), and it is apparently not quite efficient > in its memory usage. There's 1 file in particular, gmic.cpp, that sees VM > usage grow to about 20GB at one point, which is never released and may > explain why I've always had to kill clang after upwards of 30 minutes of > processing (when it in fact appeared to be almost idle). > GCC (MacPorts 4.7) has no such issues, takes less than 2.5GB of memory and > terminates in 3-4 minutes to produce a 16MB object file. My Calligra binaries > thus run with a gmic module compiled with g++ ... > > This is not just an OS X issue; I've tried compiling the file with the same > options under Linux, and a similar thing happens. Except that on my Linux box > with its hard-wired 10GB of swap, VM usage remains limited to about 7GB. > > What do we know about the way clang handles its internal memory allocations > and about possible options to limit that? I too have seen clang take an extraordinary amount of memory when compiling certain ports, to the extent that I had to disable parallel building to limit the number of clang processes, and clear off many gigs of disk space to make room for swap. I do not know of any options to change that behavior. I do not know how to file a meaningful bug report about this with the developers of clang. This was an older version of clang, on older hardware with limited memory; I haven’t noticed any such problems on my new machine which has gobs of memory and the current versions of things. On Mar 16, 2014, at 06:03, René J.V. Bertin wrote: > Heck, it even looks as if clang uses a single process that handles all the > subsequent steps for which traditional compilers use different applications > (and it's my experience that -pipe is rarely a good idea for big projects > with parallel building). Thus, if it's the preprocessor that needs those 20GB > of memory, the footprint continues to hurt performance of all subsequent > steps, which would explain why I see a clang process that's sitting at 5-15% > CPU usage for such a long time. Very annoying if the rest of the machine > grinds to an almost halt when it's basically doing nothing! Whether or not MacPorts uses the -pipe flag is controlled by the configurepipe setting in macports.conf. If changing it to no makes the build work better, please let us know. We changed the default of that setting from no to yes several years ago, presumably because we thought it would perform better, but I think that was before we were using clang. On Mar 16, 2014, at 09:34, Christopher Jones wrote: > What OSX version are you running ? 3.4 and 3.5 install just fine for me on > OSX 10.9… No idea if clang 3.4 or 3.5 are supposed to work on older OSX > releases (I know the converse has problems, clang versions older than 3.3 do > not install on OSX 10.9). clang 3.5 and later require C++11, and will say so if you try to install them on a system without C++11. Effectively, this means clang 3.5 and later require OS X 10.9 Mavericks or later. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
Hi, On 16 Mar 2014, at 11:03am, René J.V. Bertin wrote: > On Mar 16, 2014, at 11:51, Christopher Jones wrote: > >> Any reason you are using clang 3.3 though ? Have you tried a newer clang >> version. Macports Clang 3.4 or 3.5 compilers ? I was unable to do this for >> my issue, 3.3 was the only compiler I had available, but I would be very >> interested to know if the issue goes away in a newer release. > > For some reason I haven't been able to install MacPorts clang-3.4 yet, and > when I built from source a while ago it didn't work. But the test I did under > Linux was with clang-3.4, so I guess we're dealing with an issue that's > inherent to clang here. It tooks about 1.5 hours to compile there, but > without a comparison of the g++ numbers that's just an indication that an AMD > C60 netbook is a slug :) What OSX version are you running ? 3.4 and 3.5 install just fine for me on OSX 10.9… No idea if clang 3.4 or 3.5 are supposed to work on older OSX releases (I know the converse has problems, clang versions older than 3.3 do not install on OSX 10.9). >> Won’t work. There are many more differences than just resolving libraries at >> the link step (Differences in the API of std::string classes for instance). >> You have to compile *and* link against libc++, using the same compiler, >> which means clang. > > Damn. Fortunately I just learned that gmic isn't central at all to Calligra > (or even to its Krita component), so the easy way out in this particular case > is to disable it when clang is used. But maybe one ought to file a report > with Apple or the clang project, as I'm sure there are other cases where > compiler memory usage become a serious obstacle. > Heck, it even looks as if clang uses a single process that handles all the > subsequent steps for which traditional compilers use different applications > (and it's my experience that -pipe is rarely a good idea for big projects > with parallel building). Thus, if it's the preprocessor that needs those 20GB > of memory, the footprint continues to hurt performance of all subsequent > steps, which would explain why I see a clang process that's sitting at 5-15% > CPU usage for such a long time. Very annoying if the rest of the machine > grinds to an almost halt when it's basically doing nothing! What does the code in question look like here ? I know the issue I have is with auto-generated code, that is frankly pretty hideous (10k+ lines of heavily ifdef’ed stuff). There will be little point filing a bug report against clang 3.3. The first thing they will ask is if the issue is still there with a newer release. Hence my point above…. I might see if I can form a simple test case for the code I have problems with sometime… The solution I have used so for is for the file in question, when compiling with clang to disable compiler optimisations, which seem to be the issue. i.e. use -O0 instead of -O2… cheers smime.p7s Description: S/MIME cryptographic signature ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
On Mar 16, 2014, at 11:51, Christopher Jones wrote: > Any reason you are using clang 3.3 though ? Have you tried a newer clang > version. Macports Clang 3.4 or 3.5 compilers ? I was unable to do this for my > issue, 3.3 was the only compiler I had available, but I would be very > interested to know if the issue goes away in a newer release. For some reason I haven't been able to install MacPorts clang-3.4 yet, and when I built from source a while ago it didn't work. But the test I did under Linux was with clang-3.4, so I guess we're dealing with an issue that's inherent to clang here. It tooks about 1.5 hours to compile there, but without a comparison of the g++ numbers that's just an indication that an AMD C60 netbook is a slug :) > > Won’t work. There are many more differences than just resolving libraries at > the link step (Differences in the API of std::string classes for instance). > You have to compile *and* link against libc++, using the same compiler, which > means clang. Damn. Fortunately I just learned that gmic isn't central at all to Calligra (or even to its Krita component), so the easy way out in this particular case is to disable it when clang is used. But maybe one ought to file a report with Apple or the clang project, as I'm sure there are other cases where compiler memory usage become a serious obstacle. Heck, it even looks as if clang uses a single process that handles all the subsequent steps for which traditional compilers use different applications (and it's my experience that -pipe is rarely a good idea for big projects with parallel building). Thus, if it's the preprocessor that needs those 20GB of memory, the footprint continues to hurt performance of all subsequent steps, which would explain why I see a clang process that's sitting at 5-15% CPU usage for such a long time. Very annoying if the rest of the machine grinds to an almost halt when it's basically doing nothing! ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
Hi, On 16 Mar 2014, at 10:14am, René J.V. Bertin wrote: > As mentioned elsewhere, I've just spent several days trying to get the > Calligra suite to build against KDE (and other dependencies) through MacPorts > (and into /opt/local for those ports not available via MacPorts, like Vc and > libetonyek). > I've file a review request (https://git.reviewboard.kde.org/r/116816/) and > I'm happy to report that after the last changes pushed to Calligra/Master > yesterday a full rebuild took less than 2h using -j3 on my i7 MBP running > 10.6.8 (and building on a ZFS filesystem, at that :) ). > > There's one catch, though. I use clang to avoid C++ lib clashes on recent OS > X versions, MacPort's own 3.3 (because 3.0 has known issues), and it is > apparently not quite efficient in its memory usage. There's 1 file in > particular, gmic.cpp, that sees VM usage grow to about 20GB at one point, > which is never released and may explain why I've always had to kill clang > after upwards of 30 minutes of processing (when it in fact appeared to be > almost idle). > GCC (MacPorts 4.7) has no such issues, takes less than 2.5GB of memory and > terminates in 3-4 minutes to produce a 16MB object file. My Calligra binaries > thus run with a gmic module compiled with g++ ... > > This is not just an OS X issue; I've tried compiling the file with the same > options under Linux, and a similar thing happens. Except that on my Linux box > with its hard-wired 10GB of swap, VM usage remains limited to about 7GB. > > What do we know about the way clang handles its internal memory allocations > and about possible options to limit that? I have seen exactly the same as you in a linux build, when using Clang 3.3. Any reason you are using clang 3.3 though ? Have you tried a newer clang version. Macports Clang 3.4 or 3.5 compilers ? I was unable to do this for my issue, 3.3 was the only compiler I had available, but I would be very interested to know if the issue goes away in a newer release. > > Also, I understand why GCC cannot be the default compiler to handle the full > build process on 10.8 or 10.9 and later. But to what extent would it be > possible to resolve the library clash by using clang++ for the link stage > when g++ is used for compilation? On 10.6 that appears to be possible without > any particular issues but I haven't yet been able to verify this on 10.9 . > Thoughts? Won’t work. There are many more differences than just resolving libraries at the link step (Differences in the API of std::string classes for instance). You have to compile *and* link against libc++, using the same compiler, which means clang. Chris > ___ > macports-users mailing list > macports-users@lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/macports-users smime.p7s Description: S/MIME cryptographic signature ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
clang memory usage vs. gcc (and OS X 10.8, 10.9, ...)
As mentioned elsewhere, I've just spent several days trying to get the Calligra suite to build against KDE (and other dependencies) through MacPorts (and into /opt/local for those ports not available via MacPorts, like Vc and libetonyek). I've file a review request (https://git.reviewboard.kde.org/r/116816/) and I'm happy to report that after the last changes pushed to Calligra/Master yesterday a full rebuild took less than 2h using -j3 on my i7 MBP running 10.6.8 (and building on a ZFS filesystem, at that :) ). There's one catch, though. I use clang to avoid C++ lib clashes on recent OS X versions, MacPort's own 3.3 (because 3.0 has known issues), and it is apparently not quite efficient in its memory usage. There's 1 file in particular, gmic.cpp, that sees VM usage grow to about 20GB at one point, which is never released and may explain why I've always had to kill clang after upwards of 30 minutes of processing (when it in fact appeared to be almost idle). GCC (MacPorts 4.7) has no such issues, takes less than 2.5GB of memory and terminates in 3-4 minutes to produce a 16MB object file. My Calligra binaries thus run with a gmic module compiled with g++ ... This is not just an OS X issue; I've tried compiling the file with the same options under Linux, and a similar thing happens. Except that on my Linux box with its hard-wired 10GB of swap, VM usage remains limited to about 7GB. What do we know about the way clang handles its internal memory allocations and about possible options to limit that? Also, I understand why GCC cannot be the default compiler to handle the full build process on 10.8 or 10.9 and later. But to what extent would it be possible to resolve the library clash by using clang++ for the link stage when g++ is used for compilation? On 10.6 that appears to be possible without any particular issues but I haven't yet been able to verify this on 10.9 . Thoughts? ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
