Re: scope of configure.optflags, configure.compiler etc. command line variables
On Friday September 20 2019 18:27:49 Ryan Schmidt wrote: >It would seem that making `port clean` delete the ccache caches would defeat >all current use cases of it. Not necessarily. AFAIK build commands under MacPorts almost always use full paths to source files because of the way configure and cmake are called, and I'm not convinced that ccache has any use if you just build ports during upgrades (i.e. when paths have changed). Ccache is clever, but not that clever. But this is why I added the suggestion to make ccache cleaning part of clean --all or even of a new clean option. It would be good anyway to have a way to liberate that cache, which is not the case currently (AFAIK). R.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Sep 20, 2019, at 02:23, René J.V. Bertin wrote: >> Storing a separate cache for each port would quickly eat up a lot of disk >> space. > > Evidently, but > - ccache doesn't use more space than needed > - the ccache directory compresses extremely well with afsctool, a background > cleanup process could exploit that > - per-port caches could be removed with `port clean` (automatic if they're in > the work directory) or `port clean --all` (or via a dedicated option). This > would require using the -o and -k flags as long as you plan to profit from > ccache benefits, but that probably covers most of the situations anyway. It would seem that making `port clean` delete the ccache caches would defeat all current use cases of it. Anyway, I'm going to leave the discussion now.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Thursday September 19 2019 20:20:06 Ryan Schmidt wrote: > > You could increase the size of your cache. > >Storing a separate cache for each port would quickly eat up a lot of disk >space. Evidently, but - ccache doesn't use more space than needed - the ccache directory compresses extremely well with afsctool, a background cleanup process could exploit that - per-port caches could be removed with `port clean` (automatic if they're in the work directory) or `port clean --all` (or via a dedicated option). This would require using the -o and -k flags as long as you plan to profit from ccache benefits, but that probably covers most of the situations anyway. >The level of fiddling that you've mentioned doing with the command line flags >exceeds what I've heard of any other developer doing. So I don't think that >writing and debugging a lot of code to change how this works would end up >benefiting very many people. Possibly, but there are probably very few who manage as many ports as regularly as I do, by now. But I'll plead guilty to using MacPorts as a nice way to formalise builds of software I use regularly so those builds become reproduceable and I can easily roll back a version in case of trouble (but I guess that's how others got into maintaining ports, too). And you probably don't want to know the complexity of compiler (and make) wrapper scripts that I've written in the past :) R.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Sep 17, 2019, at 03:26, René J.V. Bertin wrote: > On Monday September 16 2019 16:26:12 Ryan Schmidt wrote: > >> There should be no need for you to specify anything ccache-related on the >> command line. You can enable ccache globally in macports.conf. Individual >> ports can disable it if they are not compatible with it. > > Yes, I realised that after sending my message. And then I remembered why I > don't use it: it doesn't make sense for ports that you only build once in a > while without any significant chance that you'll be rebuilding it again soon. > GIven that the ccache cache is limited in size, using it pointlessly for too > many or too big builds means you're flushing useful data from the cache. > You'd need to use a port-specific ccache directory to prevent this issue, > probably under control of a variable like distdir, or simply a token to store > that cache *in* the distdir, so that related (sub)ports can share a cache. > I'm not even certain if I build more ports with ccache than without. What I > do know is that I when I do build a bunch of ports in a single go those are > usually related and so will be done either with or without using ccache. You have a point that the cache probably gets blown away before it can get used again, if you build a lot of different ports and your cache size isn't large enough to hold them all. You could increase the size of your cache. Storing a separate cache for each port would quickly eat up a lot of disk space. > One could also think of some sort of a developer's option to the port driver > which toggles certain options for port maintainer convenience and has a scope > like -o or -k. Ideally that option would allow for profiles, but even I > cannot really justify that added complexity ;) The level of fiddling that you've mentioned doing with the command line flags exceeds what I've heard of any other developer doing. So I don't think that writing and debugging a lot of code to change how this works would end up benefiting very many people.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Sep 17, 2019, at 03:08, René J.V. Bertin wrote:
> I continue to think that compiler choice and options should be something the
> user can set the default for, without having to replace the standard system
> compiler drivers (/usr/bin/clang{,++}) with wrapper scripts.
As you know, we don't want the user to do that. Allow MacPorts to pick the
compiler for a port. If it picks wrong for a particular port, fix the port by
updating its compiler.blacklist setting.
Re: scope of configure.optflags, configure.compiler etc. command line variables
-Wl,-no_pie Is the linker option you were looking for. K
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Monday September 16 2019 16:26:12 Ryan Schmidt wrote: >There should be no need for you to specify anything ccache-related on the >command line. You can enable ccache globally in macports.conf. Individual >ports can disable it if they are not compatible with it. Yes, I realised that after sending my message. And then I remembered why I don't use it: it doesn't make sense for ports that you only build once in a while without any significant chance that you'll be rebuilding it again soon. GIven that the ccache cache is limited in size, using it pointlessly for too many or too big builds means you're flushing useful data from the cache. You'd need to use a port-specific ccache directory to prevent this issue, probably under control of a variable like distdir, or simply a token to store that cache *in* the distdir, so that related (sub)ports can share a cache. I'm not even certain if I build more ports with ccache than without. What I do know is that I when I do build a bunch of ports in a single go those are usually related and so will be done either with or without using ccache. One could also think of some sort of a developer's option to the port driver which toggles certain options for port maintainer convenience and has a scope like -o or -k. Ideally that option would allow for profiles, but even I cannot really justify that added complexity ;) R.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Tuesday September 17 2019 11:49:11 Joshua Root wrote:
I continue to think that compiler choice and options should be something the
user can set the default for, without having to replace the standard system
compiler drivers (/usr/bin/clang{,++}) with wrapper scripts.
>machine. But -mdynamic-no-pic is incompatible with PIE, which is the
>default since 10.7.
I use -mdynamic-no-pic all the time for almost all my KF5 builds, and *may*
only have an issue with a single port that had the option to build with PIE
(which I don't see the point of, and I don't think I was using -mdynamic-no-pic
at the time). I know that option is incompatible with +universal builds unless
you add a linker option I never managed to remember.
I don't even know if -mdynamic-no-pic still gives the same almost free
performance gain as it did on PPC (15% or so, I learned about it long ago
thanks to the still-lamented Shark tool). But my Mac is now old enough that it
makes sense to maximise compilation optimisation. I only don't use LTO
systematically because I never managed to get a usable backtrace from slightly
more complicated applications with it.
BTW, I do have a LTO PortGroup that adds a +LTO variant and works some magic
which seems to work pretty reliably and which I could contribute if there's an
interest for it.
R.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On 2019-9-17 07:26 , Ryan Schmidt wrote: >> I cannot agree completely with that. What if you want to build all your >> ports with maximum optimisation (including -march=native and >> -mdynamic-no-pic)? > > There is no feature in MacPorts for that. Individual ports can offer a > variant to do that if that would be of particular benefit to that port. For > example the gdal port offers a +perf variant. We should standardize on a name > for such a variant. I don't know if we've already done so. We've agreed on +native to use all the CPU features of the build machine. But -mdynamic-no-pic is incompatible with PIE, which is the default since 10.7. - Josh
Re: scope of configure.optflags, configure.compiler etc. command line variables
I don't want to discuss this at great length, but I'll answer these questions: On Sep 16, 2019, at 02:22, René J.V. Bertin wrote: > On Sunday September 15 2019 20:25:59 Ryan Schmidt wrote: > >> Setting such options on the command line is intended as a debugging >> technique only. > > I cannot agree completely with that. What if you want to build all your ports > with maximum optimisation (including -march=native and -mdynamic-no-pic)? There is no feature in MacPorts for that. Individual ports can offer a variant to do that if that would be of particular benefit to that port. For example the gdal port offers a +perf variant. We should standardize on a name for such a variant. I don't know if we've already done so. > What about `configureccache`, that cannot possible be defended as a debugging > technique (and by definition ccache won't help with code that has the be > recompiled each time because you keep changing it). There should be no need for you to specify anything ccache-related on the command line. You can enable ccache globally in macports.conf. Individual ports can disable it if they are not compatible with it. > Comparable systems on Linux all have a way to define a way to define the > default compiler options (and compiler choice), just like MacPorts has a way > to define default variants. Evidently MacPorts has made different design choices. Just because a person building software could supply an option to the build that would affect it in some way does not mean that MacPorts will provide a way for the user to access that option. It is up to the port maintainer to make good choices about what build options are exposed to users via variants and which ones will be hidden in the portfile. > Where is the code anyway that handles the reset when processing of a new port > starts? Maybe the reset itself could be made dependent on a switch, such that > it'd be on by default but can be toggled to "sticky" behaviour instead? I don't know.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Sunday September 15 2019 20:25:59 Ryan Schmidt wrote: >Setting such options on the command line is intended as a debugging technique >only. I cannot agree completely with that. What if you want to build all your ports with maximum optimisation (including -march=native and -mdynamic-no-pic)? What about `configureccache`, that cannot possible be defended as a debugging technique (and by definition ccache won't help with code that has the be recompiled each time because you keep changing it). Comparable systems on Linux all have a way to define a way to define the default compiler options (and compiler choice), just like MacPorts has a way to define default variants. Where is the code anyway that handles the reset when processing of a new port starts? Maybe the reset itself could be made dependent on a switch, such that it'd be on by default but can be toggled to "sticky" behaviour instead? R.
Re: scope of configure.optflags, configure.compiler etc. command line variables
On Sep 12, 2019, at 03:46, René J.V. Bertin wrote: > I hope I didn't ask this before: > > Is there a way to let command line options like the ones in the subject apply > to all ports on the command line, instead of only to the one given just > before of the option in question? I would guess the answer is no. Setting such options on the command line is intended as a debugging technique only. It's expected that you're only debugging one port at a time, so this limitation shouldn't be a problem.
