Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Sat, 29 Sep 2018 07:59:07 -0400 Josh Fisher wrote: > Bacula uses autoconf, so to cross compile, start > with the --build and --host flags for the configure > script. See > https://www.gnu.org/software/automake/manual/html_node/Cross_002dCompilation.html. Thanks. -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On 9/29/2018 5:52 AM, George Anchev via Bacula-users wrote: Can you please advise about this: On Mon, 24 Sep 2018 12:12:42 +0300 George Anchev wrote: Also: I would like to use a faster machine to create builds for other slower machines (one of them even 32-bit). Do I need anything more than just finding the proper "-march" setting for the target CPU? Bacula uses autoconf, so to cross compile, start with the --build and --host flags for the configure script. See https://www.gnu.org/software/automake/manual/html_node/Cross_002dCompilation.html. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Sat, 29 Sep 2018 19:30:09 +1000 Gary R. Schmidt wrote: > Well, take some advice from someone who has been > writing it for more than thirty-five years, and > still does - just leave it alone. Thanks. Can you please advise about this: On Mon, 24 Sep 2018 12:12:42 +0300 George Anchev wrote: > Also: I would like to use a faster machine to create > builds for other slower machines (one of them even > 32-bit). Do I need anything more than just finding > the proper "-march" setting for the target CPU? -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On 29/09/2018 18:57, George Anchev via Bacula-users wrote: On Sat, 29 Sep 2018 12:09:24 +1000 Gary R. Schmidt wrote: You appear to be suffering from what one of my lecturers back at $GOSHWHATTAUNIVERSITY called "Premature Optimisation." (That was back in the 1980s, and he had worked on CSIRAC :-) ) Perhaps because I haven't written any C code for 20 years :) Well, take some advice from someone who has been writing it for more than thirty-five years, and still does - just leave it alone. Cheers, GaryB-) ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Sat, 29 Sep 2018 12:09:24 +1000 Gary R. Schmidt wrote: > You appear to be suffering from what one of my > lecturers back at $GOSHWHATTAUNIVERSITY called > "Premature Optimisation." (That was back in the > 1980s, and he had worked on CSIRAC :-) ) Perhaps because I haven't written any C code for 20 years :) -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Sat, 29 Sep 2018 01:36:46 +0200 Tilman Schmidt wrote: > No. As Martin already wrote, you can remove it from > the built executables by using the "strip" command, > see > > man strip Great. Thanks. -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On 9/28/18 10:09 PM, Gary R. Schmidt wrote: > Most of the time the various Bacula programs are waiting for I/O > operations to complete, the only time the CPU gets heavily involved is > if you are running compression. > > Just use the defaults. You won't save enough CPU, memory, or time to > justify your efforts, and if you optimise at a level that triggers some > pathological behaviour in Bacula you may not find out that critical > parts of your back-ups are garbled until it is too late. I agree 100%. Bacula is in general I/O bound except when performing a large database operation, and *then*, most of the time, the database operation it is waiting on is I/O bound. If you can get 10% faster storage you'll probably see ten times the real-world performance gain you'd see from 20% Bacula run-time optimization. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On 29/09/2018 04:00, George Anchev via Bacula-users wrote:
On Fri, 28 Sep 2018 14:47:46 +0100 Martin Simmons
wrote:
This is not Bacula-specific: producing debugging
information is useful, just in case you need it.
And an end user generally does not need it. Also AFAIK
adding debugging info to binaries makes them slower.
>
This has not been true for decades.
The debugging information is useful in Bacula because when a Bacula
binary crashes it tries to run the btraceback.{dbx,gdb} script to help
find out what has gone wrong.
You appear to be suffering from what one of my lecturers back at
$GOSHWHATTAUNIVERSITY called "Premature Optimisation." (That was back
in the 1980s, and he had worked on CSIRAC :-) )
If you think about what the three major components of Bacula do, very
little is dependent on the CPU.
- Reads data from disk.
- Transmit data across the network.
- Write data to disk (if you are spooling).
- Write data to tape/disk.
- Send data to a database server.
Most of the time the various Bacula programs are waiting for I/O
operations to complete, the only time the CPU gets heavily involved is
if you are running compression.
Just use the defaults. You won't save enough CPU, memory, or time to
justify your efforts, and if you optimise at a level that triggers some
pathological behaviour in Bacula you may not find out that critical
parts of your back-ups are garbled until it is too late.
Cheers,
GaryB-)
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
Am 28.09.2018 um 22:05 schrieb George Anchev via Bacula-users: > On Fri, 28 Sep 2018 21:22:31 +0200 Tilman Schmidt wrote: > [debuging information] >> What's more, it's easy to remove should it bother >> you, but imposssible to add later if you didn't >> create it while building the executable. > > Thanks for explaining. How do you remove it? Doesn't that need rebuilding? No. As Martin already wrote, you can remove it from the built executables by using the "strip" command, see man strip HTH Tilman ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Fri, 28 Sep 2018 21:22:31 +0200 Tilman Schmidt wrote: > As long as everything runs smoothly you don't. > As soon as you want to create a crash report you do. > And it costs next to nothing so it is a good thing > to have in place should a crash occur. > What's more, it's easy to remove should it bother > you, but imposssible to add later if you didn't > create it while building the executable. Thanks for explaining. How do you remove it? Doesn't that need rebuilding? -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
Am 28.09.2018 um 20:00 schrieb George Anchev via Bacula-users: > On Fri, 28 Sep 2018 14:47:46 +0100 Martin Simmons wrote: > >> This is not Bacula-specific: producing debugging >> information is useful, just in case you need it. > > And an end user generally does not need it. As long as everything runs smoothly you don't. As soon as you want to create a crash report you do. And it costs next to nothing so it is a good thing to have in place should a crash occur. What's more, it's easy to remove should it bother you, but imposssible to add later if you didn't create it while building the executable. > Also AFAIK adding debugging info to binaries makes them slower. No, it doesn't. It just consumes a bit of disk space. HTH Tilman ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Fri, 28 Sep 2018 14:47:46 +0100 Martin Simmons
wrote:
> This is not Bacula-specific: producing debugging
> information is useful, just in case you need it.
And an end user generally does not need it. Also AFAIK
adding debugging info to binaries makes them slower.
> You can remove it using strip during deployment if
> you don't want it.
In my build script I use
CFLAGS='-O2 -pipe -march=native' \
CXXFLAGS="${CFLAGS}" \
./configure \
...
I suppose this doesn't include debug info (please
correct me if I am wrong).
--
George
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
> On Thu, 27 Sep 2018 19:26:40 +0300, George Anchev via Bacula-users said:
>
> As for standard options - according to the manual:
>
> 'We recommend [...] CFLAGS="-g -O2 -Wall" [...]'
>
> but it is not clear why e.g. "-g" is needed ("Produce
> debugging information" (ref. `man gcc`)) if one is
> building a release version. So perhaps adding a little
> extra explanation about the reasons for the
> recommendation may be useful. I don't find anything
> about CFLAGS in the developer's guide either.
This is not Bacula-specific: producing debugging information is useful, just
in case you need it. You can remove it using strip during deployment if you
don't want it.
__Martin
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On Thu, 27 Sep 2018 15:55:38 +0200 Kern Sibbald wrote:
> You are free to try anything, but it is to your own
> peril, and we cannot support anything but standard
> Bacula compile options.
Of course I can play with things but the question is
would it make sense to use -O3 instead of -O2 which it
seems to depend directly on how the software is
written.
As for standard options - according to the manual:
'We recommend [...] CFLAGS="-g -O2 -Wall" [...]'
but it is not clear why e.g. "-g" is needed ("Produce
debugging information" (ref. `man gcc`)) if one is
building a release version. So perhaps adding a little
extra explanation about the reasons for the
recommendation may be useful. I don't find anything
about CFLAGS in the developer's guide either.
--
George
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
Hello, You are free to try anything, but it is to your own peril, and we cannot support anything but standard Bacula compile options. Best regards, Kern On 09/24/2018 11:12 AM, George Anchev via Bacula-users wrote: Hi, https://wiki.gentoo.org/wiki/GCC_optimization explains that: -ftree-vectorize is an optimization option (default at -O3 and -Ofast), which attempts to vectorize loops using the selected ISA if possible. The reason it isn't enabled at -O2 is that it doesn't always improve code, it can make code slower as well, and usually makes the code larger; it really depends on the loop etc. What is the case with Bacula? Would the build benefit or suffer from using "-O3" instead of "-O2"? Also: I would like to use a faster machine to create builds for other slower machines (one of them even 32-bit). Do I need anything more than just finding the proper "-march" setting for the target CPU? -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
Hi Josh, You seem to be replying to an earlier message (about CFLAGS optimizations) quoting the latest one (which is about building for another architecture), so at first that got me confused. FWIW: I don't use software compression (the storage has hardware compression). -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
On 9/25/2018 5:18 AM, George Anchev via Bacula-users wrote: ... Also: I would like to use a faster machine to create builds for other slower machines (one of them even 32-bit). Do I need anything more than just finding the proper "-march" setting for the target CPU? I'm not sure you need to do even that. The default should be to produce code that will run on all supported CPUs (of the same instruction set architecture). Unless you need to cross-compile, or to target a CPU so old that it is no longer supported by default (e.g., a 80x86 with x<6), you probably don't need to do anything special. Creating a 32-bit build on a 64-bit system may require you to use -m32 (or whatever the exact syntax is) or (my preferred approach) to build in a 32-bit environment (chroot, container or virtual machine). I am particularly interested in building FDs for two types of CPUs. I am attaching the /proc/cpuinfo for both. Could you advise on specifics? I'm not sure whether compiler optimizations will make a huge difference. The FD is quite i/o bound, in general. However, there are exceptions. On clients with weak CPUs, enabling compression and/or encryption will very much affect performance. If software compression is required, then try to use LZO. Weak clients will usually perform better with compression disabled, in spite of the increased network traffic. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] CFLAGS optimizations and building for another architecture
Hi Sergio, I suppose you replied personally to me instead of to the list by mistake. I will provide my answer here, quoting yours: On Tue, 25 Sep 2018 08:56:30 +0200 Sergio Gelato wrote: > I think you're asking the wrong question. The > question I would ask is: is the performance of > Bacula limited by the CPU or by other factors > (especially I/O and system calls)? If Bacula isn't > CPU-bound, the difference between -O2 and -O3 isn't > going to be worth worrying about. I understand what you are saying but still I was interested to know. > I haven't looked at the Bacula code base in enough > detail to be sure, but I'd expect its loops to be > linked-list traversals rather than iterations over > large arrays. Vectorization isn't going to help much > in that case. Hopefully someone who has more in-depth look at the code could provide more clarity. > Ultimately, the best way to answer such questions is > by direct benchmarking: build it both ways, and see > if you can measure a difference. I don't know how to create a benchmark comparison scenario, otherwise I wouldn't mind doing it. > I just wouldn't bother to do it with Bacula since > its main job is I/O, not computation. Aren't DB operations, sorting, (maybe something else) affected by certain optimizations? > (Some things in Bacula are memory-bound. For > example, doing an accurate backup of a filesystem > with lots of inodes is going to require lots of RAM; > if you're short of memory the page swapping activity > will kill performance. None of this is going to be > affected by -O2 vs. -O3, and I don't think even -Os > is going to help much; what's needed is an algorithm > that makes smarter use of the available memory. Most > people just buy more RAM instead.) Algorithm efficiency is surely essential, I am not even questioning that. But since I am unaware of what algorithms are used and which of them may benefit or suffer from -O3, I decided to ask. > > Also: I would like to use a faster machine to > > create builds for other slower machines (one of > > them even 32-bit). Do I need anything more than > > just finding the proper "-march" setting for the > > target CPU? > > I'm not sure you need to do even that. The default > should be to produce code that will run on all > supported CPUs (of the same instruction set > architecture). Unless you need to cross-compile, or > to target a CPU so old that it is no longer > supported by default (e.g., a 80x86 with x<6), you > probably don't need to do anything special. Creating > a 32-bit build on a 64-bit system may require you to > use -m32 (or whatever the exact syntax is) or (my > preferred approach) to build in a 32-bit environment > (chroot, container or virtual machine). I am particularly interested in building FDs for two types of CPUs. I am attaching the /proc/cpuinfo for both. Could you advise on specifics? -- George # cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 13 model name : Intel(R) Celeron(R) M processor 1.50GHz stepping: 8 microcode : 0x20 cpu MHz : 1496.392 cache size : 1024 KB physical id : 0 siblings: 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fdiv_bug: no f00f_bug: no coma_bug: no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss tm pbe bts cpuid bugs: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf bogomips: 2992.78 clflush size: 64 cache_alignment : 64 address sizes : 32 bits physical, 32 bits virtual power management:# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 3 model name : Intel(R) Xeon(TM) CPU 2.80GHz stepping: 4 microcode : 0x17 cpu MHz : 2799.886 cache size : 1024 KB physical id : 0 siblings: 2 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall lm constant_tsc pebs bts nopl cpuid pni dtes64 monitor ds_cpl cid xtpr pti bugs: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf bogomips: 5599.77 clflush size: 64 cache_alignment : 128 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 15 model : 3 model name : Intel(R) Xeon(TM) CPU 2.80GHz stepping: 4 microcode : 0x17 cpu MHz : 2799.886 cache size : 1024 KB physical id : 3 siblings: 2 core id : 0 cpu cores : 1
