Hi Shawn,
On Fri, Jun 14, 2019 at 09:03:33PM -0600, Shawn Heisey wrote:
> What I've noticed is that for the most part, you can classify source code
> downloaders in one of two camps: Either the complete novice, or the
> experienced user. The complete novice wants steps that are very simple, and
> the experienced user wants the build to be customizable with myriad knobs
> and switches, and wants them all documented. I find that membership in an
> "intermediate" camp is very small ... people tend to either remain novices
> or gravitate towards expert.
I totally agree.
> Dividing the README into two parts, one for the novice and one for the
> expert, would be reasonable to me.
This is a good proposal. I think it's what I tried to do recently but
maybe it's not enough. It starts with a "quick build & install" section
proposing a command to type on most common platforms.
> The haproxy build is one of the more complex that I've encountered.
This is interesting because it's not the feedback I got nor my experience
either. I got many times some in person greetings like "thank you for not
pissing us off with autoconf", which I personally totally share. But I
agree that the doc on the build process is probably lacking a lot. And
what is not easy is to add extra defines, CFLAGS or LDFLAGS because we
tried to remain mostly compatible with command lines people have been
using for a while, resulting in multiple ways to do this.
> I've
> written a little shell script to handle building and installing it, because
> I'll never remember all the make options I need without checking the docs:
>
> ---------------------
> root@smeagol:/usr/local/src# cat new-haproxy
> #!/bin/sh
> if [ -e "$1" ]; then
> if [ -d "$1" ]; then
> cd $1
> make clean &&
> make TARGET=linux2628 USE_PCRE_JIT=1 USE_OPENSSL=1 USE_ZLIB=1
> USE_SYSTEMD=1 CPU=native
> RETVAL="$?"
> if [ "$RETVAL" = 0 ]; then
> make install
> else
> echo Build failed\! skipping install\!
> fi
> else
> echo location \($1\) is not a directory.
> fi
> else
> echo location \($1\) does not exist.
> fi
> ---------------------
So the really useful part of it is :
make clean &&
make TARGET=linux2628 USE_PCRE_JIT=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1
CPU=native &&
make install
Right ? Thus I guess the complex aspect you're seeing here is to remember the
names of the USE_* options and the target.
In a patchset I'm going to merge now, I've improved this a bit so that
the "help" target lists these options and the ones enabled or disabled
on your current target.
> It would be really nice if a novice could simply type "make install" and
> have it build with "sane" options for whatever OS they're on.
That's about what William proposed with the "recommended" options. But
if we do so we need to do it cross-systems, as Linux clearly is the most
used one but not the only one.
> Maybe having
> one static command (or set of commands) for almost every user isn't
> achievable without additional software that you don't want to use, such as
> autoconf.
The problem with autoconf precisely is that it does *not* set the options
the user *wants* but instead will prevent the user from having more options
than what autoconf *believes* his *current* system supports. That makes a
lot of wrongs resulting in every user having to spend countless hours
reading horrible configure scripts to figure the secret "ac_cv_*" variables
that have to be forced by hand to make the script believe it previously
checked and found a usable result, or patch the script. And while this
could be acceptable for a lot of software that you're going to run locally,
it makes no sense for something like haproxy that you are rarely going to
build from sources to use on your desktop machine or laptop : if you're
just a novice user or a web application developer, and have an instance
on your machine to test your application, you just want to pick the last
version shipped with your distro. Tricking autoconf scripts to force them
to build for your servers' options from your desktop is another level of
difficulty that's a real pain even for advanced users.
> That said, some simplification would be welcome. Aliases so that
> "TARGET=linux" produces a generally useful build on most current Linux
> distros would be AWESOME.
That was William's proposal initially and I'd tend to agree on this. I'd
like to see openssl enable by default on most platforms for example.
> The build should, as much as possible, check that
> all options requested are possible, and fail fast if they're not, with
> useful error messages.
We could possibly create a "check" target in the makefile to validate
if it thinks it will be able to build, and possibly to suggest *some*
package names depending on the system.
> General statement, not directed specifically at haproxy: I find it
> frustrating to have a build get started, run for several minutes,
Often if it takes several minutes it's precisely because you have autoconf :-)
(then it can be several hours to work around it when it decides it doesn't
like you).
> and then
> suddenly fail with a compiler message that may or may not be easily
> translatable into "you don't have the XXX development library installed." I
> can usually figure out what those errors mean, but the novice user probably
> won't have any clue.
I agree. Similarly when you build some software that automatically downloads
some sources, and find a build error due to a 404 after several minutes, or
discover that it spends lots of time downloading while you could have done
it way faster at work, it is quite annoying.
> A new "linux" target should probably internally choose the most common
> numbered variant. At the moment, that's the latest (2628), but at some
> point in the future it might not be the latest. It does seem reasonable to
> have the "linux" alias *not* enable any other options, and have other
> aliases that enable things like ZLIB, SSL, PCRE, etc.
That's the current goal, to have it set only what is provided by the libc.
> > > I'd like very much to get rid of this laughing "2.6.28" now but if we
> > > only use "linux" nobody will update it and we'll be back to the same
> > > situation in one or two versions. With the split above we can have
> > > fast moving targets ("complete", updated during dev; "recent", updated
> > > with new distro announces) and slow moving ones ("ancient", "minimal")
> > > and that might be a sweet spot.
>
> I've wondered about that, actually. Occasionally I wonder whether 3.x or
> 4.x kernels (and now 5.x) might offer something new that could improve
> haproxy.
Yep. But surprisingly the kernel is the easiest thing to detect support
for and most of it is done at run time. There is a little bit of build
time selection (e.g. disable features you're not interested in for code
size reasons) and build time detection (add some missing socket options
mostly), but it's mostly painless thanks to the excellent API and ABI
compatibility the kernel provides (and the libc as well for that matter).
> Because the target name hasn't changed in such a long time, I have
> to assume that there hasn't been anything earth-shattering, and that the
> options for 2.6.28 really do offer the best possible performance on recent
> kernels.
In fact there were lots of new stuff added till 2.6.28 (when splice really
started to work flawlessly) and from there the rest is dynamically detected
so we ended up not changing this option, which is very confusing for new
users, reason why I wanted to get rid of it.
> I personally have no use for older Linux targets, but I feel pretty sure
> that there are still plenty of systems running on ancient kernels, with
> admins that want to build haproxy.
I've reviewed the options and ended up adding these ones in the "linux-glibc"
platform :
- getaddrinfo (was still not enabled due to being broken on some
alternative libcs)
- TCP Fast Open (automatically detected, present since 3.7 or so, no
impact if absent)
- namespaces (present for a very long time, autodetected at run time,
may fail at build time with very old libcs but trivial to disable).
That results in making linux at the same level as other platforms, with
everything supported by the system enabled by default. On top of this
we can add the optional stuff (ssl, pcre, zlib, lua, ...) like for any
other system.
Actually maybe we should have some super-options separate from the target
to decide what feature set to enable. Instead of having just TARGET being
mandatory, we could have both TARGET and OPTIONS for example. Then one
could just build like this :
make TARGET=linux-glibc OPTIONS=all
or :
make TARGET=freebsd OPTIONS=ssl,zlib,pcre
or :
make TARGET=cygwin OPTIONS=ssl
At this point we'll need to pursue this discussion for 2.1 I guess, and
this will not prevent us from backporting some improvements to help users
of 2.0. But let's not forget that novices should definitely use their
distro's packages first.
Thanks!
Willy
