Re: How to compile with packaged openssl when custom openssl installed?

[Илья Шипицин]

чт, 4 нояб. 2021 г. в 22:31, Shawn Heisey :

> On 11/4/21 7:55 AM, Willy Tarreau wrote:
> > Normally you just have to specify SSL_INC and SSL_LIB at build time to
> > specify the one you want to build with. I'm doing exactly this when I
> > want to build with older versions:
>
>
> I tried this.  My make command (building 2.4.8) had these env additions:
>
>SSL_INC=/usr/include/openssl \
>SSL_LIB=/usr/lib/x86_64-linux-gnu \
>
> Which should have told it to use the openssl provided by Ubuntu
> packages.  But that didn't work, it still found the 3.x version in
> /usr/local (installed with openssl default locations for ./Configure),
> and failed to compile.
>
> I thought I found an error in the Makefile where setting USE_OPENSSL
> clears SSL_INC and SSL_LIB, but even with that problem handled (I think
> ... my Makefile experience is slim), it STILL finds the 3.x version and
> tries to use it.
>

it does not clear.

at least gnu make and bsd make.
SSL_INC=
means "if SSL_INC is defined as environment variable, clear it", but if you
specify SSL_INC from make command line, it has priority over Makefile


>
> Changing the prefix on the openssl compile to something nonstandard
> (/usr/local/ssl3 in my case) is the only way I have found to keep the
> haproxy build from finding it.  This is less than ideal, but sufficient
> for my needs.
>
> Thanks,
> Shawn
>
>
>
>

Re: How to compile with packaged openssl when custom openssl installed?

[Shawn Heisey]

On 11/6/2021 3:22 AM, Erwan Le Pape wrote:
Try building with SSL_INC=/usr/include and you should be good to build. 
When running HAProxy you'll have to make sure that the library 
resolution order is consistent with the library you linked against (ie. 
/usr/local/lib is searched after /usr/lib) or you'll face some weird 
issues and crashes.


I thought you might be on to something here.   But it didn't help.  Even 
with SSL_INC=/usr/include it still failed just as before, trying to 
compile against openssl 3.x.


I guess I'll just have to use the nonstandard prefix when building 
openssl 3.x.  That is an acceptable workaround for me.  I had hoped I 
could install it in the standard location, but I don't actually NEED it 
to be in that location.


Thanks,
Shawn

Re: How to compile with packaged openssl when custom openssl installed?

[Erwan Le Pape]

You've specified the wrong SSL_INC folder, which led to the build system 
failing to find it in the provided location and searching for it itself.


OpenSSL headers live at ${includedir}/openssl by default. This means 
specifying SSL_INC as /usr/include/openssl leads to the build system 
looking for OpenSSL headers in /usr/include/openssl/openssl which it 
cannot find. It moves on to looking for OpenSSL by itself which leads to 
the same result as not having provided SSL_INC.


Try building with SSL_INC=/usr/include and you should be good to build. 
When running HAProxy you'll have to make sure that the library 
resolution order is consistent with the library you linked against (ie. 
/usr/local/lib is searched after /usr/lib) or you'll face some weird 
issues and crashes.


Erwan

On 04/11/2021 18:28, Shawn Heisey wrote:

On 11/4/21 7:55 AM, Willy Tarreau wrote:

Normally you just have to specify SSL_INC and SSL_LIB at build time to
specify the one you want to build with. I'm doing exactly this when I
want to build with older versions:



I tried this.  My make command (building 2.4.8) had these env additions:

  SSL_INC=/usr/include/openssl \
  SSL_LIB=/usr/lib/x86_64-linux-gnu \

Which should have told it to use the openssl provided by Ubuntu 
packages.  But that didn't work, it still found the 3.x version in 
/usr/local (installed with openssl default locations for ./Configure), 
and failed to compile.


I thought I found an error in the Makefile where setting USE_OPENSSL 
clears SSL_INC and SSL_LIB, but even with that problem handled (I 
think ... my Makefile experience is slim), it STILL finds the 3.x 
version and tries to use it.


Changing the prefix on the openssl compile to something nonstandard 
(/usr/local/ssl3 in my case) is the only way I have found to keep the 
haproxy build from finding it.  This is less than ideal, but 
sufficient for my needs.


Thanks,
Shawn

Re: How to compile with packaged openssl when custom openssl installed?

[Shawn Heisey]

On 11/4/21 7:55 AM, Willy Tarreau wrote:

Normally you just have to specify SSL_INC and SSL_LIB at build time to
specify the one you want to build with. I'm doing exactly this when I
want to build with older versions:



I tried this.  My make command (building 2.4.8) had these env additions:

  SSL_INC=/usr/include/openssl \
  SSL_LIB=/usr/lib/x86_64-linux-gnu \

Which should have told it to use the openssl provided by Ubuntu 
packages.  But that didn't work, it still found the 3.x version in 
/usr/local (installed with openssl default locations for ./Configure), 
and failed to compile.


I thought I found an error in the Makefile where setting USE_OPENSSL 
clears SSL_INC and SSL_LIB, but even with that problem handled (I think 
... my Makefile experience is slim), it STILL finds the 3.x version and 
tries to use it.


Changing the prefix on the openssl compile to something nonstandard 
(/usr/local/ssl3 in my case) is the only way I have found to keep the 
haproxy build from finding it.  This is less than ideal, but sufficient 
for my needs.


Thanks,
Shawn

Re: How to compile with packaged openssl when custom openssl installed?

[Willy Tarreau]

On Thu, Nov 04, 2021 at 09:53:59PM +0500,  ??? wrote:
> we do not fail build if SSL_LIB points to wrong folder ?

For sure we do, since libs will be missing, and the linking will
fail!

Willy

Re: How to compile with packaged openssl when custom openssl installed?

[Илья Шипицин]

чт, 4 нояб. 2021 г. в 18:58, Willy Tarreau :

> Hi Shawn,
>
> On Wed, Nov 03, 2021 at 10:56:02AM -0600, Shawn Heisey wrote:
> > On 11/3/21 9:25 AM,  ??? wrote:
> > > you either need to specify LD_LIBRARY_PATH or add rpath during link,
> > > here's example how to use rpath via ADDLIB haproxy/.travis.yml at
> > > 57610c694e56a6b0d55bf42f1170bad93b7b3297 · haproxy/haproxy (github.com)
> <
> https://github.com/haproxy/haproxy/blob/57610c694e56a6b0d55bf42f1170bad93b7b3297/.travis.yml#L68-L85
> >
> >
> >
> > I can't tell how to actually use that for my setup from the highlighted
> > lines in that github page.
> >
> > Everything I have seen says that haproxy's build system is ignoring the
> > SSL_INC and SSL_LIB settings I told it to use, and autodetecting the
> openssl
> > in /usr/local.
> >
> > But even if I am wrong about that, I did work out how to achieve my
> goals.
> > I built openssl with --prefix=/usr/local/ssl3 and made a symlink for its
> > "openssl" binary to /usr/local/bin/ossl. I get to have the custom openssl
> > installed and available with an altered command, but now haproxy's build
> > system won't find it.
>
> Normally you just have to specify SSL_INC and SSL_LIB at build time to
> specify the one you want to build with. I'm doing exactly this when I
> want to build with older versions:
>
>   $ ls -1d /opt/openssl-*
>   /opt/openssl-0.9.8/
>   /opt/openssl-1.0.0/
>   /opt/openssl-1.0.2/
>   /opt/openssl-1.1.0/
>
>   $ make -j$(nproc) TARGET=linux-glibc USE_OPENSSL=1 \
> SSL_INC=/opt/openssl-1.0.2/include SSL_LIB=/opt/openssl-1.0.2/lib
>   ...
>   LD  haproxy
>   $ ./haproxy -v
>   HAProxy version 2.5-dev12-726635-14 2021/11/03 - https://haproxy.org/
>
> When used on the same machine you used to build, it's also possible to
> use -rpath to store the lib's path into the executable:
>
>   $ make -j$(nproc) TARGET=linux-glibc USE_OPENSSL=1 \
> SSL_INC=/opt/openssl-1.0.2/include \
> SSL_LIB="/opt/openssl-1.0.2/lib -Wl,-rpath=/opt/openssl-1.0.2/lib"
>
>   $ ldd ./haproxy
> linux-vdso.so.1 (0x7ffce1ff9000)
> libcrypt.so.1 => /lib64/libcrypt.so.1 (0x7f5fa6f83000)
> libdl.so.2 => /lib64/libdl.so.2 (0x7f5fa6f7e000)
> librt.so.1 => /lib64/librt.so.1 (0x7f5fa6f74000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x7f5fa6f52000)
> libssl.so.1.0.0 => /opt/openssl-1.0.2/lib/libssl.so.1.0.0
> (0x7f5fa6ce2000)
> libcrypto.so.1.0.0 => /opt/openssl-1.0.2/lib/libcrypto.so.1.0.0
> (0x7f5fa689d000)
> libc.so.6 => /lib64/libc.so.6 (0x7f5fa66b6000)
> /lib64/ld-linux-x86-64.so.2 (0x7f5fa700e000)
>
>   $ ./haproxy -vv | grep -i ssl
>   OPTIONS = USE_OPENSSL=1
>   Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT -PCRE2
> -PCRE2_JIT +POLL +THREAD +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY
> +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL -LUA
> +ACCEPT4 -CLOSEFROM -ZLIB +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS
> -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL -PROCCTL +THREAD_DUMP
> -EVPORTS -OT -QUIC -PROMEX -MEMORY_PROFILING
>   Built with OpenSSL version : OpenSSL 1.0.2j  26 Sep 2016
>   Running on OpenSSL version : OpenSSL 1.0.2j  26 Sep 2016
>   OpenSSL library supports TLS extensions : yes
>   OpenSSL library supports SNI : yes
>   OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
>
> There's no reason that wouldn't work for you, as it's commonly used.
> I suspect you just have one option wrong (possibly missing /lib at
> the end of the SSL_LIB for example).
>

wow.
we do not fail build if SSL_LIB points to wrong folder ?


>
> Willy
>
>

Re: How to compile with packaged openssl when custom openssl installed?

[Willy Tarreau]

Hi Shawn,

On Wed, Nov 03, 2021 at 10:56:02AM -0600, Shawn Heisey wrote:
> On 11/3/21 9:25 AM,  ??? wrote:
> > you either need to specify LD_LIBRARY_PATH or add rpath during link,
> > here's example how to use rpath via ADDLIB haproxy/.travis.yml at
> > 57610c694e56a6b0d55bf42f1170bad93b7b3297 · haproxy/haproxy (github.com) 
> > 
> 
> 
> I can't tell how to actually use that for my setup from the highlighted
> lines in that github page.
> 
> Everything I have seen says that haproxy's build system is ignoring the
> SSL_INC and SSL_LIB settings I told it to use, and autodetecting the openssl
> in /usr/local.
> 
> But even if I am wrong about that, I did work out how to achieve my goals. 
> I built openssl with --prefix=/usr/local/ssl3 and made a symlink for its
> "openssl" binary to /usr/local/bin/ossl. I get to have the custom openssl
> installed and available with an altered command, but now haproxy's build
> system won't find it.

Normally you just have to specify SSL_INC and SSL_LIB at build time to
specify the one you want to build with. I'm doing exactly this when I
want to build with older versions:

  $ ls -1d /opt/openssl-* 
  /opt/openssl-0.9.8/
  /opt/openssl-1.0.0/
  /opt/openssl-1.0.2/
  /opt/openssl-1.1.0/

  $ make -j$(nproc) TARGET=linux-glibc USE_OPENSSL=1 \
SSL_INC=/opt/openssl-1.0.2/include SSL_LIB=/opt/openssl-1.0.2/lib
  ...
  LD  haproxy
  $ ./haproxy -v
  HAProxy version 2.5-dev12-726635-14 2021/11/03 - https://haproxy.org/

When used on the same machine you used to build, it's also possible to
use -rpath to store the lib's path into the executable:

  $ make -j$(nproc) TARGET=linux-glibc USE_OPENSSL=1 \
SSL_INC=/opt/openssl-1.0.2/include \
SSL_LIB="/opt/openssl-1.0.2/lib -Wl,-rpath=/opt/openssl-1.0.2/lib"

  $ ldd ./haproxy
linux-vdso.so.1 (0x7ffce1ff9000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x7f5fa6f83000)
libdl.so.2 => /lib64/libdl.so.2 (0x7f5fa6f7e000)
librt.so.1 => /lib64/librt.so.1 (0x7f5fa6f74000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x7f5fa6f52000)
libssl.so.1.0.0 => /opt/openssl-1.0.2/lib/libssl.so.1.0.0 
(0x7f5fa6ce2000)
libcrypto.so.1.0.0 => /opt/openssl-1.0.2/lib/libcrypto.so.1.0.0 
(0x7f5fa689d000)
libc.so.6 => /lib64/libc.so.6 (0x7f5fa66b6000)
/lib64/ld-linux-x86-64.so.2 (0x7f5fa700e000)

  $ ./haproxy -vv | grep -i ssl
  OPTIONS = USE_OPENSSL=1
  Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT 
+POLL +THREAD +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY 
+LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL -LUA +ACCEPT4 -CLOSEFROM 
-ZLIB +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL 
-SYSTEMD -OBSOLETE_LINKER +PRCTL -PROCCTL +THREAD_DUMP -EVPORTS -OT -QUIC 
-PROMEX -MEMORY_PROFILING
  Built with OpenSSL version : OpenSSL 1.0.2j  26 Sep 2016
  Running on OpenSSL version : OpenSSL 1.0.2j  26 Sep 2016
  OpenSSL library supports TLS extensions : yes
  OpenSSL library supports SNI : yes
  OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2

There's no reason that wouldn't work for you, as it's commonly used.
I suspect you just have one option wrong (possibly missing /lib at
the end of the SSL_LIB for example).

Willy

Re: How to compile with packaged openssl when custom openssl installed?

[Shawn Heisey]

On 11/3/21 10:56 AM, Shawn Heisey wrote:
Everything I have seen says that haproxy's build system is ignoring 
the SSL_INC and SSL_LIB settings I told it to use, and autodetecting 
the openssl in /usr/local.



I thought the following patch would take care of it, but it did not 
work.  So I will go back to installing openssl v3 to a non-standard 
prefix, which does work.



--- Makefile.orig    2021-11-03 11:30:26.043154770 -0600
+++ Makefile    2021-11-03 11:30:48.876099955 -0600
@@ -574,9 +574,10 @@
 OPTIONS_OBJS   += src/cpuset.o
 endif

-ifneq ($(USE_OPENSSL),)
+ifeq ($(USE_OPENSSL),)
 SSL_INC =
 SSL_LIB =
+else
 # OpenSSL is packaged in various forms and with various dependencies.
 # In general -lssl is enough, but on some platforms, -lcrypto may be 
needed,
 # reason why it's added by default. Some even need -lz, then you'll 
need to



There is obviously more to the build system than I have the knowledge to 
grok.


Thanks,
Shawn

Re: How to compile with packaged openssl when custom openssl installed?

[Shawn Heisey]

On 11/3/21 9:25 AM, Илья Шипицин wrote:

you either need to specify LD_LIBRARY_PATH or add rpath during link,
here's example how to use rpath via ADDLIB haproxy/.travis.yml at 
57610c694e56a6b0d55bf42f1170bad93b7b3297 · haproxy/haproxy 
(github.com) 




I can't tell how to actually use that for my setup from the highlighted 
lines in that github page.


Everything I have seen says that haproxy's build system is ignoring the 
SSL_INC and SSL_LIB settings I told it to use, and autodetecting the 
openssl in /usr/local.


But even if I am wrong about that, I did work out how to achieve my 
goals.  I built openssl with --prefix=/usr/local/ssl3 and made a symlink 
for its "openssl" binary to /usr/local/bin/ossl. I get to have the 
custom openssl installed and available with an altered command, but now 
haproxy's build system won't find it.


Thanks,
Shawn

Re: How to compile with packaged openssl when custom openssl installed?

[Илья Шипицин]

ср, 3 нояб. 2021 г. в 18:57, Shawn Heisey :

> I ran into a problem when I compiled haproxy 2.4.8.  I had installed
> openssl 3.0.1-dev from source between installing haproxy 2.4.7 and
> 2.4.8, and haproxy's build system picked up the newer openssl instead of
> the one packaged by Ubuntu.
>
> I tried adding SSL_INC and SSL_LIB to my make command:
>
>  make TARGET=linux-glibc \
>USE_PCRE2_JIT=1 \
>USE_OPENSSL=1 \
>SSL_INC=/usr/include/openssl \
>SSL_LIB=/usr/lib/x86_64-linux-gnu \
>

you either need to specify LD_LIBRARY_PATH or add rpath during link,
here's example how to use rpath via ADDLIB haproxy/.travis.yml at
57610c694e56a6b0d55bf42f1170bad93b7b3297 · haproxy/haproxy (github.com)




>USE_ZLIB=1 \
>USE_SYSTEMD=1 \
>CPU=native
>
> This didn't work.  The build continued to find openssl in /usr/local,
> which caused it to fail because haproxy doesn't support openssl 3.x
> yet.  Is there any way to get this working other than uninstalling the
> custom openssl?  Uninstalling the custom openssl does work, but I would
> prefer to not take that step.
>
> Thanks,
> Shawn
>
>
>

Re: How to compile with packaged openssl when custom openssl installed?

[Lukas Tribus]

Use the instructions in INSTALL to build openssl statically. Building
and installing a custom shared build of openssl on a OS is something
that I'd suggest you avoid, because it will become complicated.

Lukas

How to compile with packaged openssl when custom openssl installed?

[Shawn Heisey]

I ran into a problem when I compiled haproxy 2.4.8.  I had installed 
openssl 3.0.1-dev from source between installing haproxy 2.4.7 and 
2.4.8, and haproxy's build system picked up the newer openssl instead of 
the one packaged by Ubuntu.


I tried adding SSL_INC and SSL_LIB to my make command:

    make TARGET=linux-glibc \
  USE_PCRE2_JIT=1 \
  USE_OPENSSL=1 \
  SSL_INC=/usr/include/openssl \
  SSL_LIB=/usr/lib/x86_64-linux-gnu \
  USE_ZLIB=1 \
  USE_SYSTEMD=1 \
  CPU=native

This didn't work.  The build continued to find openssl in /usr/local, 
which caused it to fail because haproxy doesn't support openssl 3.x 
yet.  Is there any way to get this working other than uninstalling the 
custom openssl?  Uninstalling the custom openssl does work, but I would 
prefer to not take that step.


Thanks,
Shawn