Re: OpenSSL version 3.3.0 published

[Neil Horman]

You are correct, the files you reference (most of them in fact) get built
into separate objects in the event the build flags are different for shared
and static libraries, and should be unrelated to the issue you are seeing

As for the undefined symbols, thats definitely a mystery.  most notably,
the symbols referenced are internal.  That is to say they shouldn't be
exported in the symbol table for libssl.so (though a quick look with
objectdump shows they are, which may be a separate issue)

Looking at the sources, I can see what _might_ be happening

cert_comp_tests.c includes "../ssl/ssl_local.h"
if quic is enabled (i.e. OPENSSL_NO_QUIC is undefined), ssl_local.h
includes quic/quic_local.h
quic_local.h includes internal/quic_txp.h
quic_txp.h includes internal/quic_stream_map.h
quic_stream_map.h defines a static inline function
named ossl_quic_stream_recv_get_final_size which calls
ossl_quic_rxfc_get_final_size, which in turn
calls ossl_quic_rxfc_get_final_size

I'm guessing the other symbols have simmilar patterns.

As to why its happening my first guess would be that more recent compilers
like gcc and clang do lazy symbol resolution, only resolving a subordonate
symbol, when its calling parent is found to be used.  Given
ossl_quic_stream_recv_get_final_size isn't called anywhere in
comp_stream_test.c, the compiler disposes of the symbol prior to any need
to resolve its called symbols, and so everything is ok.

conversely (again, I'm guessing here) the solaris 5.10 compiler likely take
a more bulldozer-ish approach, leaving everything in the object file and
only stripping symbols after all resolutions are complete, leading to the
missing symbols error, despite its not being needed.

As to what to do about this...I'm unsure.  The quick hack I would imagine
would be to move the definition of ossl_quic_stream_recv_get_final_size
into a c file (likely quic_stream_map.c) and just declare a prototype in
the quic_stream_map.h header, so as to avoid the unneeded symbol
resolution.  You would have to lather rinse  repeat with the other missing
symbols of course.

As to your prior question about how long the ability to support SunOS will
last, well, unfortunately I don't think any of us can say.  I believe the
platoform you are building on is on our unadpoted platform list:
https://www.openssl.org/policies/general-supplemental/platforms.html

And while we endeavor to keep openssl building on as many platforms as
possible, its not feasible to cover all the currently
unmaintained platforms.  You do have some agency here however. If you are
willing and interested, you could volunteer to be a community platform
maintainer for your target platform.  This would entail you building
openssl on your adopted platform, and running the test suite routinely,
reporting bugs and fixing errors as they occur.  Its not a small amount of
work, but it would be a significant contribution toward ensuring that
openssl stays viable on the targets you need.

Regards
Neil



On Wed, May 15, 2024 at 4:12 PM Dennis Clarke  wrote:

> On 5/13/24 03:34, Matt Caswell wrote:
> >
> >
> > On 13/05/2024 02:42, Neil Horman wrote:
> >> We added support for RCU locks in 3.3 which required the use of
> >> atomics (or emulated atomic where they couldn't be supported), but
> >> those were in libcrypro not liberal
> >>
> >
> > Right - its supposed to fallback to emulated atomic calls where atomics
> > aren't available on a particular platform.
> >
> > Some platforms have some atomics support but you have to link in a
> > separate atomics library to get it to work. You might try adding
> > "-latomic" to Configure command line and see if that helps at all.
> >
> Well first the good news : managed to get past the need for C11 atomics
> with the bundled libatomic.so.1 that the Oracle people provide in the
> dev tools.
>
>   So that works now.  Yay.
>
> Now comes the next horrible hurdle to jump and that seems to be called
> the quic protocol goodness.  For the record I am able to get a good
> result if I go with "no-quic" in the config :
>
> hubble $ $PERL ./Configure solaris64-sparcv9-cc \
>  > --prefix=/opt/bw no-asm no-engine shared zlib-dynamic \
>  > no-quic enable-weak-ssl-ciphers -DPEDANTIC 2>&1
> Configuring OpenSSL version 3.3.0 for target solaris64-sparcv9-cc
> Using os-specific seed configuration
> Created configdata.pm
> Running configdata.pm
> Created Makefile.in
> Created Makefile
> Created include/openssl/configuration.h
>
> **
> ******
> ***   OpenSSL has been successfully configured ***
> ******
> ***   If you encounter a problem while building, please open an***
> ***   issue on GitHub   ***
> ***   and include the output from the following command:   ***
> ***

Re: OpenSSL version 3.3.0 published

[Dennis Clarke via openssl-users]

On 5/13/24 03:34, Matt Caswell wrote:



On 13/05/2024 02:42, Neil Horman wrote:
We added support for RCU locks in 3.3 which required the use of 
atomics (or emulated atomic where they couldn't be supported), but 
those were in libcrypro not liberal




Right - its supposed to fallback to emulated atomic calls where atomics 
aren't available on a particular platform.


Some platforms have some atomics support but you have to link in a 
separate atomics library to get it to work. You might try adding 
"-latomic" to Configure command line and see if that helps at all.



Well first the good news : managed to get past the need for C11 atomics
with the bundled libatomic.so.1 that the Oracle people provide in the
dev tools.

 So that works now.  Yay.

Now comes the next horrible hurdle to jump and that seems to be called
the quic protocol goodness.  For the record I am able to get a good
result if I go with "no-quic" in the config :

hubble $ $PERL ./Configure solaris64-sparcv9-cc \
> --prefix=/opt/bw no-asm no-engine shared zlib-dynamic \
> no-quic enable-weak-ssl-ciphers -DPEDANTIC 2>&1
Configuring OpenSSL version 3.3.0 for target solaris64-sparcv9-cc
Using os-specific seed configuration
Created configdata.pm
Running configdata.pm
Created Makefile.in
Created Makefile
Created include/openssl/configuration.h

**
******
***   OpenSSL has been successfully configured ***
******
***   If you encounter a problem while building, please open an***
***   issue on GitHub   ***
***   and include the output from the following command:   ***
******
***   perl configdata.pm --dump***
******
***   (If you are new to OpenSSL, you might want to consult the***
***   'Troubleshooting' section in the INSTALL.md file first)  ***
******
**
hubble $


That all builds neatly on this old platform and all the testsuite looks
to be sweet :

.
.
.
All tests successful.
Files=312, Tests=3182, 6723 wallclock secs (25.17 usr  3.15 sys + 
6375.57 cusr 171.52 csys = 6575.41 CPU)

Result: PASS
`test' is up to date.

So that is cute.

However, if I leave in the "quic"-ness then I eventually land on this
weird linking problem :

Undefined   first referenced
 symbol in file
ossl_quic_rxfc_get_final_size   test/cert_comp_test-bin-cert_comp_test.o
ossl_quic_sstream_get_final_sizetest/cert_comp_test-bin-cert_comp_test.o
ossl_quic_vlint_decode_uncheckedtest/cert_comp_test-bin-cert_comp_test.o
ld: fatal: symbol referencing errors. No output written to 
test/cert_comp_test

*** Error code 2
make: Fatal error: Command failed for target `test/cert_comp_test'
Current working directory /opt/bw/build/openssl-3.3.0_SunOS_5.10_SPARC64.004
*** Error code 1
make: Fatal error: Command failed for target `build_sw'

These files refer to the above symbols :

1) headers
-rw-r--r--   1 dclarke  devl4670 Apr  9 12:12 
./include/internal/packet_quic.h
-rw-r--r--   1 dclarke  devl   10769 Apr  9 12:12 
./include/internal/quic_fc.h
-rw-r--r--   1 dclarke  devl   17692 Apr  9 12:12 
./include/internal/quic_stream.h
-rw-r--r--   1 dclarke  devl   34987 Apr  9 12:12 
./include/internal/quic_stream_map.h
-rw-r--r--   1 dclarke  devl4212 Apr  9 12:12 
./include/internal/quic_vlint.h


2) C sources
-rw-r--r--   1 dclarke  devl2060 Apr  9 12:12 ./crypto/quic_vlint.c
-rw-r--r--   1 dclarke  devl  121348 Apr  9 12:12 ./ssl/quic/quic_impl.c
-rw-r--r--   1 dclarke  devl   12010 Apr  9 12:12 
./ssl/quic/quic_sstream.c
-rw-r--r--   1 dclarke  devl   26592 Apr  9 12:12 
./ssl/quic/quic_stream_map.c
-rw-r--r--   1 dclarke  devl   17658 Apr  9 12:12 
./ssl/quic/quic_tserver.c

-rw-r--r--   1 dclarke  devl  114209 Apr  9 12:12 ./ssl/quic/quic_txp.c

Looking into my compile logs I see that quic_vlint.c gets processed into
three output objects :

{CC foo} -c -o crypto/libcrypto-lib-quic_vlint.o   crypto/quic_vlint.c
{CC foo} -c -o crypto/libcrypto-shlib-quic_vlint.o crypto/quic_vlint.c
{CC foo} -c -o crypto/libssl-shlib-quic_vlint.ocrypto/quic_vlint.c

I see that quic_impl.c gets processed into two output objects :

{CC foo} -c -o ssl/quic/libssl-lib-quic_impl.o ssl/quic/quic_impl.c
{CC foo} -c -o ssl/quic/libssl-shlib-quic_impl.o   ssl/quic/quic_impl.c


Similarly we see that quic_sstream.c results in two objects also :

output object file  input source