[Touch-packages] [Bug 1763870] Re: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item)
Is there anything I can do to accelerate the resolution of this bug for Xenial and Bionic? I already posted a proposed solution for the specific symbol that I am missing, but in theory fixing this should be as easy as running whatever script the package mantainers used in the first place to generate the file debian/patches/version-script.patch for this package. ** Also affects: openssl1.0 (Ubuntu) Importance: Undecided Status: New ** No longer affects: openssl1.0 (Ubuntu Xenial) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssl in Ubuntu. https://bugs.launchpad.net/bugs/1763870 Title: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item) Status in openssl package in Ubuntu: Fix Released Status in openssl1.0 package in Ubuntu: New Status in openssl source package in Xenial: New Status in openssl package in Debian: Fix Released Bug description: I'm developing an ENGINE for OpenSSL, and close to release, I noticed that in Ubuntu 16.04 LTS and in Debian oldstable-backports the build fails with the following output: ``` /usr/bin/cc -fPIC -g -shared -Wl,-soname,liblibsuola.so -o liblibsuola.so CMakeFiles/suola.dir/suola.c.o CMakeFiles/suola.dir/suola_keypair.c.o CMakeFiles/suola.dir/debug/debug.c.o CMakeFiles/suola.dir/meths/X25519_meth.c.o CMakeFiles/suola.dir/meths/ed25519_meth.c.o CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o CMakeFiles/suola.dir/meths/suola_md_identity_meth.c.o CMakeFiles/suola.dir/ossl/ossl_compat.c.o CMakeFiles/suola.dir/ossl/suola_err.c.o CMakeFiles/suola.dir/ossl/suola_objects.c.o CMakeFiles/suola.dir/providers/libsodium/base.c.o CMakeFiles/suola.dir/providers/libsodium/curve25519.c.o CMakeFiles/suola.dir/providers/libsodium/ed25519.c.o -lssl -lcrypto /opt/libsodium-stable/lib/libsodium.so -Wl,-z,defs -Wl,-rpath,/opt/libsodium-stable/lib: CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o: In function `suola_register_asn1_meth': /usr/local/src/libsuola/meths/suola_asn1_meth.c:505: undefined reference to `EVP_PKEY_asn1_set_item' collect2: error: ld returned 1 exit status make[2]: *** [liblibsuola.so] Error 1 CMakeFiles/suola.dir/build.make:412: recipe for target 'liblibsuola.so' failed make[2]: Leaving directory '/usr/local/src/libsuola/build' make[1]: *** [CMakeFiles/suola.dir/all] Error 2 make: *** [all] Error 2 ``` This does not happen linking against the same exact release of openssl compiled from source on the same system. I then learned that one of the patches applied by Debian and inherited also by Ubuntu has the goal of versioning library symbols to avoid conflicts. Unfortunately said patch is not updated regularly with each release of OpenSSL, resulting, like in my case, in symbols available in the public header files but masked through versioning in the shared library binary. The attached patch fixes my need by adding `EVP_PKEY_asn1_set_item` to the list, but you might consider an internal review of your release process to make sure that the list of symbols is updated whenever a new upstream releases makes new functions publicly available. I believe this bug is important, as it stops everyone using official packages from using third-party ENGINEs that require to use that function to set special handling of ASN.1 format, which basically includes every ENGINE that would add support for cryptosystems that upstream OpenSSL does not support (defying the purpose of using some ENGINEs). The patch I propose covers my use case, but basically the package as is results unusable to any user of any application that may require functions available in the public headers but accidentally masked in the symbol versioning step. The ideal outcome of fixing this issue would consist in making the versioning patch dynamic, checking when symbols are added (or removed) in newer releases and updating the list accordingly. You might have the same versioning patch applied in other releases, so it's worth tagging this bug also for those to make the handling of the exposed symbols consistent. Finally I also opened a similar bug against the equivalent Debian oldstable-backports package, but I opened an Ubuntu-specific bug report because 16.04 ships a more recent version of the package and the inherited symbol versioning patch has already been modified compared with the original Debian one. For reference this is the Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895547 -- System information -- # lsv_release -rd Description:Ubuntu 16.04.4 LTS Release:16.04 # apt-cache policy libssl-dev libssl-dev: Installed: 1.0.2g-1ubuntu4.11 Candidate: 1.0.2g-1ubuntu4.11 Version table: *** 1.0.2g-1ubuntu4.11 500 500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
[Touch-packages] [Bug 1763870] Re: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item)
Here is a minimal working example to test the issue: https://gist.github.com/romen/b95e99b3563a8ba4c27d88512c7932ff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssl in Ubuntu. https://bugs.launchpad.net/bugs/1763870 Title: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item) Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Xenial: New Status in openssl package in Debian: Fix Released Bug description: I'm developing an ENGINE for OpenSSL, and close to release, I noticed that in Ubuntu 16.04 LTS and in Debian oldstable-backports the build fails with the following output: ``` /usr/bin/cc -fPIC -g -shared -Wl,-soname,liblibsuola.so -o liblibsuola.so CMakeFiles/suola.dir/suola.c.o CMakeFiles/suola.dir/suola_keypair.c.o CMakeFiles/suola.dir/debug/debug.c.o CMakeFiles/suola.dir/meths/X25519_meth.c.o CMakeFiles/suola.dir/meths/ed25519_meth.c.o CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o CMakeFiles/suola.dir/meths/suola_md_identity_meth.c.o CMakeFiles/suola.dir/ossl/ossl_compat.c.o CMakeFiles/suola.dir/ossl/suola_err.c.o CMakeFiles/suola.dir/ossl/suola_objects.c.o CMakeFiles/suola.dir/providers/libsodium/base.c.o CMakeFiles/suola.dir/providers/libsodium/curve25519.c.o CMakeFiles/suola.dir/providers/libsodium/ed25519.c.o -lssl -lcrypto /opt/libsodium-stable/lib/libsodium.so -Wl,-z,defs -Wl,-rpath,/opt/libsodium-stable/lib: CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o: In function `suola_register_asn1_meth': /usr/local/src/libsuola/meths/suola_asn1_meth.c:505: undefined reference to `EVP_PKEY_asn1_set_item' collect2: error: ld returned 1 exit status make[2]: *** [liblibsuola.so] Error 1 CMakeFiles/suola.dir/build.make:412: recipe for target 'liblibsuola.so' failed make[2]: Leaving directory '/usr/local/src/libsuola/build' make[1]: *** [CMakeFiles/suola.dir/all] Error 2 make: *** [all] Error 2 ``` This does not happen linking against the same exact release of openssl compiled from source on the same system. I then learned that one of the patches applied by Debian and inherited also by Ubuntu has the goal of versioning library symbols to avoid conflicts. Unfortunately said patch is not updated regularly with each release of OpenSSL, resulting, like in my case, in symbols available in the public header files but masked through versioning in the shared library binary. The attached patch fixes my need by adding `EVP_PKEY_asn1_set_item` to the list, but you might consider an internal review of your release process to make sure that the list of symbols is updated whenever a new upstream releases makes new functions publicly available. I believe this bug is important, as it stops everyone using official packages from using third-party ENGINEs that require to use that function to set special handling of ASN.1 format, which basically includes every ENGINE that would add support for cryptosystems that upstream OpenSSL does not support (defying the purpose of using some ENGINEs). The patch I propose covers my use case, but basically the package as is results unusable to any user of any application that may require functions available in the public headers but accidentally masked in the symbol versioning step. The ideal outcome of fixing this issue would consist in making the versioning patch dynamic, checking when symbols are added (or removed) in newer releases and updating the list accordingly. You might have the same versioning patch applied in other releases, so it's worth tagging this bug also for those to make the handling of the exposed symbols consistent. Finally I also opened a similar bug against the equivalent Debian oldstable-backports package, but I opened an Ubuntu-specific bug report because 16.04 ships a more recent version of the package and the inherited symbol versioning patch has already been modified compared with the original Debian one. For reference this is the Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895547 -- System information -- # lsv_release -rd Description:Ubuntu 16.04.4 LTS Release:16.04 # apt-cache policy libssl-dev libssl-dev: Installed: 1.0.2g-1ubuntu4.11 Candidate: 1.0.2g-1ubuntu4.11 Version table: *** 1.0.2g-1ubuntu4.11 500 500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages 100 /var/lib/dpkg/status 1.0.2g-1ubuntu4 500 500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: libssl-dev 1.0.2g-1ubuntu4.11 ProcVersionSignature: Ubuntu 4.13.0-36.40~16.04.1-generic 4.13.13 Uname: Linux 4.13.0-36-generic x86_6
[Touch-packages] [Bug 1763870] Re: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item)
I can also further confirm that even on the latest LTS, bionic, the same problem still exists today when using the latest version of the official packages `openssl1.0`, `libssl1.0.0`, and `libssl1.0-dev` to have access OpenSSL 1.0.2. So I would also formally request to reevaluate the "fixed" status for Ubuntu releases after xenial that are still offering official packages for OpenSSL <1.1.0. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssl in Ubuntu. https://bugs.launchpad.net/bugs/1763870 Title: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item) Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Xenial: New Status in openssl package in Debian: Fix Released Bug description: I'm developing an ENGINE for OpenSSL, and close to release, I noticed that in Ubuntu 16.04 LTS and in Debian oldstable-backports the build fails with the following output: ``` /usr/bin/cc -fPIC -g -shared -Wl,-soname,liblibsuola.so -o liblibsuola.so CMakeFiles/suola.dir/suola.c.o CMakeFiles/suola.dir/suola_keypair.c.o CMakeFiles/suola.dir/debug/debug.c.o CMakeFiles/suola.dir/meths/X25519_meth.c.o CMakeFiles/suola.dir/meths/ed25519_meth.c.o CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o CMakeFiles/suola.dir/meths/suola_md_identity_meth.c.o CMakeFiles/suola.dir/ossl/ossl_compat.c.o CMakeFiles/suola.dir/ossl/suola_err.c.o CMakeFiles/suola.dir/ossl/suola_objects.c.o CMakeFiles/suola.dir/providers/libsodium/base.c.o CMakeFiles/suola.dir/providers/libsodium/curve25519.c.o CMakeFiles/suola.dir/providers/libsodium/ed25519.c.o -lssl -lcrypto /opt/libsodium-stable/lib/libsodium.so -Wl,-z,defs -Wl,-rpath,/opt/libsodium-stable/lib: CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o: In function `suola_register_asn1_meth': /usr/local/src/libsuola/meths/suola_asn1_meth.c:505: undefined reference to `EVP_PKEY_asn1_set_item' collect2: error: ld returned 1 exit status make[2]: *** [liblibsuola.so] Error 1 CMakeFiles/suola.dir/build.make:412: recipe for target 'liblibsuola.so' failed make[2]: Leaving directory '/usr/local/src/libsuola/build' make[1]: *** [CMakeFiles/suola.dir/all] Error 2 make: *** [all] Error 2 ``` This does not happen linking against the same exact release of openssl compiled from source on the same system. I then learned that one of the patches applied by Debian and inherited also by Ubuntu has the goal of versioning library symbols to avoid conflicts. Unfortunately said patch is not updated regularly with each release of OpenSSL, resulting, like in my case, in symbols available in the public header files but masked through versioning in the shared library binary. The attached patch fixes my need by adding `EVP_PKEY_asn1_set_item` to the list, but you might consider an internal review of your release process to make sure that the list of symbols is updated whenever a new upstream releases makes new functions publicly available. I believe this bug is important, as it stops everyone using official packages from using third-party ENGINEs that require to use that function to set special handling of ASN.1 format, which basically includes every ENGINE that would add support for cryptosystems that upstream OpenSSL does not support (defying the purpose of using some ENGINEs). The patch I propose covers my use case, but basically the package as is results unusable to any user of any application that may require functions available in the public headers but accidentally masked in the symbol versioning step. The ideal outcome of fixing this issue would consist in making the versioning patch dynamic, checking when symbols are added (or removed) in newer releases and updating the list accordingly. You might have the same versioning patch applied in other releases, so it's worth tagging this bug also for those to make the handling of the exposed symbols consistent. Finally I also opened a similar bug against the equivalent Debian oldstable-backports package, but I opened an Ubuntu-specific bug report because 16.04 ships a more recent version of the package and the inherited symbol versioning patch has already been modified compared with the original Debian one. For reference this is the Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895547 -- System information -- # lsv_release -rd Description:Ubuntu 16.04.4 LTS Release:16.04 # apt-cache policy libssl-dev libssl-dev: Installed: 1.0.2g-1ubuntu4.11 Candidate: 1.0.2g-1ubuntu4.11 Version table: *** 1.0.2g-1ubuntu4.11 500 500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages 100 /var/lib/dpkg/status 1.0.2g-1ubu
[Touch-packages] [Bug 1763870] Re: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item)
I can confirm that while this is "fixed" by updating to a newer release of Ubuntu (or Debian, as can be read in the Debian bug tracker), this problem is still present in Xenial. The reason it is fixed in newer releases is that they ship OpenSSL >= 1.1.0 where upstream takes care of symbol versioning for the public symbols in the library. Xenial uses OpenSSL 1.0.2, and the package maintainers have a custom script to tag known symbols in the library with a version number, and mask all the symbols not explicitly whitelisted in the script. As reported more than 1 year ago, this script has not been properly maintained, so in Xenial, today, there are still symbols like `EVP_PKEY_asn1_set_item` that are exposed to application developers as part of the public API in the headers contained in the latest package version for libssl-dev, but that are not linkable because they have never been whitelisted by the package maintainers. In Debian this is now considered Fixed, as the currently supported releases all ship OpenSSL >=1.1.0, but Ubuntu Xenial is supposed to be supported until April 2021. What can I do to get some attention to this issue and fix the problem in Xenial? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssl in Ubuntu. https://bugs.launchpad.net/bugs/1763870 Title: openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item) Status in openssl package in Ubuntu: Fix Released Status in openssl source package in Xenial: New Status in openssl package in Debian: Fix Released Bug description: I'm developing an ENGINE for OpenSSL, and close to release, I noticed that in Ubuntu 16.04 LTS and in Debian oldstable-backports the build fails with the following output: ``` /usr/bin/cc -fPIC -g -shared -Wl,-soname,liblibsuola.so -o liblibsuola.so CMakeFiles/suola.dir/suola.c.o CMakeFiles/suola.dir/suola_keypair.c.o CMakeFiles/suola.dir/debug/debug.c.o CMakeFiles/suola.dir/meths/X25519_meth.c.o CMakeFiles/suola.dir/meths/ed25519_meth.c.o CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o CMakeFiles/suola.dir/meths/suola_md_identity_meth.c.o CMakeFiles/suola.dir/ossl/ossl_compat.c.o CMakeFiles/suola.dir/ossl/suola_err.c.o CMakeFiles/suola.dir/ossl/suola_objects.c.o CMakeFiles/suola.dir/providers/libsodium/base.c.o CMakeFiles/suola.dir/providers/libsodium/curve25519.c.o CMakeFiles/suola.dir/providers/libsodium/ed25519.c.o -lssl -lcrypto /opt/libsodium-stable/lib/libsodium.so -Wl,-z,defs -Wl,-rpath,/opt/libsodium-stable/lib: CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o: In function `suola_register_asn1_meth': /usr/local/src/libsuola/meths/suola_asn1_meth.c:505: undefined reference to `EVP_PKEY_asn1_set_item' collect2: error: ld returned 1 exit status make[2]: *** [liblibsuola.so] Error 1 CMakeFiles/suola.dir/build.make:412: recipe for target 'liblibsuola.so' failed make[2]: Leaving directory '/usr/local/src/libsuola/build' make[1]: *** [CMakeFiles/suola.dir/all] Error 2 make: *** [all] Error 2 ``` This does not happen linking against the same exact release of openssl compiled from source on the same system. I then learned that one of the patches applied by Debian and inherited also by Ubuntu has the goal of versioning library symbols to avoid conflicts. Unfortunately said patch is not updated regularly with each release of OpenSSL, resulting, like in my case, in symbols available in the public header files but masked through versioning in the shared library binary. The attached patch fixes my need by adding `EVP_PKEY_asn1_set_item` to the list, but you might consider an internal review of your release process to make sure that the list of symbols is updated whenever a new upstream releases makes new functions publicly available. I believe this bug is important, as it stops everyone using official packages from using third-party ENGINEs that require to use that function to set special handling of ASN.1 format, which basically includes every ENGINE that would add support for cryptosystems that upstream OpenSSL does not support (defying the purpose of using some ENGINEs). The patch I propose covers my use case, but basically the package as is results unusable to any user of any application that may require functions available in the public headers but accidentally masked in the symbol versioning step. The ideal outcome of fixing this issue would consist in making the versioning patch dynamic, checking when symbols are added (or removed) in newer releases and updating the list accordingly. You might have the same versioning patch applied in other releases, so it's worth tagging this bug also for those to make the handling of the exposed symbols consistent. Finally I also opened a similar bug against the equivalent Debian oldstable-backports package, but I opened an Ubuntu-sp
[Touch-packages] [Bug 1763870] [NEW] openssl: After symbol versioning, distributed pkgs are missing API symbols (e.g. EVP_PKEY_asn1_set_item)
Public bug reported: I'm developing an ENGINE for OpenSSL, and close to release, I noticed that in Ubuntu 16.04 LTS and in Debian oldstable-backports the build fails with the following output: ``` /usr/bin/cc -fPIC -g -shared -Wl,-soname,liblibsuola.so -o liblibsuola.so CMakeFiles/suola.dir/suola.c.o CMakeFiles/suola.dir/suola_keypair.c.o CMakeFiles/suola.dir/debug/debug.c.o CMakeFiles/suola.dir/meths/X25519_meth.c.o CMakeFiles/suola.dir/meths/ed25519_meth.c.o CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o CMakeFiles/suola.dir/meths/suola_md_identity_meth.c.o CMakeFiles/suola.dir/ossl/ossl_compat.c.o CMakeFiles/suola.dir/ossl/suola_err.c.o CMakeFiles/suola.dir/ossl/suola_objects.c.o CMakeFiles/suola.dir/providers/libsodium/base.c.o CMakeFiles/suola.dir/providers/libsodium/curve25519.c.o CMakeFiles/suola.dir/providers/libsodium/ed25519.c.o -lssl -lcrypto /opt/libsodium-stable/lib/libsodium.so -Wl,-z,defs -Wl,-rpath,/opt/libsodium-stable/lib: CMakeFiles/suola.dir/meths/suola_asn1_meth.c.o: In function `suola_register_asn1_meth': /usr/local/src/libsuola/meths/suola_asn1_meth.c:505: undefined reference to `EVP_PKEY_asn1_set_item' collect2: error: ld returned 1 exit status make[2]: *** [liblibsuola.so] Error 1 CMakeFiles/suola.dir/build.make:412: recipe for target 'liblibsuola.so' failed make[2]: Leaving directory '/usr/local/src/libsuola/build' make[1]: *** [CMakeFiles/suola.dir/all] Error 2 make: *** [all] Error 2 ``` This does not happen linking against the same exact release of openssl compiled from source on the same system. I then learned that one of the patches applied by Debian and inherited also by Ubuntu has the goal of versioning library symbols to avoid conflicts. Unfortunately said patch is not updated regularly with each release of OpenSSL, resulting, like in my case, in symbols available in the public header files but masked through versioning in the shared library binary. The attached patch fixes my need by adding `EVP_PKEY_asn1_set_item` to the list, but you might consider an internal review of your release process to make sure that the list of symbols is updated whenever a new upstream releases makes new functions publicly available. I believe this bug is important, as it stops everyone using official packages from using third-party ENGINEs that require to use that function to set special handling of ASN.1 format, which basically includes every ENGINE that would add support for cryptosystems that upstream OpenSSL does not support (defying the purpose of using some ENGINEs). The patch I propose covers my use case, but basically the package as is results unusable to any user of any application that may require functions available in the public headers but accidentally masked in the symbol versioning step. The ideal outcome of fixing this issue would consist in making the versioning patch dynamic, checking when symbols are added (or removed) in newer releases and updating the list accordingly. You might have the same versioning patch applied in other releases, so it's worth tagging this bug also for those to make the handling of the exposed symbols consistent. Finally I also opened a similar bug against the equivalent Debian oldstable-backports package, but I opened an Ubuntu-specific bug report because 16.04 ships a more recent version of the package and the inherited symbol versioning patch has already been modified compared with the original Debian one. For reference this is the Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895547 -- System information -- # lsv_release -rd Description:Ubuntu 16.04.4 LTS Release:16.04 # apt-cache policy libssl-dev libssl-dev: Installed: 1.0.2g-1ubuntu4.11 Candidate: 1.0.2g-1ubuntu4.11 Version table: *** 1.0.2g-1ubuntu4.11 500 500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages 100 /var/lib/dpkg/status 1.0.2g-1ubuntu4 500 500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: libssl-dev 1.0.2g-1ubuntu4.11 ProcVersionSignature: Ubuntu 4.13.0-36.40~16.04.1-generic 4.13.13 Uname: Linux 4.13.0-36-generic x86_64 ApportVersion: 2.20.1-0ubuntu2.16 Architecture: amd64 Date: Fri Apr 13 21:10:58 2018 ProcEnviron: TERM=xterm PATH=(custom, no user) SourcePackage: openssl UpgradeStatus: No upgrade log present (probably fresh install) ** Affects: openssl Importance: Unknown Status: Unknown ** Affects: openssl (Ubuntu) Importance: Undecided Status: New ** Affects: openssl (Debian) Importance: Unknown Status: Unknown ** Tags: amd64 apport-bug openssl xenial ** Patch added: "Patch for the symbol versioning patch" https://bugs.launchpad.net/bugs/1763870/+attachment/5114701/+files/add_EVP_PKEY_asn1_set_item.patch ** Bug watch added: Debian Bug tracker #
