It's been a while since the last discussion of potential approaches to
improving RPM's elfdeps:
https://lists.fedorahosted.org/archives/list/[email protected]/thread/EZFX5ARQWOXBUXID4HH74ETD2QBF2DPB/#EZFX5ARQWOXBUXID4HH74ETD2QBF2DPB
Since then, we continue to see occasional bug reports that rpm/dnf will
install packages that fail at runtime because their binary dependencies
are incomplete, in Fedora and in RHEL.
## The problem:
The short version of the problem is that for libraries that do not
provide versioned symbols (about 85% of libraries in Fedora, I think),
rpm's "elfdeps" will generate a dependency expression that effectively
provides only a major version.
So, the libsoup3 rpm requires "libnghttp2.so.14()(64bit)" which is
"libnghttp2.so, ABI version >= 14.0 and < 15", but the binary may depend
on symbols that were introduced sometime after the original release of
libnghttp2. The binary's dependency might be on ABI version >= 14.28,
even if rpm's expression of that dependency only includes the major
version (which is part of the soname).
## For example:
If libnghttp2 is rebased during a Fedora release (or if the maintainers
decide to introduce new symbols and bump the ABI minor version,
mid-release), any binaries built after that could require the new ABI.
So you might install from the installation media, and then:
$ sudo dnf install chezdav
$ chezdav
chezdav: symbol lookup error: /lib64/libsoup-3.0.so.0: undefined symbol:
nghttp2_option_set_no_rfc9113_leading_and_trailing_ws_validation
## The solution:
rpm's elfdeps should generate "Provides" and "Requires" that include at
least a major and minor number. Those numbers could be the major and
minor number of the DSO version ("libnghttp2.so.14.28.3" might result in
providing "libnghttp2.so.14()(64bit) = 14.28") or they could be the
version of the rpm in the build root ("libexpat.so.1.11.0" might result
in providing "libexpat.so.1()(64bit) = 2.7.2"), or some other version
source.
Today, there are hundreds of packages that appear to be solving this
problem by manually recording more specific dependency versions:
rpm-specs]$ grep -r -E '^Requires:\s*lib.*>=' .| grep -v '{version}' |
cut -f1 -d: | uniq | wc -l
267
But in many cases, that requires developers to react to bug reports
caused by runtime failures.
## Approaches to consider:
We could use the DSO version, extracted from the symlink. In this case,
the expat-2.7.3 package would provide "libexpat.so.1()(64bit) = 1.11.1"
and any package that links to libexpat.so.1 would require
"libexpat.so.1()(64bit) >= 1.11.1"
One drawback to this approach is that a dependent binary package would
require expat 2.7.3, even if they did not use any of the symbols that
are new in that version, relative to expat-2.7.1. Some users might want
to build on a system with expat-2.7.3 and deploy on a system with
expat-2.7.1, as long as the binary is able to execute. Adding a more
specific version to the requirement would result in dnf updating expat
when the new binary is installed.
Another potential drawback is that there are a few cases of
interchangeable implementations of shared libraries in Fedora, and this
would impose the additional requirement that anyone developing a library
intended as an interoperable substitute would need to keep their DSO
version in sync with the other implementations.
Personally, I think that all of this is the correct behavior. It is also
the simplest implementation that makes rpm dependency resolution
reliable (in the vast majority of cases).
https://github.com/rpm-software-management/rpm/pull/2372
We could also use a version that the package maintainer specifies (one
that defaults to %{version} but can be overridden by defining a global
value in the rpm spec). When generating the Requires: dependency
expression, the version would be generated by examining the rpm DB.
In this case, the expat-2.7.3 package might provide
"libexpat.so.1()(64bit) = 2.7.3" and any package that links to
libexpat.so.1 would require "libexpat.so.1()(64bit) >= 2.7.3". Or, if
the package maintainer chose to closely monitor and specify the ABI
version, the expat-2.7.3 package might provide "libexpat.so.1()(64bit) =
1.11" and any package that links to libexpat.so.1 would require
"libexpat.so.1()(64bit) >= 1.11".
This approach has some of the drawbacks of the one above, by default. If
the maintainer does not override the ABI version value, then dependent
packages still get a dependency on the version of libraries present in
their build root. But if they do specify the ABI version, and don't
monitor the DSO for changes, they might recreate the problem that exists
today.
This version is a little bit better for developers / maintainers of
interchangeable libraries, but risky for maintainers that choose to
specify an ABI version rather than using the default.
https://github.com/rpm-software-management/rpm/pull/4081
We could also do what Debian does, and require package maintainers to
keep a symbol version map. In this case, after a build, the DSO can be
analyzed for new symbols, and those symbols can be added to a file that
lists the symbols exported by a shared library and the version in which
they first appeared. That file needs to be kept in the package repo, and
included in the -devel sub-package.
I dislike that approach because it requires extra work from package
maintainers, and because the symbol analysis has to be done post-build,
which might mean doing another build to include the updated symbol map file.
It's also nearly identical to the work required to simply add versioned
symbols to shared libraries, which is a much better option, and doing
the same amount of work to get a result that isn't as good doesn't make
sense to me.
abidb is a repository of ABI information generated and maintained by the
libabigail project. A global DB of files, with symbols and the rpm
version in which they first appeared could back a solution that is
largely like Debian's approach, but without individual package
maintainer overhead
The abidb is currently around 22GB for Fedora 43 x86_64, but the
information needed for the feature can be extracted into an sqlite3 file
of roughly 500MB, which can be xz compressed to around 75MB. So the
overhead for this tool would be sending an extra 75MB file to every
build job, and uncompressing that file before the build, adding another
~ 4s to the runtime.
Because this approach uses symbol-level analysis, the Requires:
dependency expressions generated by elfdeps would be no more specific
than actually necessary to produce an installation that will pass
runtime linking. That contrasts with the first two approaches, which
will both typically depend on the version that appeared in the build
root, regardless of whether any new symbols were used.
https://github.com/gordonmessmer/build-abidb-sqlite
https://github.com/gordonmessmer/rpm/commit/464340b739fab0481d972751f80ac8a5fdc7418a
--
_______________________________________________
devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
