# [JFYI] Introduction of _rpms.lock.yaml_ file
## Context
My team is currently working on the implementation of a hermetic build process
for containers that use RPMs. The build process runs in a network-isolated
build environment. To be able to implement this, we need to pre-fetch all
required RPMs and a full chain of their transitive dependencies to be available
during the build process (except for packages that are already installed in the
parent container image). As part of this requirement, we also want to strive
towards reproducibility. To prefetch all required RPMs, including dependencies,
and to be able to pre-fetch the same set of them when we re-run the build with
the same input parameters, we need a "lock" file similar to one known from
Python - _requirements.txt_ that is programmatically generated from an input
file called _requirements.in_.
To be transparent and to give you a chance to provide feedback as RPM ecosystem
SMEs I would like to present to you the format of the lock file we designed.
For more details about our requirements for the container build process, you
can see SLSA requirements, especially these:
*
[https://slsa.dev/spec/v0.1/requirements#hermetic](https://slsa.dev/spec/v0.1/requirements#hermetic)
*
[https://slsa.dev/spec/v0.1/requirements#reproducible](https://slsa.dev/spec/v0.1/requirements#reproducible)
*
[https://slsa.dev/spec/v0.1/requirements#dependencies-complete](https://slsa.dev/spec/v0.1/requirements#dependencies-complete)
## rpms.lock.yaml
A file that contains a list of fully resolved dependencies (their URLs) that
cachi2
([https://github.com/containerbuildsystem/cachi2/](https://github.com/containerbuildsystem/cachi2/))
will need to download for a hermetic build. This file contains a different
list of RPMs per architecture. Only the RPMs listed in this file will be
available during the build process as the build process has no access to the
internet.
Note: This file contains only RPMs that will be installed on top of the parent
image - i.e. RPMs that are required but are already installed in the parent
container image are not included in this file.
This will be generated and maintained programmatically based on an input file
(_rpms.in.yaml_) that is out of the scope of this doc.
### đź“” Notes about format design
* YAML format is extensible, allows to write schema to allow simple validation,
is platform independent, and easily consumable by computers as well as humans.
* “repoid” is optional. If defined it will be propagated to the container build
so RPMs that will be installed during the container build process will have
this repoid listed as an origin repo ("From repo" when you run `dnf info
$PKG`). This is beneficial for example for a container vulnerability scanning
tool Clair.
* “sources” are optional. If provided, they will be collected by the cachi2
during the container build process to allow generation of source containers.
### ⚙️ Example
```
lockfileVersion: 1
arches:
- arch: x86_64
packages:
- url:
https://mirrors.nic.cz/pub/fedora/linux/updates/38/Everything/x86_64/Packages/v/vim-enhanced-9.1.031-1.fc38.x86_64.rpm
repoid: updates
- url:
https://mirrors.nic.cz/pub/fedora/linux/updates/38/Everything/x86_64/Packages/v/vim-common-9.1.031-1.fc38.x86_64.rpm
repoid: updates
- url:
https://mirrors.nic.cz/pub/fedora/linux/updates/38/Everything/x86_64/Packages/v/vim-filesystem-9.1.031-1.fc38.noarch.rpm
repoid: updates
- url:
https://mirrors.nic.cz/pub/fedora/linux/releases/38/Everything/x86_64/os/Packages/g/gpm-libs-1.20.7-42.fc38.x86_64.rpm
repoid: fedora
sources:
- url:
https://mirrors.nic.cz/pub/fedora/linux/updates/38/Everything/source/tree/Packages/v/vim-9.1.031-1.fc38.src.rpm
repoid: updates-source
- url:
https://mirrors.nic.cz/pub/fedora/linux/releases/38/Everything/source/tree/Packages/g/gpm-1.20.7-42.fc38.src.rpm
repoid: fedora-source
- arch: aarch64
packages:
... SNIP ...
```
We understand that managing such a lock file manually is going to be very
cumbersome and difficult. The long-term plan is to have a tool that will be
able to automate it. This however is not within the scope of our minimal viable
product.
## Possible future extensions
These are some possible extensions that we can envision may become relevant at
some point in future and can be easily added because of the YAML format, but
they are not planned right now as our use case doesn't need them.
* Support for metalink and mirrorlists.
* Support for modules.
* Checksums of the files.
* Mapping of RPMs to the source RPMs.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/discussions/2908
You are receiving this because you are subscribed to this thread.
Message ID: <rpm-software-management/rpm/repo-discussions/2...@github.com>_______________________________________________
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
