Hello Guix Users!
In the past I thought I had it figured out, thanks to help from someone on the
mailing list, how to run Ocaml projects in a reproducible (hashes/checksums
verified!) way.
It is December and I thought I would give Ocaml a try at Advent of Code this
year. However, I learned, that my previous project structure/setup no longer
works. I thought: "Why not check out how people, who don't use or know guix do
this?" and looked into opam. Long story short: Unfortunately, it does not
provide a lock file that one could commit into a repo to ensure reproducibility.
Even the tool opam-lock merely pins version numbers without hashes.
Disappointing. So I am back to trying to fix my guix using project setup. Last
stop before giving up on Ocaml for another year or so. In the following I will
describe my project setup, and why it hopefully is reproducible not only on my
machine, but also on other machines.
*The guix part*
I have the following files:
(1) guix-env/channels.scm: Specifying the exact guix channel that is used, and
thereby locking down hashes, which are specified in the guix repository itself.
~~~~
(list (channel
(name 'guix)
(url"https://git.guix.gnu.org/guix.git";)
(branch "master")
(commit
"7c6d8a6224cf3209efa179dbe1509759a580cb05")
(introduction
(make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
"BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA")))))
~~~~
(2) guix-env/manifest.scm: Specifying the package, that I want to use for my
project.
~~~~
(specifications->manifest
'("[email protected]"
"ocaml-utop"
"dune"
"bash"
"make"
"ocaml-findlib"
"ocaml-zarith"))
~~~~
(3) Makefile: Used as a task runner, basically, to avoid having to type
complicated commands.
~~~~ Makefile (variable definitions) ~~~~
.POSIX:
.RECIPEPREFIX = >
.DELETE_ON_ERROR:
MAKEFLAGS += --warn-undefined-variables
MAKEFLAGS += --no-builtin-rules
SHELL ::= guix time-machine --channels=guix-env/channels.scm -- shell --check
--manifest=guix-env/manifest.scm -- bash -c
.SHELLFLAGS ::= -Euxo pipefail -c
BUILD_DIR ::= _build
OCAML_PACKAGES ::= zarith
...
~~~~
Here one can see, that I am setting the `SHELL` environment variable to using
`bash -c`, which will evaluate the actual commands that are defined later in
Makefile targets. `bash` runs in the `guix shell`, that is created in a `guix
time-machine` call, which refers to the `guix-env/channels.scm` file, to make
sure, that I am using exactly the versions defined in that commit of the guix
repository, implying that I am using exactly the hashes defined on that commit
of the guix repository. That should make things reproducible and make it so that
when I run things on another machine, the same software, the exact same packages
are used.
I also define a list of Ocaml packages I am using, currently only one: `zarith`
which is important for big integers.
Then come the actual targets:
~~~~ Makefile (targets) ~~~~
...
.PHONY: repl
repl:
utop
.PHONY: shell
shell:
bash
...
~~~~
The target `shell` is merely for debugging purposes, or for when I need to run a
command in a shell that has the dependencies installed.
When I run `make shell` and in that resulting shell run `ocaml --version`, I get
the output: `The OCaml toplevel, version 5.3.0`. Great so far!
*(Issue 1)*
One problem is with `utop`. Somehow it does not use the correct Ocaml version
and I don't know where it gets another Ocaml version even from. When I run `make
repl` I get a `utop` REPL that says:
~~~~ utop on command line ~~~~
───────────┬───────────────────────────────────────────────────────────────────┬
│ Welcome to utop version %%VERSION%% (using OCaml version 4.14.1)! │
└───────────────────────────────────────────────────────────────────┘
Type #utop_help for help about using utop.
~~~~
What? Why is it using Ocaml version 4.14.1?? Where does it get that from? No
idea. It shouldn't be using anything but the installed packages. Maybe it
internally depends on an older Ocaml version implicitly, and guix installs that,
so that utop can run at all, satisfying utop's dependencies? But on
https://hpc.guix.info/package/ocaml-utop I click the link to
https://codeberg.org/guix/guix/src/commit/23dbcfaef682c41e9cb2a6aea9ad69feea65f26a/gnu/packages/ocaml.scm#L5660,
where I see the inputs:
~~~~ ocaml.scm ~~~~
(native-inputs
(list ocaml-cppo))
(propagated-inputs
(list ocaml-lambda-term
ocaml-logs
ocaml-lwt
ocaml-lwt-react
ocaml-react
ocaml-zed))
~~~~
There I at least don't see any direct dependency on an older Ocaml version.
Might be one of those packages listed there is only available for older Ocaml
version. I don't know.
But this issue is not the main issue. The main issue is with other targets. The
Makefile also has targets to run an Ocaml file:
~~~~ Makefile (targets for running ocaml things) ~~~~
%.byte: %.ml
ocamlfind ocamlc $(foreach OCAML_PACKAGE,$(OCAML_PACKAGES),-package
$(OCAML_PACKAGE)) -linkpkg -o $*.byte $<
# Simply running depends on building the byte files, which
# are to be considered intermediate files.
.PRECIOUS: %.byte
%: %.byte
./[email protected]
~~~~
Here I define a target `%`. This target will be used when I write things like
`make main`. It depends on `%.byte`, which means that `main` depends on
`main.byte`. The other target `%.byte` describes how to build that. It depends
on there being a `%.ml` which in case of `main` would be `main.ml`. It runs the
long command written there, which makes use of `ocamlfind`. The hint to use
`ocamlfind` is the decisive hint someone gave me on this mailing list some time
ago. If there is a better way, please let me know. I need to be able to specify
arbitrary packages, that I install using guix, by writing them in the
`guix-env/manifest.scm` file.
*(Issue 2)*
So lets see what actually happens, when I try to run some `main.ml` file with
some code:
~~~~ main.ml ~~~~
(* In REPL instead use:
#require "zarith";; *)
open Z
~~~~
`make main`:
~~~~ make main result ~~~~
ocamlfind ocamlc -package zarith -linkpkg -o main.byte main.ml
guix shell: checking the environment variables visible from shell '/bin/bash'...
guix shell: All is good! The shell gets correct environment variables.
+ ocamlfind ocamlc -package zarith -linkpkg -o main.byte main.ml
findlib: [WARNING] Package unix has multiple definitions in
/gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/unix/META,
/gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/unix/META,
/gnu/store/yjik976n23235nhkr0amkrymb6kyfkxs-ocaml-findlib-1.9.5/lib/ocaml/site-lib/unix/META
findlib: [WARNING] Package threads has multiple definitions in
/gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/threads/META,
/gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/threads/META,
/gnu/store/yjik976n23235nhkr0amkrymb6kyfkxs-ocaml-findlib-1.9.5/lib/ocaml/site-lib/threads/META
File "main.ml", line 1:
Error:
/gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/zarith/z.cmi
is not a compiled interface for this version of OCaml.
It seems to be for an older version of OCaml.
make: *** [Makefile:23: main.byte] Error 2
~~~~
For starters, the `findlib` warnings seem to be worrisome. Why does findlib ever
search somewhere, where it finds multiple definitions? I don't remember this
issue from this project setup appearing 1 or 2 years ago. This might already be
the cause for the next part of the issue.
~~~~
Error:
/gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/zarith/z.cmi
is not a compiled interface for this version of OCaml.
It seems to be for an older version of OCaml.
~~~~
Well, seems like it is not using the correct package, or Ocaml version specified
in manifest.scm, or the package got compiled using another Ocaml version, or I
don't know what's going on and what is going wrong.
The issue with utop showing another version is already suspect. Maybe somehow
the `zarith` package got compiled using that older version of Ocaml too!
But I don't know how to check that. Or how to stop Guix from doing that. Or how
to nail down the versions in a way that doesn't allow this to happen.
I already tried running `guix gc` to maybe delete old `guix shell`s, but to no
avail.
Here is a repository, which contains the project setup I described:
https://codeberg.org/ZelphirKaltstahl/advent-of-code-2025/src/commit/aa7d26056fc0ff442978d4bdebe615d533e54db4.
Here is my `guix --version` output, in case it is relevant:
~~~~ guix --version output ~~~~
guix (GNU Guix) 7c6d8a6224cf3209efa179dbe1509759a580cb05
Copyright (C) 2025 the Guix authors
License GPLv3+: GNU GPL version 3 or later<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
~~~~
Here is my OS:
~~~~ lsb_release -a ~~~~
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm
~~~~
So my questions are at this point:
(1) How do people make reproducible Ocaml project setups? I don't mean only
version numbers ... It must be hashes and verifying those hashes. Otherwise I
don't think I will pursue things further with Ocaml.
(2) How to fix my guix-using Ocaml project setup, so that I can run my code,
including ocaml-* packages that I install via guix?
I am aware, that I could also create a guix profile. However, I don't think that
will change things and I am fine with a guix shell. All I want is something
reproducible, whether that is a guix profile or a guix shell, doesn't really
matter to me. guix shells are also cached, so it is not like I need to rebuild
the guix shell at every call of my Makefile targets either.
I am at the end of my wits. If anyone can point me to a reproducible project
setup, or tell me what I am doing wrong in my setup, that would be great : )
Best regards,
Zelphir
--
repositories:https://codeberg.org/ZelphirKaltstahl
