Hi, Late to the party…
>> https://nesbitt.io/2025/12/24/package-managers-keep-using-git-as-a-database.html …I’ve just read this. On Fri, 27 Feb 2026 at 15:24, Ludovic Courtès <[email protected]> wrote: > There are several ways forward I think, from keeping fewer packages in > Guix proper and moving peripheral packages to channels (still maintained > by Guix, but as separate repositories) This is my opinion since years. ;-) I think that many modules as emacs-xyz, bioinformatics, cran, bioconductor, etc. should live in separated channels. However, it’s not as easy as it is when saying this. I mean, it’s difficult to keep synchronized all the channels in order to make them work all together; it’s much easier with this current big Git repository approach. That’s said, I think what we’re currently doing on guix-science and friends channels is an real-world experiment for this multi-channels approach. Somehow, on Guix-Science we’re acquiring feedback by spotting all the concrete flaws and ways to mitigate them. For one example, this idea of guix-science channel release. Well, I don’t know if it’s the good approach and the only way to know is to try. ;-) Based on this, maybe we’ll be able to revisit the question later. > No rush, but we should start thinking about it! Keeping Git as the database backend, one thing I’m thinking is about a dedicated design for the Git packfiles. Not when we are collaborating on the source code itself but for when we are distributing this source code (what live under ~/.cache/guix/checkouts/). Concretely, in short – yes my description’s super imprecise but enough IMHO for discussing – plain Git stores blobs (file), trees (folder) and commits [1]. A commit points to a tree and a tree lists other trees or blobs – hey DAG and Merkle Tree! This implies deduplication (blob/file is stored only once) but the modification of one bit creates two blobs (i.e., two full files are stored although they differ from only one tiny bit). It’s how the Git content-address system works. Then when the history of modifications increases it means all the full files are stored for each modification. When it’d be better to only store the difference: it saves many bytes. That’s the job of Git packfiles [2]. Packfiles are not part of the primary design of Git but somehow an optimization. In short, it builds an index of addresses associated to some offsets and stores the diff between the blobs and trees. The format and the strategy of this packfile might provide some optimization. For instance, Git has two versions [3]. And Git folks are still working on improving [4,5,6]. With this in mind, the questions are: Could we exploit the fact that our code is S-expressions, + for reducing the size when diffing? + for a better compression than the standard one? + for a more adapted compression/packing/indexing? Somehow, this how I’d explore how we could mitigate Guix problems from Git as database that will appear asymptotically*, maybe. Cheers, simon *asymptotically: if the project is still live when they might pop up. :-) 1: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects 2; https://git-scm.com/book/en/v2/Git-Internals-Packfiles 3: https://git-scm.com/docs/pack-format 4: https://youtu.be/-h7TzllaWU0 5: https://youtu.be/f1ewms9yOXk 6: https://youtu.be/ItvGVENGet0
