On Mon, Dec 13, 2021 at 10:09 AM Lukáš Hrázký <lhra...@redhat.com> wrote: > > On Fri, 2021-12-10 at 13:53 -0500, Chris Murphy wrote:
> > For my non-emphemeral systems: > > > > * /usr /var /etc are directories contained in btrfs subvolume > > "root-x86-64:fedora.35" > > * read-only snapshots have an ISO 8601 datetime format tacked onto the end > > > > * /var/log is a directory contained in subvolume "varlog-x86-64:fedora.35" > > * /var/lib/libvirt/images is a directory contained in subvolume > > "varlibvirtimages-x86-64:fedora.35" > > * no snapshots are ever made of these; by virtue of themselves being > > subvolumes, any snapshot of "root-x86-64:fedora.35" doesn't contain > > /var/log or /var/lib/libvirt/images so just like /home, they only ever > > carry forward in time. > > > > Therefore in my case, snapshot and rollback doesn't affect any logs, > > including the systemd journal. They only ever carry forward in time. > > Same for VM images and for ~/ since that lives in /home and is a > > separate subvolume too by default in Fedora. But both RPM database and > > DNF history do get rolled back in this case. Which works out OK > > because usr/ var/ etc/ are all in one root. A rollback means DNF > > history still accurately describes that root's history and state. > > > > As soon as there's mix and match any of usr/ var/ and etc/ it gets trickier. > > So, is this basically the summary of your motivation for this change? > That rollbacks of your /usr is always in sync with the dnf-stored > system state? Tentative answer is yes. But does dnf-stored system state mean /usr state or /var state or both? And is anything else included? > FWIW we could do with a practical evaluation of impact and issues this > is causing now, are we talking desktops/servers/containers, etc.? Desktops, servers, virtual machines. My thought about containers is that they are self-contained, i.e. /usr and /var are paired together for the life of a container, and there isn't a roll for mixing and matching /usr and /var for a given container. There are technologies that can make this trivially possible so will it happen? *shrug* no idea. But right now if you had to do a rollback, I'd expect you start a new container predicated on a previous state of a container image, i.e. manage container versioning+snapshot+rollback within the container realm, not with the tools inside of it. The idea of using snapper to maintain many container variants inside a container just seems a bit mind boggling to me but I guess it could happen. > > > There are also issues with sqlite creating some lock files even if just > > > reading, which wouldn't be possible on a RO FS (IIRC how it was > > > explained to me), so there are other reasons why the history db will > > > likely not move to /usr. > > > > Detect it and use overlayfs on a volatile /run directory for this use > > case? I'm not sure the use case really calls for a persistent file > > that survives a crash for reading read-only files? > > AFAICT it doesn't require a persistent file. But if you're suggesting > for dnf itself to set up that overlayfs, I may be misjudging, but it > seems way out of bounds of what dnf should be doing. Neither sqlite nor dnf must have this ability, I'm suggesting they could do it. As in, it's not improper. If dnf must create lock files in order to read files on a read-only system, then the choice is (I'm guessing) to either fail or to setup an overlay or even use /tmp in order to not fail. And i'm not suggesting which choice is right or wrong, just that no matter what there's some consequence. Ideally, I'd expect a system should anticipate the read-only /usr requirements, and that its initramfs would have the ability to detect read-only /usr and setup a volatile overlay on /run using overlayfs, similar to a LiveOS boot. In such a case, it's transparent to software. It'll look like a read-write system. It's just that any changes will be lost at reboot because all the changes are stored on a volatile storage. And the UI is either "user expressly chose a rescue boot which implies this kind of volatile environment" or "the environment notifies the user that the environment is on volatile storage". > But the situation may be solvable in other ways, e.g. storing those > files in a writable temporary location. I haven't investigated. Sure. > Tbh. given the ugliness and complexity added by history DB and how it > stores the system state (you seem to agree the transaction log > shouldn't be on /usr, but those two are stored together now) I doubt > we'll resolve this before dnf 5, unless there's a contribution which > neatly ties the loose ends (again, dnf 4 is ugly in a lot of ways...) I'm definitely reluctant to consider conflating logs and state in the same file. Those seem like really different things. A log can't reliably infer state by replaying the log unless that log contains all things that can affect state, including events that dnf doesn't control. That's a tall order. If dnf can affect /home then maybe it calls for separate ~/ state tracking because users can do rollbacks of individual user homes. The ordinary case of rollback is really just restoring a file from a previous state. The extraordinary case is restoring a user's home from backup. We only explore the various technologies for achieving this, in order to deconflict. Not because any one method should be promoted. It doesn't matter if a user's home is restored by rsync, or by a btrfs send/receive of a snapshot. Either way the whole user home has in effect been rolled back to the state it was in when last backedup (and is now restored). So you probably don't want to have a database for /home but rather for each user's /home. > > > > If there's anything in history db that is required for modularity to be > > > functional on a system, it should be moved to the dnf system state (in > > > dnf 5), history db should just be a log and nothing else (at most we > > > could use it to reconstruct the system state if it gets damaged while > > > the history db remains intact). > > > > > > I wonder if the dnf history ought to go in /var/log and be exempt like > > the systemd journal? Or maybe for auditing purposes it should write to > > both its own history file as well as into the systemd journal? > > I guess for auditing you could do with a simple text log of the > transactions. The history DB supports querying and also undo/redo, so > it's a more sophisticated solution. There have been requests for that > simple log (in addition to the history DB) exactly because of > simplicity. But I don't think we'd want to store history DB in > /var/log. > > /var/lib still seems like the arguably best place for it to me. Well... the difficulty is if /var is rolled back but /usr isn't or vice versa. The log in /var/lib will somehow be wrong no matter what in such a case. Maybe worth exploring the question as true or false: dnf doesn't support mix and match /usr /var states, those two directories are assumed to be paired together for life If it's false or you want it to be false, then what needs to be changed so dnf can independently track state changes to them? -- Chris Murphy _______________________________________________ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
